Category Archives: cookbook

Simple logging with rotation and custom format in Python

Posted on by

Sometimes I find myself having to write a short Python script with proper logging capabilities.

This code snippet demonstrates how to do use Python’s official logging module using very few lines but still with all these features in:

  • basicConfig
  • log rotation
  • formating
import logging
from logging.handlers import RotatingFileHandler
logging.basicConfig(
	handlers=[
		RotatingFileHandler(
			'/path/to/log/file.log',
			maxBytes=10240000,
			backupCount=5
		)
	],
	level=logging.INFO,
	format='%(asctime)s %(levelname)s PID_%(process)d %(message)s'
)
logging.info('Hello world')
logging.error('Oh no, an error occurred!')

The following lines are interesting:

  • ‘/path/to/log/file.log’ -> lets you specify where the log file is
  • maxBytes=10240000 -> maximum size of a log before being rotated
  • backupCount=5 -> how many rotated files to keep
  • format=’%(asctime)s %(levelname)s PID_%(process)d %(message)s’ -> format of each log line; pay attention to the PID, very useful if your script can be invoked simultaneously multiple times, because this will let you split logs from different parallel runs

Quick and dirty fix for VMware Linux guests loosing clock accuracy

Posted on by

I covered on a previous post how to keep the clock synchronized for VMware Linux guest(s). Well this seems to not work at least for recent versions VMware Server 2 (i.e. the one with web based management console). For now the quick& dirty solution I am using is putting a cron job that executes ntpdate pretty often…

My cron job looks like this:

#
# Temporary fix for the time getting lost
#
0-59/10 * * * * /usr/sbin/ntpdate north-america.pool.ntp.org > /dev/null 2>1

Yes, this fix requires to have NTPDATE installed (apt-get install ntpdate under Debian).

How to install MySQL Server on Debian Linux

Posted on by

While installing MySQL Server it is always good to keep in mind that the logs and data folders will potentially have a big size. By default MySQL keeps them in the root mount point (i.e. ‘/’). That may cause your database server system disk to get full, which is never a good idea.

This article describes how to move these two folders to ‘/home’ which is ideally mounted into another disk and has enough space to keep your database data and logs.

First, I install the required apt-get packages as follows:

apt-get update
apt-get install mysql-server

To check the status:

/etc/init.d/mysql status

/usr/bin/mysqladmin  Ver 8.41 Distrib 5.0.51a, for debian-linux-gnu on i486
Copyright (C) 2000-2006 MySQL AB
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license

Server version          5.0.51a-24
Protocol version        10
Connection              Localhost via UNIX socket
UNIX socket             /var/run/mysqld/mysqld.sock
Uptime:                 3 sec

Threads: 1  Questions: 78  Slow queries: 0  Opens: 23  Flush tables: 1
Open tables: 17  Queries per second avg: 26.000.

Now, stop MySQL, move the folders to the right location, reconfigure MySQL and start again:

# Stop MySQL
/etc/init.d/mysql stop

# Move and reconfigure data
mkdir /home/mysql
mv /var/lib/mysql /home/mysql/mysql-data
ln -s /home/mysql/mysql-data/ /var/lib/mysql

# Move and reconfigure logs
mv /var/log/mysql/ /home/mysql/mysql-logs
ln -s /home/mysql/mysql-logs/ /var/log/mysql

# Start MySQL and check that everything is OK
/etc/init.d/mysql start
/etc/init.d/mysql status
/usr/bin/mysqladmin  Ver 8.41 Distrib 5.0.51a, for debian-linux-gnu on i486
Copyright (C) 2000-2006 MySQL AB
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license

Server version          5.0.51a-24
Protocol version        10
Connection              Localhost via UNIX socket
UNIX socket             /var/run/mysqld/mysqld.sock
Uptime:                 14 sec

Threads: 1  Questions: 78  Slow queries: 0  Opens: 23  Flush tables: 1
Open tables: 17  Queries per second avg: 5.571.

These are some settings that I usually put on the /etc/mysql/my.cnf configuration file:

# Here you can see queries with especially long duration
log_slow_queries        = /var/log/mysql/mysql-slow.log
long_query_time         = 1
log-queries-not-using-indexes

# A server-id unique
server-id                = 177
log-bin                  = /var/log/mysql/mysql-bin.log
log-bin-index            = /var/log/mysql/mysql-bin.log
innodb_file_per_table
# Unique log names (this prevents replication breaking upon hostname change :-)
relay-log                = iamalsounique98127-relay-bin
relay-log-index          = iamalsounique98127-relay-bin

# Taking care of the auto-increment values (for multi-master replication)
auto_increment_increment      = 10
auto_increment_offset         = 1

For these changes to take effect, you would need to restart MySQL:

/etc/init.d/mysql restart

If you want to ignore databases or tables you may use the following options:

binlog_ignore_db        = information_schema
replicate_ignore_db     = information_schema
binlog_ignore_db        = mysql
replicate_ignore_db     = mysql

# Ignore all the cache* tables which have caused DUPLICATE
# ENTRY issues. Unai.
replicate_wild_ignore_table = exampledb.cache%

Having ‘binlog_ignore_db’ is enough to exclude databases from replication BUT having ‘replicate_ignore_db’ as well will make things clearer since the databases that are being ignored will appear in both the ‘SHOW SLAVE STATUSG’ and ‘SHOW MASTER STATUSG’.

Granting MySQL Permissions

Posted on by

Usage

mysql> grant PERMISSIONS on DBNAME.* TO 'USERNAME'[@'SERVER'] 
    -> identified by 'PASSWORD' [with grant option];
mysql> flush privileges;

Examples

mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER,INDEX on exampledb.* 
    -> TO 'exampleuser'@'10.10.10.10' IDENTIFIED BY 'secret';
Query OK, 0 rows affected (0.07 sec)
mysql> GRANT ALL PRIVILEGES on exampledb.* TO 'exampleuser'@'10.10.10.10' 
    -> IDENTIFIED BY 'secret' WITH GRANT OPTION;
Query OK, 0 rows affected (0.06 sec)

Notes

  • WITH GRANT OPTION should only be added when really needed; this privilege allows a user to grant to others (more info).
  • The location from where a user can connect (i.e. ‘username’@’server’) can be also a network if specified as  ‘username’@’10.120.%.%’

See Also