Yesterday, I did release new version of OmniPITR – 1.1.0.

You can get it from github or from PgXN.

What's new? Couple of things:

• Fix: omnipitr-monitor had a bug with parsing timestamps from logs (in cases of “negative" timezones)
• Fix: sanity-check.sh now returns correct status when exiting, so you can verify if everything is OK by simply checking $?, and not reading its output (helpful for automation scripts)
• Fix: temp directories are made using PID in path, so you can now have (for example) two separate PG clusters on the same machine, each running omnipitr-restore, without them stepping on each others toes
• New: added –version to all programs, so you can easily check which version you have installed
• New: added –help option to all programs, so you can see the documentation without manually finding docs, and running perldoc on them

All of above are pretty cosmetic, but there are two bigger changes too. First of them – –remove-before option to omnipitr-restore. Let's imagine you have normal replication, but for whatever reason slave is lagging. Maybe Pg on slave machine was disabled, or maybe master is just so much faster. In any way – slave is lagging. And this means that walarchive is growing (I'm talking about xlog-file based replication, not streaming).

Walarchive would be growing, because omnipitr-archive delivers new xlogs, but omnipitr-restore will remove them only when it will catchup. Now, with this new option, omnipitr-restore always first checks if there is anything to delete, and if yes – removes single “batch" of files (according to –remove-at-a-time option).

This is not enabled by default, mostly not to change behaviour from previous versions, but also – because “remove-before" is slightly slower – before providing file, it has to run pg_controldata, and optionally remove some (3 by default) files.

The overhead is (in my opinion) minimal, but it is there, so it can't be enabled by default. But – if you're ever in position where walarchive grew too much, and slave was lagging – this is an option for you.

The other larger news is ability to provide config files. This has been added to all programs, but I think it's most important for omnipitr-restore (where changing options required restart of slave because options were in recovery.conf file read only on Pg start).

The file has very simple format:

• one option per line
• options are given just like in command line, with the exception that you don't need to quote values
• both short and long version of switches are acceptable (-v/–verbose)
• empty lines are ignored
• lines starting with # are treated as comments

This means that instead of having (for example) in your postgresql.conf:

archive_command = '/opt/OMNIpitr/bin/omnipitr-archive -PJ 5 -s /var/lib/pgsql/omnipitr/state -l /var/lib/pgsql/data/pg_log/omnipitr-^Y-^m-^d.log -v -dr rsync://pg-slave/xlogs/ -dl /mnt/nas/walarchive/ -dr rsync://backup/dba/pg/xlogs/ "%p"'

you can have it now:

archive_command = '/opt/OMNIpitr/bin/omnipitr-archive --config-file=/var/lib/pgsql/omnipitr/archive.conf "%p"'

and in /var/lib/pgsql/omnipitr/archive.conf, you'd put:

# Logging
--log=/var/lib/pgsql/data/pg_log/omnipitr-^Y-^m-^d.log
--verbose
 
# State - needed because we have more than 1 destination:
--state-dir=/var/lib/pgsql/omnipitr/state
 
# Destinations
--dst-local=/mnt/nas/walarchive/
--dst-remote=rsync://pg-slave/xlogs/
--dst-remote=rsync://backup/dba/pg/xlogs/
 
# Speed up deliveries by running them concurrently
--parallel-jobs=5

Also – since config file is read on each run of omnipitr-* program, changes in the file work immediately, and do not require restart of PostgreSQL (nor reload).

Hope you'll find it useful.