Tag: replication

OmniPITR 0.6.0

Just released new version, 0.6.0 (it should be visible on pgxn soon) of OmniPITR set of tools.

New version has one new feature – parallelism.

This works in omnipitr-archive and omnipitr-backup-* programs, and allows for parallel delivery to remote destinations (multiple -dr switches).

Also – if you're using compresses wal archive and omnipitr-backup-slave reading from it – all the wal files have to be decompressed before making backup – and this decompression can be parallelized too.

All parallelization is controled using -PJ option (–parallel-jobs), so you can add “-PJ 10" to get up to 10 decompressions at the same time or up to 10 deliveries at the same time.

Waiting for 9.2 – pg_basebackup from slave

On 25th of January, Simon Riggs committed patch:

Allow pg_basebackup from standby node with safety checking.
Base backup follows recommended procedure, plus goes to great
lengths to ensure that partial page writes are avoided.
 
Jun Ishizuka and Fujii Masao, with minor modifications

Continue reading Waiting for 9.2 – pg_basebackup from slave

OmniPITR 0.3.0

Just released version 0.3.0 of our tool for handling WAL based replication in PostgreSQL – OmniPITR.

Version jump is related to addition of another tool – omnipitr-synch. This tool is used to copy PostgreSQL data dir (including all tablespaces of course) to remote location(s).

While this process is usually simple (call pg_start_backup(), transfer data, call pg_stop_backup()), thanks to the tool it can be wrapped as single call, with standardized logging, and tested logic. It also makes it trivial, and cheap, to setup more than one new slave at a time, without need to read data off master more than once.

Waiting for 9.2 – cascading streaming replication

On 19th of July, Simon Riggs committed patch:

Cascading replication feature FOR streaming log-based replication.
Standby servers can now have WALSender processes, which can WORK WITH
either WALReceiver OR archive_commands TO pass DATA. Fully updated
docs, including NEW conceptual terms OF sending server, upstream AND
downstream servers. WALSenders TERMINATED WHEN promote TO master.
 
Fujii Masao, review, rework AND doc rewrite BY Simon Riggs

Continue reading Waiting for 9.2 – cascading streaming replication

Waiting for 9.1 – Synchronous replication

On 6th of March, Simon Riggs committed patch:

Efficient transaction-controlled synchronous replication.
If a standby is broadcasting reply messages and we have named
one or more standbys in synchronous_standby_names then allow
users who set synchronous_replication to wait for commit, which
then provides strict data integrity guarantees. Design avoids
sending and receiving transaction state information so minimises
bookkeeping overheads. We synchronize with the highest priority
standby that is connected and ready to synchronize. Other standbys
can be defined to takeover in case of standby failure.
 
This version has very strict behaviour; more relaxed options
may be added at a later date.
 
Simon Riggs and Fujii Masao, with reviews by Yeb Havinga, Jaime
Casanova, Heikki Linnakangas and Robert Haas, plus the assistance
of many other design reviewers.

Continue reading Waiting for 9.1 – Synchronous replication

OmniPITR – update

As of yesterday OmniPITR got following changes/fixes:

  1. Fixed bug which caused immediate finish request be treated the same as smart finish request.
  2. Fixed problem with using omnipitr-backup-slave on PostgreSQL 9.0 slave, which is using streaming replication.
  3. Added option to omnipitr-restore, so that you can now use it for streaming-replication slaves

I'm very ashamed of the first thing (smart/immediate finish request), as it was simply my lack of test.

Second thing – the problem was that with streaming replication slave behaves a bit differently than normally, and so the backup procedure had to be modified.

As for third – restore command in streaming replication environment has to behave differently than normal restore command – i.e. it should finish, with error, as soon as it will be called for wal file that does not exist in wal archive. Which is direct opposite of what should be done normally.

Now, omnipitr-restore got switch (-sr) to make it work correctly in SR situation.

Links for svn/docs are listed on project page.

Waiting for 9.1 – pg_basebackup

On 23rd of January, Magnus Hagander committed patch which adds:

Add pg_basebackup tool for streaming base backups
 
This tool makes it possible to do the pg_start_backup/
copy files/pg_stop_backup step in a single command.
 
There are still some steps to be done before this is a
complete backup solution, such as the ability to stream
the required WAL logs, but it's still usable, and
could do with some buildfarm coverage.
 
In passing, make the checkpoint request optionally
fast instead of hardcoding it.
 
Magnus Hagander, reviewed by Fujii Masao and Dimitri Fontaine

Continue reading Waiting for 9.1 – pg_basebackup