Waiting for 9.4 – Add new wal_level, logical, sufficient for logical decoding.

On 11th of December, Robert Haas committed patch:

Add new wal_level, logical, sufficient for logical decoding.
 
When wal_level=logical, we'll log columns from the old tuple as
configured by the REPLICA IDENTITY facility added in commit
<a class="text" href="/gitweb/?p=postgresql.git;a=object;h=07cacba983ef79be4a84fcd0e0ca3b5fcb85dd65">07cacba983ef79be4a84fcd0e0ca3b5fcb85dd65</a>.  This makes it possible
a properly-configured logical replication solution to correctly
follow table updates even if they change the chosen key columns,
or, with REPLICA IDENTITY FULL, even if the table has no key at
all.  Note that updates which do not modify the replica identity
column won't log anything extra, making the choice of a good key
(i.e. one that will rarely be changed) important to performance
when wal_level=logical is configured.
 
Each insert, update, or delete to a catalog table will also log
the CMIN and/or CMAX values of stamped by the current transaction.
This is necessary because logical decoding will require access to
historical snapshots of the catalog in order to decode some data
types, and the CMIN/CMAX values that we may need in order to judge
row visibility may have been overwritten by the time we need them.
 
Andres Freund, reviewed in various versions by myself, Heikki
Linnakangas, KONDO Mitsumasa, and many others.

One and a half year ago Andres Freund sent mail to pgsql-hackers, with his proposition for new feature in PostgreSQL: Logical Replication.

Since then there have been MANY mails, discussion, and countless hours of his (and others) work.

The original features that were suggested were:

  • in core
  • fast
  • async
  • robust
  • multi-master
  • modular
  • as unintrusive as possible implementation wise
  • basis for other technologies (sharding, replication into other DBMSs, …)

The goal seemed to be great, but far away.

To be honest – we're not there yet. The discussion is ~ 400 mails long. The patch itself is currently in version 6!

But it's coming.

I'm not going to show you anything cool about this newly committed patch, mostly because I don't think that it can be used to anything usable at the moment.

But I wanted to write about it, because it's clearly visible piece of the big thing. And the big thing is perhaps slow to arrive, but it's getting closer.

With logical, multi-master, in-core replication, we will really be able to say “ok, topic of replication is closed".

I would like to express my thanks to Andres, Heikki, KONDO, and every other person that worked on this.

3 thoughts on “Waiting for 9.4 – Add new wal_level, logical, sufficient for logical decoding.”

  1. Great. I have used MySQL’s replication and gone through many issues and resolutions through versions (how stored procedure execution is replicated, how to handle temp tables, handling variables used in stored procedures and so on). I am sure PostgreSQL team will do a better job the first version itself..

  2. I WANT THIS SO BAD.

    What can I do to help make this a reality? I’m not much of a C programmer but if there’s any other way I can contribute…

    @angrynoah on twitter

  3. @Noah: ask on pgsql-hackers. They will definitely be happy for any help – proofreading, testing.

Comments are closed.