Waiting for 9.3 – Dramatically reduce System V shared memory consumption.

On 28th of June, Robert Haas committed patch:

Dramatically reduce System V shared memory consumption.
 
Except when compiling with EXEC_BACKEND, we'll now allocate only a tiny
amount of System V shared memory (as an interlock to protect the data
directory) and allocate the rest as anonymous shared memory via mmap.
This will hopefully spare most users the hassle of adjusting operating
system parameters before being able to start PostgreSQL with a
reasonable value for shared_buffers.
 
There are a bunch of documentation updates needed here, and we might
need to adjust some of the HINT messages related to shared memory as
well.  But it's not 100% clear how portable this is, so before we
write the documentation, let's give it a spin on the buildfarm and
see what turns red.

Continue reading Waiting for 9.3 – Dramatically reduce System V shared memory consumption.

Changes on explain.depesz.com

One of the features that is actually disliked is anonymization. But, regardless of the dislike – it has some users. And one of the user mailed me with information about a bug – namely – foreign table file names were not anonymized.

So, I wrote a patch, tests, released new version of underlying parsing library.

Continue reading Changes on explain.depesz.com

OmniPITR 1.0.0 released

Finally, after all these years, version 1.0.0 of OmniPITR got Released.

The reason I went to 1.0.0, and not 0.8.0 is very simple – finally, all programs in bin/ actually work 🙂

By that I mean: since beginning there was “omnipitr-monitor" – which simply didn't work, because work on it was always postponed. But now, it does. It's functionality is not all that great now, but it works, checks some basic data about replication, and can be used in production.

Now, there is still a todo but these things are less important.

I have to say that writing, and maintaining OmniPITR taught me a lot about PostgreSQL – how it works, and what really WAL is. It was really cool.

How to get shortest connection between two cities

Yesterday, on #postgresql on irc some guy asked:

22:28 < rafasc> i am trying to use plpgsql to find the shortest path between two cities, each pair of cities has one or more edges, each edge has a different wheight.
22:28 < rafasc> Is there a easy way to compute the shortest path between two cities?

Well, I was not really in a mood to solve it, so I just told him to try with recursive queries, and went on my way.

But I thought about it. And decided to see if I can write the query.

Continue reading How to get shortest connection between two cities

How to send mail from database?

Similar question has been asked many times on mailing lists and on IRC. Sometimes it's not mail sending, but file/directory creation, or something else that generally requires some interaction with “world outside of database".

Can it be done? Sure. How, then?

Continue reading How to send mail from database?

How much RAM is PostgreSQL using?

(disclaimer: all the data and examples in here are on Linux – the same data can be probably obtained on other systems too, it's just that I work on Linux and don't know other systems well).

This question pops occasionally in various places – PostgreSQL is using too much memory, why is that, and how can it be mitigated?

Before we can go to “optimizing", we should understand the problem. But do we? Both standard tools – ps and top – lie. How/why? Let's see.

Continue reading How much RAM is PostgreSQL using?

“= 123” vs. “= ‘depesz'”. What is faster?

There is this idea that normal form in databases require you to use integer, auto incrementing, primary keys.

The idea was discussed by many people, I will just point you to series of three blog posts on the subject by Josh Berkus ( part 1, 2 and 3, and reprise).

One of the points that proponents of surrogate keys (i.e. those based on integer and sequences) raise is that comparing integers is faster than comparing texts. So,

SELECT * FROM users WHERE id = 123

is faster than

SELECT * FROM users WHERE username = 'depesz'

Is it?

Continue reading “= 123" vs. “= ‘depesz'". What is faster?

OmniPITR 0.7.0

Just released new version of OmniPITR.

This version has one important new feature: when you're calling omnipitr-backup-slave, it will make backups only of required xlog files, and not, as previously, of all in walarchive directory.

This is important, especially in case you have multiple slaves, or you keep shared long-term walarchive. Previously – backups would get all files from walarchive (-s option to omnipitr-backup-slave), but now, it picks just the ones that are needed.

On somehow related note – I will be working now, finally, to get omnipitr-monitor functionality working.