Just go through any directory for a PostgreSQL database, and you will find a pg_internal.init file located within global/, and base/ directories. The file has an interesting life cycle; the startup process throws away the file, and the first connection for the database backend reconnects the file and alter table commands, as well as other DDLs, can disconnect the file again. The file does not appear during backups, and there is no loss at all for its disappearance, except for a few milliseconds.
Now, what is the reason behind this strange life of the file? The reason is that the file solves the problem for every backend that needs to execute its first query.
In order to be able to read the table describing the index, one needs an index.
All data describing tables in PostgreSQL is stored in system catalogs: pg_class - for relations, pg_attribute - for attributes, pg_index - for indexes, etc. The system catalogs are regular tables that have indexes in turn to speed up lookups.
It works well while the server is running. However, there is a problem at startup time. Let's say we have a new backend that needs to get an answer on a very basic question: "Which columns does table X have?" In order to find this information, the backend needs to query the pg_attribute table, and in order to do that efficiently, it needs an index of the pg_attribute table. In order to use this index, it needs to know the in-memory descriptors of the pg_attribute table and its indexes, but in order to construct them, it needs to query pg_class, pg_attribute, and pg_index, etc which was the goal in the first place.
How PostgreSQL breaks the cycle
There are two ways, and PostgreSQL utilizes both depending on the time.
The first one is called the forceful way. In case there are no indexes to use, which might be true during the initdb stage, we will do simple sequential scans. Not very effective, but it does not depend on any indexes existing in the system. This way is also used whenever there is no cache available. Critical descriptors are created by a routine called formrdesc and also through the loading of indexes, without which the system cannot work.
The second way is the fast path and is done by pg_internal.init. After paying the price of generating those descriptors using the hard way, they get serialized on the disk in the layout that the relcache is expecting. The next backend does not need to build anything; it only has to read the pre-generated descriptors from the disk.
Note: the file contains more than the minimal set. Those who are necessary so-called nailed relations, namely, the relations which are so important that they are permanently cached in the memory. However, PostgreSQL takes it even further and caches all catalogs and indexes accessed by the catalogs.
Two files, not one
There are actually two files that sit in two different places, which matters for a multi-database cluster:
- The shared file, placed at global/pg_internal.init, holds descriptors for catalogs that are shared across the whole cluster, things like pg_database and pg_authid. A backend loads this before it has even picked a database.
- The local file, one per database at base//pg_internal.init holds that database's own non-shared catalogs. For example, base/3516849/pg_internal.init.
The split shows up directly in the code that builds the path; shared picks the global/ prefix, otherwise it uses the current DatabasePath.
You can find this code block in src/backend/utils/cache/relcache.c:
if (shared)
snprintf(initfilename, sizeof(initfilename), "global/%s",
RELCACHE_INIT_FILENAME);
else
snprintf(initfilename, sizeof(initfilename), "%s/%s",
DatabasePath, RELCACHE_INIT_FILENAME);
Writing the pg_internal.init file
The write happens in the write_relcache_init_file function (inside relcache.c). It does a little more than a plain cache dump:
- Warms up first. Just before writing, the server calls InitCatalogCachePhase2() to finish loading all the catalog descriptors worth saving, so the snapshot it takes is as complete as possible.
if (needNewCacheFile)
{
InitCatalogCachePhase2();
/* now write the files */
write_relcache_init_file(true);
write_relcache_init_file(false);
}
- Writes atomically. It writes to a temporary name and then renames it into place, so no other backend can ever read a half-finished file.
if (shared)
{
snprintf(tempfilename, sizeof(tempfilename), "global/%s.%d",
RELCACHE_INIT_FILENAME, MyProcPid);
snprintf(finalfilename, sizeof(finalfilename), "global/%s",
RELCACHE_INIT_FILENAME);
}
else
{
snprintf(tempfilename, sizeof(tempfilename), "%s/%s.%d",
DatabasePath, RELCACHE_INIT_FILENAME, MyProcPid);
snprintf(finalfilename, sizeof(finalfilename), "%s/%s",
DatabasePath, RELCACHE_INIT_FILENAME);
}
- Stamps a version number. The file starts with a magic number that changes whenever the relcache layout changes, so a file from a different build is rejected instead of misread.
magic = RELCACHE_INIT_FILEMAGIC;
if (fwrite(&magic, 1, sizeof(magic), fp) != sizeof(magic))
ereport(FATAL,
errcode_for_file_access(),
errmsg_internal("could not write init file: %m"));
- Filters what goes in. Shared catalogs go to the shared file, local ones to the local file. A relation is only cached if a later change to it would also remove the file, you'd end up with a stale entry nobody cleans up.
if (!shared && !RelationIdIsInInitFile(RelationGetRelid(rel)))
{
Assert(!rel >rd_isnailed);
continue;
}
- Skips a doubtful write. If the catalog changed while this backend was starting, it doesn't write the file at all.
if (relcacheInvalsReceived != 0L)
return;
Process-specific data, like function pointers, is never written; the loader rebuilds it instead.
Reading the pg_internal.init file
Loading is performed using the load_relcache_init_file() function. It tries to open the file, but if it doesn’t exist, it just returns false and no error – this is a perfectly fine cold-start scenario. Then it checks the magic number and rebuilds each Relation structure back to CacheMemoryContext, repining the nailed ones.
Loader also takes all precautions while closing. Having checked every entry, it confirms whether the number of nailed relations and indexes it read was correct according to expectations. If something went wrong with the count or reading, the whole file will be considered broken, and false will be returned. Back-end will fallback to sequential scan build path. The corrupt cache file will never hang up the startup process; the most it can do is cost you one slow backend launch while it gets rebuilt
The Invalidation Process:
A cache of the system catalog can be safely invalidated only when it is invalidated exactly when the catalog changes.
Just like the shared invalidation (SI) mechanism flushes in-memory entries of the relcache and catcache, the SI process also removes the init file from the disk if there is any modification made to the relation in the commit phase of the transaction.
- RelationCacheInitFilePreInvalidate() takes an exclusive RelCacheInitLock and unlinks the file.
- Then the callers send SI messages.
- RelationCacheInitFilePostInvalidate() will release the lock then.
Here, the ordering is actually performing a job. Unlinking takes place before invalidation messages get sent, and this locking serializes the whole process for write_relcache_init_file. Combining both these facts prevents a race condition where an existing backend cannot place an old snapshot after unlinking its predecessor, and another backend that has started cannot read the outdated file and yet misses invalidation messages that could have saved it. Since the startup process takes care of registering invalidation in the shared array before attempting to read the file, it is airtight.
pg_internal.init holds almost all of the rules needed to be followed for cache-derived states despite the small size of the file. The file is never read in an incomplete form because it gets written under a temporary name before renaming. Stale file format is rejected and not misinterpreted due to the presence of a version stamp. Its deletion uses the same invalidation path as that of memory caches, and the on-disk unlink gets scheduled before sending out a broadcast message that keeps any backend from booting up with the old state. Now that the problem of catalog bootstrap has been solved, the solution recorded, and the backends can skip over the hard part.