46.6. Logical Decoding Output Plugins
An example output plugin can be found in the contrib/test_decoding subdirectory of the PostgreSQL source tree.
46.6.1. Initialization Function
An output plugin is loaded by dynamically loading a shared
library with the output plugin's name as the library basename.
The normal library search path is used to locate the library.
To provide the required output plugin callbacks and to indicate
that the library is actually an output plugin it needs to
provide a function named _PG_output_plugin_init
. This function is
passed a struct that needs to be filled with the callback
function pointers for individual actions.
typedef struct OutputPluginCallbacks { LogicalDecodeStartupCB startup_cb; LogicalDecodeBeginCB begin_cb; LogicalDecodeChangeCB change_cb; LogicalDecodeCommitCB commit_cb; LogicalDecodeShutdownCB shutdown_cb; } OutputPluginCallbacks; typedef void (*LogicalOutputPluginInit)(struct OutputPluginCallbacks *cb);
The begin_cb
, change_cb
and commit_cb
callbacks are required, while
startup_cb
and shutdown_cb
are optional.
46.6.2. Capabilities
To decode, format and output changes, output plugins can use most of the backend's normal infrastructure, including calling output functions. Read only access to relations is permitted as long as only relations are accessed that either have been created by initdb in the pg_catalog schema, or have been marked as user provided catalog tables using
ALTER TABLE user_catalog_table SET (user_catalog_table = true); CREATE TABLE another_catalog_table(data text) WITH (user_catalog_table = true);
Any actions leading to xid assignment are prohibited. That, among others, includes writing to tables, performing DDL changes and calling txid_current().
46.6.3. Output Plugin Callbacks
An output plugin gets notified about changes that are happening via various callbacks it needs to provide.
Concurrent transactions are decoded in commit order and only changes belonging to a specific transaction are decoded inbetween the begin and commit callbacks. Transactions that were rolled back explicitly or implicitly never get decoded. Successful SAVEPOINTs are folded into the transaction containing them in the order they were executed within that transaction.
Note: Only transactions that have already safely been flushed to disk will be decoded. That can lead to a COMMIT not immediately being decoded in a directly following pg_logical_slot_get_changes() when synchronous_commit is set to off.
46.6.3.1. Startup Callback
The optional startup_cb
callback is called whenever a replication slot is created or
asked to stream changes, independent of the number of changes
that are ready to be put out.
typedef void (*LogicalDecodeStartupCB) ( struct LogicalDecodingContext *ctx, OutputPluginOptions *options, bool is_init );
The is_init parameter will be true when the replication slot is being created and false otherwise. options points to a struct of options that output plugins can set:
typedef struct OutputPluginOptions { OutputPluginOutputType output_type; } OutputPluginOptions;
output_type has to either be set to OUTPUT_PLUGIN_TEXTUAL_OUTPUT or OUTPUT_PLUGIN_BINARY_OUTPUT.
The startup callback should validate the options present in ctx->output_plugin_options. If the output plugin needs to have a state, it can use ctx->output_plugin_private to store it.
46.6.3.2. Shutdown Callback
The optional shutdown_cb
callback is called whenever a formerly active replication
slot is not used anymore and can be used to deallocate
resources private to the output plugin. The slot isn't
necessarily being dropped, streaming is just being
stopped.
typedef void (*LogicalDecodeShutdownCB) ( struct LogicalDecodingContext *ctx );
46.6.3.3. Transaction Begin Callback
The required begin_cb
callback is called whenever a start of a commited transaction
has been decoded. Aborted transactions and their contents
never get decoded.
typedef void (*LogicalDecodeBeginCB) ( struct LogicalDecodingContext *, ReorderBufferTXN *txn );
The txn parameter contains meta information about the transaction, like the timestamp at which it has been committed and its XID.
46.6.3.4. Transaction End Callback
The required commit_cb
callback is called whenever a transaction commit has been
decoded. The change_cb
callbacks for all modified rows will have been called before
this, if there have been any modified rows.
typedef void (*LogicalDecodeCommitCB) ( struct LogicalDecodingContext *, ReorderBufferTXN *txn );
46.6.3.5. Callback called for each individual change in a transaction
The required change_cb
callback is called for every individual row modification
inside a transaction, may it be an INSERT, UPDATE or
DELETE. Even if the original command
modified several rows at once the callback will be called
indvidually for each row.
typedef void (*LogicalDecodeChangeCB) ( struct LogicalDecodingContext *ctx, ReorderBufferTXN *txn, Relation relation, ReorderBufferChange *change );
The ctx and txn parameters have the same contents as for
the begin_cb
and commit_cb
callbacks, but additionally the
relation descriptor relation
points to the relation the row belongs to and a struct
change describing the row
modification are passed in.
Note: Only changes in user defined tables that are not unlogged (see UNLOGGED) and not temporary (see TEMPORARY or TEMP) can be extracted using logical decoding.
46.6.4. Functions for producing output from an output plugin
To actually produce output, output plugins can write data to
the StringInfo output buffer in
ctx->out when inside the
begin_cb
, commit_cb
or change_cb
callbacks. Before writing to the
output buffer OutputPluginPrepareWrite(ctx, last_write)
has
to be called, and after finishing writing to the buffer
OutputPluginWrite(ctx,
last_write)
has to be called to perform the write. The
last_write indicates whether a
particular write was the callback's last write.
The following example shows how to output data to the consumer of an output plugin:
OutputPluginPrepareWrite(ctx, true); appendStringInfo(ctx->out, "BEGIN %u", txn->xid); OutputPluginWrite(ctx, true);