Flex Your Tungsten Replicator Part 2: Another New Role

Rock and Role!

In this blog post, we explore how the new Tungsten Replicator thl-applier role allows a Replica node to apply THL even when the upstream source Replicator is OFFLINE. This new behavior provides for better control and performance under a variety of conditions.

REVIEW — Existing Tungsten Replicator Roles: Primary & Replica

As we know, the core of operations for Tungsten Replication are the extract and apply functions. The other key operation is the transfer of THL from extractor to applier via the network.

Primary (Master)

• Summary: a database node in the master role performs the extraction and thl-server operations when online.
• A Tungsten Replicator in the master role (aliases: primary or extractor) extracts data from a source database.
• The extractor reads the binary log from a MySQL server and generates THL on disk.
• The extractor uses two stages, binlog-to-q and q-to-thl.
• The Primary replicator also serves THL to other replicators over a network connection (thl-server).
• During normal operations, a master role Replicator will have both an extractor and a thl-server running.

Replica (Slave)

• Summary: the slave role when online performs the thl-client and applier operations.
• A Tungsten Replicator in the slave role (aliases: replica, secondary or applier) pulls MySQL events in the form of THL data from an upstream THL server (thl-client) and writes them to disk.
• The applier then reads those MySQL events from the local, on-disk copy of THL and applies them to a target database using JDBC.
• The applier uses three stages, remote-to-thl, thl-to-q and q-to-dbms.
• During normal operations, a slave role Replicator will have both a thl-client and an applier running.

For more information about Replicator stages, please see this blog post.

REVIEW — Additional Tungsten Replicator Roles: thl-server & thl-client

Additional roles have previously been added to the Tungsten Replicator to make the system more flexible.

thl-server

• The thl-server role is used on Primaries to temporarily enable the THL server when a Primary is taken offline.
• Normally, when a Primary is taken offline, both the extractor and the thl-server operations are stopped. There may be times when we wish to stop extracting, but continue to serve THL to downstream clients.
• This will allow downstream Secondary replicators to download and apply any TH that hasn't yet been processed by that Applier even though extraction on the Primary is stopped.
• The thl-server role enables the actual listener port and server operations to handle downstream THL requests.

thl-client

• The thl-client role is used on a Replica to temporarily enable the THL client when the replicator is taken offline.
• Normally, when a Replica is taken offline, both the thl-client and applier operations are stopped. There may be times when we wish to continue downloading THL without applying it to the target database.
• This will allow the Secondary replicator to download any THL available from the upstream THL server, but does NOT apply the THL to the target database.
• The thl-client role enables the remote-to-thl stage on the Replica.

NEW Tungsten Replicator Role: thl-applier

A new role has been added to the Tungsten Replicator to make it possible to get the Replica to apply-only, which can be very useful when the upstream Primary is offline for whatever reason.

thl-applier

• The thl-applier role is used on a Replica to temporarily enable the database applier when the replicator is taken offline.
• Normally, when a Replica is taken offline, both the thl-client and applier operations are stopped. There may be times when we wish to continue applying downloaded THL to the target database without continuing to download THL from an upstream source THL server.
• This will allow the Secondary replicator to take already-downloaded THL on disk and apply it to the target database, allowing the Replica to catch up as far as possible.
• The thl-applier role enables the thl-to-q and q-to-dbms stages on the Replica.

Using the New Role: thl-applier

Used on a Secondary node, the thl-applier role allows the Replicator to continue applying already-downloaded THL to the local MySQL server even when it has downloading events from an upstream THL server.

To enable the thl-applier role, issue the following statements in MAINTENANCE mode:

shell> trepctl offline
shell> trepctl setrole -role thl-applier
shell> trepctl online

To disable the thl-applier role and revert to full replica operations, issue the following statements in MAINTENANCE mode:

shell> trepctl offline
shell> trepctl setrole -role slave
shell> trepctl online

New Tungsten Replicator Role: Wrap-Up

In this blog post, we explored how the new Tungsten Replicator role thl-applier allows for already-downloaded THL events to be applied to the target database even when not actively downloading THL from an upstream THL server. This new behavior provides for better control and performance under a variety of conditions.