Lock It Down and Tighten It Up: THL Encryption and Compression Comes With Tungsten Replicator v7

Introduction

Performance tuning any system provides more speed for the same hardware spend, gives the end-user a better, faster experience and typically reduces the stress on staff all around.

Security tuning locks down critical data to prevent unauthorized access.

Today we will explore both the security and performance enhancements available for the Tungsten Replicator THL sub-system as of version 7.x.

THL files store MySQL extracted binary log events in a normalized format for consumption by downstream Tungsten Replicator appliers running on Replica nodes.

As of Tungsten Clustering v7.0.0, you may now enable both on-disk THL encryption and on-disk THL compression. Even better, THL in-flight compression is now available too!

This blog post will walk you through the feature and how to enable, configure and disable Tungsten Replicator THL Compression and Encryption.

The Brief

Encryption is applied to THL on disk, in-flight encryption is handled by enabling the various SSL features of the Replicator.

Compression can be enabled in-flight by changing the various configuration properties, and Compression on disk can be enabled/disabled either dynamically or by changing the various configuration properties.

The following sections explain enabling/disabling these features in more detail.

Getting Started with On-Disk THL Encryption and Compression

By default, both on-disk THL encryption and on-disk THL compression are disabled.

You have the option of individually configuring on-disk THL encryption and on-disk THL compression via the tpm tool options, or dynamically using either the trepctl command, or the APIv2. We will get into all of those ways here.

Configure On-Disk THL Features At Install Time

Here are the tpm options and values needed to configure on-disk THL features:

• replicator-store-thl-encrypted=[true|false]
• replicator-store-thl-compressed=[true|false]

You would add these to your tungsten.ini file, or if you are using the Staging deployment method, just add two hyphens in front of each option for use with `tpm configure`, then either install or update via tpm.

Configure On-Disk THL Features Dynamically

To change these settings dynamically, all desired services must be first taken offline. This forces the Replicator to create a new THL log file for each service that uses the new settings.

These settings will persist through a replicator restart, even if enabled or disabled dynamically.

Via the Command Line

The commands to enable or disable these settings are:

shell> trepctl [-service servicename] thl -compression {enable|disable}
shell> trepctl [-service servicename] thl -encryption {enable|disable}

Here is a full session example of how to enable both on-disk encryption and compression for a single service, alpha:

shell> cctrl
cctrl> set policy maintenance
cctrl> exit

shell> trepctl -service alpha offline
shell> trepctl -service alpha thl -encryption enable
shell> trepctl -service alpha thl -compression enable
shell> trepctl -service alpha online

shell> cctrl
cctrl> set policy automatic
cctrl> exit

After enabling on-disk THL encryption, the `thl index` command will show the rotated THL log file as encrypted:

shell> thl -service alpha index
...
LogIndexEntry thl.data.0000000008(42:42)
LogIndexEntry thl.data.0000000009(43:43) - ENCRYPTED (tls)

Via the APIv2

To access the APIv2, you can use the `curl` command or the `tapi` script provided with Tungsten or your choice of clients like the Postman application.

Let’s start by getting the current state of on-disk encryption and compression using both the tapi tool and the curl command:

shell> tapi -R –run getEncryption alpha
{
   "payloadType" : "BooleanPayload",
   "payload" : {
      "value" : true
   },
   "payloadVersion" : "1"
}

shell> /usr/bin/curl -s --user tungsten:secret --insecure --request GET 'https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/encryption'
{"payloadType":"BooleanPayload","payloadVersion":"1","payload":{"value":true}}

Since the value returned above is “true”, then on-disk THL encryption is ENABLED for service alpha.

shell> tapi -R –run getCompression alpha
{
   "payloadType" : "BooleanPayload",
   "payload" : {
      "value" : false
   },
   "payloadVersion" : "1"
}

shell> /usr/bin/curl -s --user tungsten:secret --insecure --request GET 'https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/compression
{"payloadType":"BooleanPayload","payloadVersion":"1","payload":{"value":false}}

Since the value returned above is “false”, then on-disk THL compression is DISABLED for service alpha.

Note that the output from the tapi command is pretty-printed, while the curl command output is not.

Enable on-disk encryption and compression using tapi

Here is a full session example of how to enable both on-disk encryption and compression for a single service, alpha via the APIv2 tapi tool:

## Enable maintenance mode
shell> tapi --setpolicy maintenance
MAINTENANCE

## Gracefully take the Replicator service OFFLINE
shell> tapi -R --run serviceOffline alpha
{
   "payloadType" : "TaskPayload",
   "payload" : {
      "taskId" : "4c9187b9-a11c-4548-8f5c-c63f3a1a3bca",
      "operation" : "OfflineTask",
      "state" : "in_progress"
   },
   "payloadVersion" : "1"
}

## Ensure the Replicator service is OFFLINE
shell> trepctl services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: -1
appliedLatency  : -1.0
role            : master
serviceName     : alpha
serviceType     : unknown
started         : true
state           : OFFLINE:NORMAL
Finished services command...

## What is the current state of THL on-disk encryption?
shell> tapi -R --run getEncryption --includeServiceName
{
   "value" : false,
   "serviceName" : "alpha"
}

## Enable the THL on-disk encryption 
shell> tapi -R --run setEncryption --true
{
   "value" : "THL encryption is now turned on"
}

## What is the current state of THL on-disk encryption AFTER enablement?
shell> tapi -R --run getEncryption
{
   "value" : true,
}

## What is the current state of THL on-disk compression?
shell> tapi -R --run getCompression --includeServiceName
{
   "value" : false,
   "serviceName" : "alpha"
}

## Enable the THL on-disk compression 
shell> tapi -R --run setCompression --true
{
   "value" : "THL compression is now turned on"
}

## What is the current state of THL on-disk compression AFTER enablement?
shell> tapi -R --run getCompression
{
   "value" : true,
}

## Take the Replicator service ONLINE
shell> tapi -R --run serviceOnline alpha
{
   "payloadType" : "TaskPayload",
   "payload" : {
      "taskId" : "b8794743-3c1c-4242-a1ff-1824bdbb5fbb",
      "operation" : "OnlineTask",
      "state" : "in_progress"
   },
   "payloadVersion" : "1"
}

## List all Replicator tasks
shell> tapi -R --run tasks

## List all Replicator services in a compact way, especially good for CAA
shell> multi_trepctl --skip-headers --hosts `hostname` --fields=servicename,role,state,appliedlastseqno,appliedlatency | sed 's/| //g' | sed 's/|//g'
alpha master ONLINE 21734943 0.456 

## Return the cluster to Automatic mode
shell> tapi --setpolicy automatic
AUTOMATIC


SHORTCUTS
shell> tapi -R --run offline
shell> tapi -R --run online

Enable on-disk encryption and compression using curl

Here is a full session example of how to enable both on-disk encryption and compression for a single service, alpha via the APIv2 using the curl command:

## Enable maintenance mode
shell> /usr/bin/curl -s --user tungsten:secret --insecure --request POST 'https://127.0.0.1:8090/api/v2/manager/service/alpha/policy/maintenance'
{"payloadType":"StringPayload","payloadVersion":"1","payload":{"value":"MAINTENANCE"}}

## Gracefully take the Replicator service OFFLINE
shell> /usr/bin/curl --user tungsten:secret --insecure --request POST https://127.0.0.1:8097/api/v2/replicator/service/alpha/offline
{"payloadType":"TaskPayload","payloadVersion":"1","payload":{"taskId":"f270a007-1a60-4d36-90af-d05ba031623b","state":"in_progress","operation":"OfflineTask"}}

## Gracefully take the Replicator service OFFLINE
shell> /usr/bin/curl --user tungsten:secret --insecure --request POST https://127.0.0.1:8097/api/v2/replicator/service/alpha/offline
{"payloadType":"TaskPayload","payloadVersion":"1","payload":{"taskId":"f270a007-1a60-4d36-90af-d05ba031623b","state":"in_progress","operation":"OfflineTask"}}

## What is the current state of THL on-disk encryption?
shell> /usr/bin/curl --user tungsten:secret --insecure --request GET https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/encryption
{"payloadType":"BooleanPayload","payloadVersion":"1","payload":{"value":false}}

## Enable the THL on-disk encryption 
shell> /usr/bin/curl --user tungsten:secret --insecure --request POST  --header 'Content-Type: application/json' --data '{ "payload" : {"value":"true"}, "payloadVersion" : 1.0, "payloadType" : "BooleanPayload" }' https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/encryption
{"payloadType":"StringPayload","payloadVersion":"1","payload":{"value":"THL encryption is now turned on"}}

## What is the current state of THL on-disk encryption AFTER enablement?
shell> /usr/bin/curl --user tungsten:secret --insecure --request GET https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/encryption
{"payloadType":"BooleanPayload","payloadVersion":"1","payload":{"value":true}}

## What is the current state of THL on-disk compression?
shell> /usr/bin/curl --user tungsten:secret --insecure --request GET https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/compression
{"payloadType":"BooleanPayload","payloadVersion":"1","payload":{"value":false}}

## Enable the THL on-disk compression
shell> /usr/bin/curl --user tungsten:secret --insecure --request POST  --header 'Content-Type: application/json' --data '{ "payload" : {"value":"true"}, "payloadVersion" : 1.0, "payloadType" : "BooleanPayload" }' https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/compression
{"payloadType":"StringPayload","payloadVersion":"1","payload":{"value":"THL compression is now turned on"}}

## What is the current state of THL on-disk compression AFTER enablement?
shell> /usr/bin/curl --user tungsten:secret --insecure --request GET https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/compression
{"payloadType":"BooleanPayload","payloadVersion":"1","payload":{"value":true}}

## Take the Replicator service ONLINE
shell> /usr/bin/curl --user tungsten:secret --insecure --request POST https://127.0.0.1:8097/api/v2/replicator/service/alpha/online
{"payloadType":"TaskPayload","payloadVersion":"1","payload":{"taskId":"16e69041-b254-494a-892f-888a0d393f9e","state":"in_progress","operation":"OnlineTask"}}

## Return the cluster to Automatic mode
shell> /usr/bin/curl -s --user tungsten:secret --insecure --request POST 'https://127.0.0.1:8090/api/v2/manager/service/alpha/policy/automatic'
{"payloadType":"StringPayload","payloadVersion":"1","payload":{"value":"AUTOMATIC"}}

Important Security Note

Encryption uses a pair of dedicated keystore and truststore files (tungsten_thl_keystore.jks and tungsten_thl_truststore.ts by default). If you lose these files, the encrypted THL log files will be impossible to decode.

Getting Started with In-Flight THL Compression

Compression occurs "in-flight" and is requested by the client replicator prior to fetching THL from the remote THL Server.

By default, in-flight THL compression is disabled.

You can only configure THL In-Flight Compression via the tpm option:

repl-thl-client-serialization={LEGACY|JAVA|PROTOBUF|DEFLATE}

You would add this to your tungsten.ini file, or if you are using the Staging deployment method, just add two hyphens in front of each option for use with `tpm configure`, then either install or update via tpm.

The default is LEGACY, meaning disabled, and each of the available values are defined below:

• LEGACY - disables compression; same behavior as versions prior to v7.0.0 - this uses java native object serialization
• DEFLATE - offers the highest level of compression and is slower during the compression and decompression stages
• PROTOBUF - instead of using java native serialization, uses protobuf serialized events, as they are stored in THL on disk, making serialization both quicker and smaller

You can learn more via our online documentation: https://docs.continuent.com/tungsten-clustering-7.0/thl-compress-encrypt.html#thl-compress-inflight.

Advanced Configuration of In-Flight THL Compression

By default, a THL Server will support ALL protocols (PROTOBUF, DEFLATE, JAVA, LEGACY).

If you add the tpm option:

repl-thl-server-serialization={Protocol_list_here,comma-separated,no_spaces}

Any protocols not listed will be DISABLED.

If a THL client asks for a protocol that is not enabled, it will fall back to LEGACY (disabled).

For example, the following tpm option would disable the DEFLATE protocol:

repl-thl-server-serialization=LEGACY,PROTOBUF

Wrap-Up

In this post we explored the details of the new Tungsten Replicator THL On-Disk Encryption, On-Disk Compression, and THL In-Flight Compression features included with Tungsten Clustering version 7.0+.

We covered how to enable and disable all of the features, and how to validate the operations.

• https://docs.continuent.com/tungsten-clustering-7.0/thl-compress-encrypt.html
• https://docs.continuent.com/tungsten-clustering-7.0/thl-compress-encrypt.html#thl-compress-encrypt-disk
• https://docs.continuent.com/tungsten-clustering-7.0/thl-compress-encrypt.html#thl-compress-inflight

Smooth sailing!