Introduction
Tungsten Clustering contains many tools to monitor your cluster, and today we will look at a new one - the `tungsten_get_status` command, included with Tungsten versions 6.1.19+ and 7.0.2+.
This tool was created in response to a customer request for a simple script that could display the status of all nodes cluster-wide for any topology from a single place.
The status includes the datasource and replicator layers along with the policy for each cluster.
This blog post is a continuation of “Monitoring Made Easy: Watching Your Tungsten Cluster Using Built-In Tools” at https://www.continuent.com/resources/blog/monitoring-clusters-for-mysql.
Use-Case
The original need for this came from a customer who did not have a central monitoring service, and did not want emailed alerts, but instead wanted the most efficient way for a human staff member to perform a visual check once per day on each cluster.
You may have or think of other needs. As always, YMMV!
Command Summary
To begin with, here is the basic syntax for the command:
shell> tungsten_get_status [-h|--errors]
The Fine Print
By default, all nodes will be displayed no matter what the state.
Use the `--errors` option to limit the node display to those nodes that are not tagged as “GOOD”.
All three topologies are supported: Standalone (STD), Composite Active/Passive (CAP), and Composite Active/Active (CAA).
Getting Status
To begin, let’s examine the command output for the standalone (STD) topology in detail:
shell> tungsten_get_status
STANDALONE CLUSTER north
WARN north POLICY=MAINTENANCE
GOOD north/db7-demo.continuent.com: DATASOURCE role primary, state ONLINE, progress 0, latency 0.938
GOOD north/db7-demo.continuent.com: REPLICATOR role primary, state ONLINE
GOOD north/db8-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.151
GOOD north/db8-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db7-demo.continuent.com
GOOD north/db9-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.691
GOOD north/db9-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db7-demo.continuent.com
The first line of output always shows the topology and cluster name.
The second line shows the current policy for that cluster.
Then each node is displayed with two lines per node, one for the datasource/manager layer, and one for the replicator layer.
The datasource layer shows the following “fields”:
- {status tag}, one of GOOD, WARN or ERROR
- {service name}/{host name}
- {line type} DATASOURCE
- {role}
- {state}
- {sequence number progress}
- {replicator applied latency}
The replicator layer shows the following “fields”:
- {status tag}, one of GOOD, WARN or ERROR
- {service name}/{host name}
- {line type} REPLICATOR
- {role}
- {state}
- {upstream THL server, if on a replica node}
Now, let’s examine the command output for Composite Active/Passive (CAP) and Composite Active/Active (CAA):
shell> tungsten_get_status
COMPOSITE CLUSTER (CAP): parent global, children east west
#global (composite parent)
GOOD east: role composite active, state ONLINE
GOOD west: role composite passive, state ONLINE
##east (composite child)
WARN east POLICY=MAINTENANCE
GOOD east/db1-demo.continuent.com: DATASOURCE role primary, state ONLINE, progress 0, latency 0.661
GOOD east/db1-demo.continuent.com: REPLICATOR role primary, state ONLINE
GOOD east/db2-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 0.954
GOOD east/db2-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db1-demo.continuent.com
GOOD east/db3-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 0.785
GOOD east/db3-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db1-demo.continuent.com
##west (composite child)
WARN west POLICY=MAINTENANCE
GOOD west/db4-demo.continuent.com: DATASOURCE role relay, state ONLINE, progress 0, latency 0.939
GOOD west/db4-demo.continuent.com: REPLICATOR role relay, state ONLINE primary=db1-demo.continuent.com
GOOD west/db5-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 0.921
GOOD west/db5-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db6-demo.continuent.com
GOOD west/db6-demo.continuent.com: DATASOURCE role relay, state ONLINE, progress 0, latency 0.842
GOOD west/db6-demo.continuent.com: REPLICATOR role relay, state ONLINE primary=db1-demo.continuent.com
shell> tungsten_get_status
COMPOSITE CLUSTER (CAA): parent world, children emea usa, subServices emea_from_usa usa_from_emea
#world (composite parent)
GOOD emea: role composite active, state ONLINE, global progress -1, max latency 1.349
GOOD usa: role composite active, state ONLINE, global progress -1, max latency 1.323
##emea (composite child)
WARN emea POLICY=MAINTENANCE
GOOD emea/db13-demo.continuent.com: DATASOURCE role primary, state ONLINE, progress 0, latency 0.786
GOOD emea/db13-demo.continuent.com: REPLICATOR role primary, state ONLINE
GOOD emea/db14-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.049
GOOD emea/db14-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db13-demo.continuent.com
GOOD emea/db15-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 0.992
GOOD emea/db15-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db13-demo.continuent.com
##usa (composite child)
WARN usa POLICY=MAINTENANCE
GOOD usa/db16-demo.continuent.com: DATASOURCE role primary, state ONLINE, progress 0, latency 1.134
GOOD usa/db16-demo.continuent.com: REPLICATOR role primary, state ONLINE
GOOD usa/db17-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.316
GOOD usa/db17-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db16-demo.continuent.com
GOOD usa/db18-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.270
GOOD usa/db18-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db16-demo.continuent.com
##emea_from_usa (composite subService)
WARN emea_from_usa POLICY=MAINTENANCE
GOOD emea_from_usa/db13-demo.continuent.com: DATASOURCE role relay, state ONLINE, progress 0, latency 1.282
GOOD emea_from_usa/db13-demo.continuent.com: REPLICATOR role relay, state ONLINE primary=db16-demo.continuent.com
GOOD emea_from_usa/db14-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.349
GOOD emea_from_usa/db14-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db13-demo.continuent.com
GOOD emea_from_usa/db15-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.326
GOOD emea_from_usa/db15-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db13-demo.continuent.com
##usa_from_emea (composite subService)
WARN usa_from_emea POLICY=MAINTENANCE
GOOD usa_from_emea/db16-demo.continuent.com: DATASOURCE role relay, state ONLINE, progress 0, latency 1.197
GOOD usa_from_emea/db16-demo.continuent.com: REPLICATOR role relay, state ONLINE primary=db13-demo.continuent.com
GOOD usa_from_emea/db17-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.323
GOOD usa_from_emea/db17-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db16-demo.continuent.com
GOOD usa_from_emea/db18-demo.continuent.com: DATASOURCE role replica, state ONLINE, progress 0, latency 1.267
GOOD usa_from_emea/db18-demo.continuent.com: REPLICATOR role replica, state ONLINE primary=db16-demo.continuent.com
Wrap-Up
In this post we explored the new command `tungsten_get_status` included with Tungsten version 6.1.19+/7.0.2+.
Smooth sailing!
Comments
Add new comment