Using nodetool setstreamthroughput to adjust decomission and other streaming peformance

Posted on August 3rd, 2012

Cassandra uses streaming in several places. Streaming is used when a node joins or leaves the ring. For example, when a node is decomissioned using nodetool decommission it streams its data to other nodes before leaving the ring. This value can be adjusted in both the confirguration file and during runtime using nodetool or JMX.

How to do it…

In the conf/cassandra.yaml file the default value can be set:

# When unset, the default is 400 Mbps or 50 MB/s.
# stream_throughput_outbound_megabits_per_sec: 400

This command can be adjusted at runtime using nodetool. The value here is in MB/s

bin/nodetool -h cdbeq01 -p 8585 setstreamthroughput 51

Note: These changes need to be done per machine.

How it works…

setstreamthroughput is a rate limit. The default is 50 Mb/second. If you have a high speed network such as gigabit ethernew this value can be set higher to speed-up certain operations. For example, when doing nodetool decommission the decomission will finsih sooner if the configuration value is raised, assuming no other limiting factor is involved. However, you may wish to set the value lower to researve bandwidth and other resources for low latency operations.

There is more…

Remember that a network with Gigabit ethernet ports may not be capable of supporting 100% utilization on each port at the same time. Consult your network engineer and consider how many of these streaming or other operations bandwidth intensive are happening at once when adjusting this setting cluster wide.

Filed under Chapter 4 - Performance Tuning, Chapter 7 Administration | No Comments »