How to tune the Linux page cache.
Properly tuning page caches settings can help prevent the systems from spikes of inactivity due to the sheer volume of dirty data that needs to be flushed to disk. The currently reported Linux settings:
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_ratio = 20
Tuning these parameters is an example of a latency/throughput trade-off. Lower values result in lower and uniform latencies, while higher thresholds result in higher throughput but with higher and more variable latencies.
The default values may be high for a host with 177 GB of RAM. These often lead to latency spikes because the Linux kernel accumulates large amounts of dirty data in page cache (with 177 GB of RAM 10% means ~18 GB) and then often writes all the accumulated data in a very short period of time.
We recommend starting with lower values (e.g. 64MB or 128MB), monitor latency
of basic operations (e.g. COMMIT
and simple queries) and increase the limits
until the latency starts to deteriorate.
As the lowest possible value for the *_ratio
parameters is 1%, setting such
low limits on machines with a lot of RAM is only possible using the *_bytes
variants of the parameters. We also recommend using the *_bytes
parameters so
that the limits do not automatically change when adding more RAM.
For example, the following command sets vm.dirty_background_bytes
to 128MB:
sysctl vm.dirty_background_bytes = 134217728
This setting can be put in /etc/sysctl.conf
to make it persistent between
operating system reboots.
It's highly likely that tuning of checkpoint timeouts and checkpoint completion targets will also be useful.