My Journey to Improve Disk Performance on the Raspberry Pi

 

I switched to Gluster FS a while ago to provide easier container mobility across my Raspberry Pi Docker Cluster. Gluster worked great and was easy to get up and running but I had very poor performance. The average write speed was about 1 MB/s which is unacceptable for a filesystem that will undergo a lot of writes. I decided that it was time to take action and started looking at kernel parameters that could be changed.

From researching performance over the years, I often came across the following settings and thought that this is the best place to start. Since the Raspberry Pi is not really made for performance, I did not think it was worth the time investigating additional settings.


vm.dirty_background_ratio – This is the percentage of RAM  that can be filled with dirty memory pages  before it is written to disk. The default value on Raspbian is 10.

vm.dirty_ratio  – This is the maximum amount of RAM that can be filled with dirty pages before writing the dirty memory pages to disk. The default value on Raspbian is 20.

vm.swappiness – This controls how aggressive the system should use the swap space. The closer the value is to 100,  the system will utilize swap more often. The default value on Raspbian is 1.

vm.vfs_cache_pressure –  This controls  how aggressive the kernel reclaims memory used for dentry and inode caches. The default value on Raspbian is 100.

To see all of the current kernel parameters on a Linux system, run:

sysctl -a

Now I will run a few benchmarks on the root and Gluster filesystems to test the default performance.

I used the dd command below to create a 105 MB file:

 dd if=/dev/zero of=/tmp/test.img bs=1M count=100

I ran the test  three times to get the average throughput numbers:

Average results for the root filesystem:  41.7 MB/s
Average results for the  Gluster filesystem:  1.8 MB/s

After playing around  with the default settings, I found the perfect combination to give me the best performance.   I changed the following  settings with sysctl and performed the same dd test three more times:

 sysctl -w vm.swappiness=15
 sysctl -w vm.vfs_cache_pressure=50
 sysctl -w vm.dirty_background_ratio=15
 sysctl -w vm.dirty_ratio=20

Average results for the root filesystem:  46 MB/s
Average results for the  Gluster filesystem:  5 MB/s

Now it is time for the fun part of summarizing the changes and trying to understand the results.  For Swap and vfs_cache_pressure, the Raspberry Pi performed better by having the OS act slightly more aggressive with swapping pages and by cutting the vfs_cache_pressure in half.  The dirty_background_ratio was increased slightly while the dirty_ratio was kept the same. Now let’s take a look at how increasing the dirty_background_ratio helped.

The Pi will be writing less to disk but with slightly more data when it does. The Pi has 1 GB of memory and that means 200 MB of RAM will be used for the dirty pages and the system will start writing the pages to the disk when the cache reaches 150 MB rather than the default 100 MB.

No matter what the use case for a  system is,  finding the best performance settings will always require tedious work. My settings above may not be optimal for your needs. It will take a lot of time and effort to find the optimal settings for a system and I hope that this blog will help you get on the right path to figure out what those are.