When attempting to determine storage performance on Linux there are common tests such as doing a DD write command and measuring the speed of that. These methods leave a lot to be desired and are especially bad at measuring random read/write performance (the most important for operating system / application performance).
I solved this problem for the Raspberry Pi by creating Pi Benchmarks. This is an open source benchmark that *does* measure random read/write performance as well as several other important factors (IOPS, etc.). We’ve collected many years worth of performance data across thousands of different drives including SSDs / HDDs / others. It’s enough information to be used to make important decisions about performance using hard data.
I’d like to announce that this exact same benchmark is now available for all devices! If you’d previously taken the benchmark on a PC or device other than a Raspberry Pi those results are now retroactively live on the site.
Benchmark Instructions
The benchmark can be ran with the following command:
Once the benchmark completes you will receive a score and a breakdown of the different performance measurements that are used to calculate that score. This will display immediately in the window you are taking the benchmark.
This score is meant to be compared with other scores at https://pibenchmarks.com. Once you take the benchmark within about a minute or two the result will show live on the site. If you click the score link it will take you to a detailed breakdown of each benchmark.
You can browse the site by drive, system board, type, and many other categories. It also has a search function!
Compatibility
The benchmarks compatibility had basically grown to all flavors of Linux that run on the Raspberry Pi. On these flavors the benchmark will retrieve any dependencies needed to perform the benchmark. This should work on most distros.
Known working ones include:
Ubuntu
Debian
Mint
Raspberry Pi OS
Arch Linux
Fenix
Kali
Manjaro
CentOS
Fedora
Alpine
Pop
GalliumOS
SparkyLinux
Armbian
Deepin
openSUSE
Garuda
I am definitely expecting problems though expanding this to PC. There are so many more flavors of Linux that run on PC. If you have a distro that is having trouble with it let me know in the comments and I will definitely get it working for that distro!
Conclusion
I’m very excited to be expanding the storage benchmark as I’ve been working toward this for years now. There are a handful of other benchmarking sites out there and none of them really worked the way I wanted them to for actually making meaningful comparisons so I created this benchmark with an emphasis on random read/write (OS/application) performance.
Thanks to everyone who had already taken this over the years even though the results didn’t display live yet! If you have ideas / requests / feedback definitely let me know here in the comments or on the GitHub issues page. I’m always looking to improve it!
Hello,
When I use raspios
error: SQLSTATE[HY000]: General error: 3988 Conversion from collation utf8mb3_general_ci into utf8mb4_bin impossible for parameter
and the result is not saved, whereas with ubuntu 23.10 there is no error and the results are saved.
How do I configure raspios to avoid this error?
Best regards
to the beginning of the script which fixed this for most people. I suppose it probably has to do with your language settings. You could try configuring them with:
dpkg-reconfigure locales
I’d question how old your Raspberry Pi OS installation is though. To jump releases on Raspberry Pi OS you need to download the latest version. apt dist-upgrade won’t actually update your Raspberry Pi OS release. If you download the latest image for Raspberry Pi OS it should not do this.
Hello
After reconfiguration of the locales: ┌──(michel㉿RASPI-05)-[~]
└─$ sudo dpkg-reconfigure locales
Generating locales (this might take a while)...
en_GB.UTF-8... done
en_US.UTF-8... done
fr_FR.UTF-8... done
Generation complete.
and reboot
And I get the same error, and here are the characteristics of my system:
Raspberry Pi 5 Model B Rev 1.0
CPU: Cortex-A76 Core: 4 Speed (MHz): 2800 GPU Speed: 900
Distribution: Debian GNU/Linux 12 (bookworm)
Kernel: 6.6.21-v8-16k+ #1742 SMP PREEMPT Tue Mar 12 19:50:21 GMT 2024 aarch64
Stockage:
InterFace: GeekPi(S2Pi) N04
Modéle: Samsung model: SSD 990 PRO 1TB size: 931.51 GiB NVMe Version: 2.0
Capacité: 931.51 GiB used: 6.7 GiB (0.7%)
the problem lies in the concatenation of these 2 lines:
Test_nvme=$(nvme list -o json 2>&1 | sed 's/\x0//g')
Test_nvme+=$(nvme show-regs "$BootDrive" -H 2>&1 | sed 's/;/!/g' | sed 's/\x0//g')
Thanks for the report. I’ve removed the second line. I’d rather not remove the first but most people are saying that it’s the second line causing trouble.
I’d say it’s worth giving it another try and see if it will upload now! The first line outputs in json so I don’t think that should be breaking it (or is it). Thanks again for reporting it!
mike91390
2 months ago
Hello,
First of all, I’d like to thank you for your work. I’m contacting you because I’ve done a number of tests with a Crucial P3 Plus 1TB ssd on a Raspberry Pi 5 using Geekworm X1001 Pcie, and none of them appear on the site. Do I need to register to see my results published?
Sincerely
michel
rmkilc
7 months ago
When I run this on my Orange Pi 5 Plus with a 970 EVO Plus M.2 NVMe SSD, I don’t get the “Result submitted successfully” line. But otherwise the test appears to run correctly otherwise and gives results and a score. Why is it not uploading the results?
I’ll take a look here and see why it’s not showing up this weekend!
Geoff Palmer
1 year ago
I have just submitted my first testing. Was very disapointed with my score however it is a very odd microSD card on a Pi 3b so might not show in the reports i think. This is the card i am using
Would love for this to be added 🙂 My username i used on the report is Sabitech
Ohhhh, very interesting, a HP SD card! So your result should be on the site but it’s just not displaying yet because I need to add some pictures for the HP SD card since it hasn’t been tested before.
Once I do this your existing benchmark will show up retroactively. All the data is still recorded if that makes sense.
I have a few other new devices to get added to. This should appear over the next few days at the most (and then all future ones will be instantly recognized once I’ve added the first one). Thanks for reporting this!
UPDATE: It took me longer than I said but I saw yours in the PiBenchmarks database and got it added today. Your benchmarks should show up now (as well as any further ones will show up immediately).
Thanks and take care!
Schugy
1 year ago
It’s great that the benchmark works on PCs with almost any distro. There was no hdparm result on openSuse Leap 15.4 but my tiny 128 gb Samsung UFS card scored 7169 points. Maybe this error is the reason why the result wasn’t published. The script checked for apt and pacman but not for zypper. I also wonder whether it harms the internal SSD when it is trimmed 3 times in 5 minutes while testing a memory card in an external USB card reader. Thank you for the effort you put into this.
Thanks for trying it out on several PCs! That definitely helps me actually. Adding Zypper would probably get those ones fixed! You may be able to install the dependencies through zypper and run it again for a successful result I’m guessing but I’d definitely like to fix those.
I found a similar issue to this during my i96 testing. That one would not complete the fio test (despite it being present) which resulted in an incomplete result. Basically both your and my incomplete result are still in the submitted benchmarks but they are just not displayed at the moment.
I’ve been thinking of actually making the incomplete results show and be marked as incomplete. They would still be viewable on the site. This would make it a lot easier for me to see these incomplete entries as right now they are buried pretty deep and I have to really go looking for them and then try to figure out what went wrong (without the benefit of people telling me like you just did in the overwhelming majority of cases). They could just be scored as “0” for the tests they couldn’t complete when it comes to things like iozone. The zypper issue looks like one I should add compatibility for.
Now as far as the TRIM command hurting the SSD it basically won’t hurt it. Trim is just an optimization that trims out deleted objects on your drive by actually erasing that space instead of just marking it as free. The reason it doesn’t just do this in the first place is because it takes time / is slow. If you want to run the trim command and burn the several seconds or more for the drive to perform this function then it doesn’t do anything but speed up writing data to the drive.
Trim is particularly helpful after you have just imaged a new SD card (or SSD) for example because the entire device will be filled with garbage usually (the previous image and data before you reimaged it). If you run a trim after that you will see it trim almost the entire size of the SD card / SSD. That is literally what is happening when you do it right after imaging and all of that data from the previous image is zero’d out. This makes writing to it measurably faster which is why it’s done for the benchmark.
The reason trim doesn’t hurt SSDs or SD cards anymore is actually due to advances in the technology. It’s because they have wear leveling algorithms now but this was not always the case. On first and second generation SSDs (like circa 2010 and no later) they actually did wear out faster if you trimmed “too much” (as in there was a too much with the earliest SSDs). They didn’t have wear-leveling which caused other problems such as if you wrote to the same file too many times it would literally wear out those sectors of the drive when many other sectors might never be accessed potentially. I’d imagine this was true for SD cards at some point but not on any likely to be used with any modern SBC as the sizes at that time were dramatically smaller and limiting.
Fortunately that does not happen anymore and hasn’t for many years but in early versions of the technology that was a problem (and not just for trim, rewrites of the same sectors over and over caused the same uneven wear during what we’d consider normal use today of maybe saving a document 50 times or something while editing).
Today if you saved a word document 50 times you’d actually have 50 separate copies on your SSD (spread out throughout the space marked “free” on the drive) without running the trim command because of the wear leveling. The old way it would have wrote to the exact same sector 50 times making that sector’s lifespan significantly (and unevenly) worn from the rest of the drive. Just like with uneven wear on tires on a vehicle you can be sure that the places that are worn unevenly are where it is going to fail (usually prematurely).
It’s modern wear leveling which shifts these operations to different parts of the drive evenly to prevent damage or nuking sectors of the drive with 50,000 rewrites, etc. and these algorithms are implemented on SD cards as well. It will not write to the same part of a drive anymore over and over which was a real concern with the older technology and it extended the life of SSDs by an order of magnitude at least when they solved this.
This is why both SSDs and SD cards can be trimmed, but a lot of other types can’t either because the operation makes no sense like with a HDD or it’s not safe and the tool will just ignore those drives. Occasionally you may have a drive that you know is a SSD that it won’t let you trim. This is because for whatever reason it’s not detecting it as a drive that is safe to run the trim command on. There are guides out there to override this which helps when it is a drive that actually is supposed to have trim but it isn’t being detected because of a firmware bug or some other reason. If it’s a drive that is not safe to run trim on and you manage to override all the safety protocols then it’s possible to harm types of drives that don’t have wear leveling and aren’t meant to be trimmed (although it’s almost always a bug when I encounter this that can be fixed by drive or storage adapter firmware updates).
Hopefully that helps, I’ll definitely be making some fixes for this and I’ll definitely check into zypper as well and see if we can get it to run on the last devices that remain too stubborn to complete one yet (due to OS or the device themselves, it varies). Thanks for reporting this!
Hello,
When I use raspios
error: SQLSTATE[HY000]: General error: 3988 Conversion from collation utf8mb3_general_ci into utf8mb4_bin impossible for parameter
and the result is not saved, whereas with ubuntu 23.10 there is no error and the results are saved.
How do I configure raspios to avoid this error?
Best regards
Hey Mike,
I added:
export LC_ALL=Cto the beginning of the script which fixed this for most people. I suppose it probably has to do with your language settings. You could try configuring them with:
dpkg-reconfigure localesI’d question how old your Raspberry Pi OS installation is though. To jump releases on Raspberry Pi OS you need to download the latest version. apt dist-upgrade won’t actually update your Raspberry Pi OS release. If you download the latest image for Raspberry Pi OS it should not do this.
Hopefully that helps!
Hello
After reconfiguration of the locales:
┌──(michel㉿RASPI-05)-[~]└─$ sudo dpkg-reconfigure locales
Generating locales (this might take a while)...
en_GB.UTF-8... done
en_US.UTF-8... done
fr_FR.UTF-8... done
Generation complete.
and reboot
And I get the same error, and here are the characteristics of my system:
Raspberry Pi 5 Model B Rev 1.0
CPU: Cortex-A76 Core: 4 Speed (MHz): 2800 GPU Speed: 900
Distribution: Debian GNU/Linux 12 (bookworm)
Kernel: 6.6.21-v8-16k+ #1742 SMP PREEMPT Tue Mar 12 19:50:21 GMT 2024 aarch64
Stockage:
InterFace: GeekPi(S2Pi) N04
Modéle: Samsung model: SSD 990 PRO 1TB size: 931.51 GiB NVMe Version: 2.0
Capacité: 931.51 GiB used: 6.7 GiB (0.7%)
the problem lies in the concatenation of these 2 lines:
Test_nvme=$(nvme list -o json 2>&1 | sed 's/\x0//g')Test_nvme+=$(nvme show-regs "$BootDrive" -H 2>&1 | sed 's/;/!/g' | sed 's/\x0//g')
Thank you for the effort you put into this.
Hey Mike,
Thanks for the report. I’ve removed the second line. I’d rather not remove the first but most people are saying that it’s the second line causing trouble.
I’d say it’s worth giving it another try and see if it will upload now! The first line outputs in json so I don’t think that should be breaking it (or is it). Thanks again for reporting it!
Hello,
First of all, I’d like to thank you for your work. I’m contacting you because I’ve done a number of tests with a Crucial P3 Plus 1TB ssd on a Raspberry Pi 5 using Geekworm X1001 Pcie, and none of them appear on the site. Do I need to register to see my results published?
Sincerely
michel
When I run this on my Orange Pi 5 Plus with a 970 EVO Plus M.2 NVMe SSD, I don’t get the “Result submitted successfully” line. But otherwise the test appears to run correctly otherwise and gives results and a score. Why is it not uploading the results?
Hey rmkilc,
I’ll take a look here and see why it’s not showing up this weekend!
I have just submitted my first testing. Was very disapointed with my score however it is a very odd microSD card on a Pi 3b so might not show in the reports i think.
This is the card i am using
Would love for this to be added 🙂 My username i used on the report is Sabitech
Hey Geoff,
Ohhhh, very interesting, a HP SD card! So your result should be on the site but it’s just not displaying yet because I need to add some pictures for the HP SD card since it hasn’t been tested before.
Once I do this your existing benchmark will show up retroactively. All the data is still recorded if that makes sense.
I have a few other new devices to get added to. This should appear over the next few days at the most (and then all future ones will be instantly recognized once I’ve added the first one). Thanks for reporting this!
UPDATE: It took me longer than I said but I saw yours in the PiBenchmarks database and got it added today. Your benchmarks should show up now (as well as any further ones will show up immediately).
Thanks and take care!
It’s great that the benchmark works on PCs with almost any distro. There was no hdparm result on openSuse Leap 15.4 but my tiny 128 gb Samsung UFS card scored 7169 points. Maybe this error is the reason why the result wasn’t published. The script checked for apt and pacman but not for zypper. I also wonder whether it harms the internal SSD when it is trimmed 3 times in 5 minutes while testing a memory card in an external USB card reader. Thank you for the effort you put into this.
Hey Schugy,
Thanks for trying it out on several PCs! That definitely helps me actually. Adding Zypper would probably get those ones fixed! You may be able to install the dependencies through zypper and run it again for a successful result I’m guessing but I’d definitely like to fix those.
I found a similar issue to this during my i96 testing. That one would not complete the fio test (despite it being present) which resulted in an incomplete result. Basically both your and my incomplete result are still in the submitted benchmarks but they are just not displayed at the moment.
I’ve been thinking of actually making the incomplete results show and be marked as incomplete. They would still be viewable on the site. This would make it a lot easier for me to see these incomplete entries as right now they are buried pretty deep and I have to really go looking for them and then try to figure out what went wrong (without the benefit of people telling me like you just did in the overwhelming majority of cases). They could just be scored as “0” for the tests they couldn’t complete when it comes to things like iozone. The zypper issue looks like one I should add compatibility for.
Now as far as the TRIM command hurting the SSD it basically won’t hurt it. Trim is just an optimization that trims out deleted objects on your drive by actually erasing that space instead of just marking it as free. The reason it doesn’t just do this in the first place is because it takes time / is slow. If you want to run the trim command and burn the several seconds or more for the drive to perform this function then it doesn’t do anything but speed up writing data to the drive.
Trim is particularly helpful after you have just imaged a new SD card (or SSD) for example because the entire device will be filled with garbage usually (the previous image and data before you reimaged it). If you run a trim after that you will see it trim almost the entire size of the SD card / SSD. That is literally what is happening when you do it right after imaging and all of that data from the previous image is zero’d out. This makes writing to it measurably faster which is why it’s done for the benchmark.
The reason trim doesn’t hurt SSDs or SD cards anymore is actually due to advances in the technology. It’s because they have wear leveling algorithms now but this was not always the case. On first and second generation SSDs (like circa 2010 and no later) they actually did wear out faster if you trimmed “too much” (as in there was a too much with the earliest SSDs). They didn’t have wear-leveling which caused other problems such as if you wrote to the same file too many times it would literally wear out those sectors of the drive when many other sectors might never be accessed potentially. I’d imagine this was true for SD cards at some point but not on any likely to be used with any modern SBC as the sizes at that time were dramatically smaller and limiting.
Fortunately that does not happen anymore and hasn’t for many years but in early versions of the technology that was a problem (and not just for trim, rewrites of the same sectors over and over caused the same uneven wear during what we’d consider normal use today of maybe saving a document 50 times or something while editing).
Today if you saved a word document 50 times you’d actually have 50 separate copies on your SSD (spread out throughout the space marked “free” on the drive) without running the trim command because of the wear leveling. The old way it would have wrote to the exact same sector 50 times making that sector’s lifespan significantly (and unevenly) worn from the rest of the drive. Just like with uneven wear on tires on a vehicle you can be sure that the places that are worn unevenly are where it is going to fail (usually prematurely).
It’s modern wear leveling which shifts these operations to different parts of the drive evenly to prevent damage or nuking sectors of the drive with 50,000 rewrites, etc. and these algorithms are implemented on SD cards as well. It will not write to the same part of a drive anymore over and over which was a real concern with the older technology and it extended the life of SSDs by an order of magnitude at least when they solved this.
This is why both SSDs and SD cards can be trimmed, but a lot of other types can’t either because the operation makes no sense like with a HDD or it’s not safe and the tool will just ignore those drives. Occasionally you may have a drive that you know is a SSD that it won’t let you trim. This is because for whatever reason it’s not detecting it as a drive that is safe to run the trim command on. There are guides out there to override this which helps when it is a drive that actually is supposed to have trim but it isn’t being detected because of a firmware bug or some other reason. If it’s a drive that is not safe to run trim on and you manage to override all the safety protocols then it’s possible to harm types of drives that don’t have wear leveling and aren’t meant to be trimmed (although it’s almost always a bug when I encounter this that can be fixed by drive or storage adapter firmware updates).
Hopefully that helps, I’ll definitely be making some fixes for this and I’ll definitely check into zypper as well and see if we can get it to run on the last devices that remain too stubborn to complete one yet (due to OS or the device themselves, it varies). Thanks for reporting this!