LowEndTalk veteran jsg (previously interviewed on LowEndBox) has begun a new series of VPS benchmarks on LowEndTalk. They’re detailed, factual, and well-written and if you’re interested in benchmarks of popular providers, well worth your time to read.
What is Benchmarking?
Benchmarking consists of running various tests and comparing performance between providers. It could be done on VPSes or dedicated servers. While it could be done manually, typically scripts and tools are used. Areas such as CPU performance, disk performance, and network performance are sampled and compared.
There are many test suites available and they operate at all levels of the stack. One popular benchmark is bench.sh, which is both the name of the script and its URL. You can download that script and run it. Here’s another I used to see frequently used.
There are other types of tests – for example, different tools that generate load on web servers. The goal is to generate a set of data and then compare results between providers in order to select the optimally-performing system for your needs.
That “for your needs” is key. If you’re running a VPN proxy to Tokyo, than Asian network connectivity is going to matter far more than I/O. For a game server, ping times to where you play is more important than network speed tests pulling from the other side of the world. And if you’re running a database server, then CPU and disk performance may outweigh a better ping time.
Limitations of Benchmarking
Benchmarking is not perfect, and you cannot simply run a script and programatically determine the optimal node for your needs. Some limitations of benchmarking include:
- You may have other processes running on the nodes which consume CPU, disk, etc. that could skew benchmarking results.
- As customers are loaded onto host nodes, performance may change.
- Network performance can vary based on what neighbor VPSes are doing, network issues, intermediate hops, etc.
- Benchmarking can’t test things like provider reliability, downtimes, etc. A host that performs extremely well but is frequently down probably doesn’t fit your needs as well as one that is perhaps a few percentage points slower but has high uptime.
- And of course, benchmarks don’t provide just one number, so you need to weigh what’s important to you.
Benchmarking is simply one more tool you can use to evaluate hosts, not a definitive, binary answer.
A New Benchmark
jsg has been working on a new tool called vpsb which performs a great deal of analysis on nodes. It was born of his view of shortcomings in traditional benchmarks. To quote from its manual:
“vpsb” (VPS benchmark) is a tool to get meaningful and relevant info about a SUT (“server under test”). This information consists of
– system information. Things like cpu type, number of cores, memory but also
some less basic info like cache level sizes and flags.– performance. This is about processor and memory performance. While this might sound like not much (there’s a ton of benchmarks out there) a typical internet server isn’t much about integer and/or floating point performance but mainly about moving and processing large amounts of data, lots of strings and increasingly about crypto jobs.
In other words; many if not most benchmarks aren’t exactly helpful and useful when benchmarking, say a web server. vpsb, however, is made expressly for those jobs and the performance benchmarks make sure to trash caches, to test typical crypto operations and to move around large amounts of data. Also the performance test is done both single and multi core.
– disks. vpsb tests disks, in addition to sequential tests, in a way that trashes buffers in random access based tests (unlike sequential ones) which are truly and seriously random. Both the sectors (the location accessed) and the data itself are purely random (using a good quality PRNG)
– networking. This is an obvious one. After all the network is to a server what the pond is to a fish. vpsb approaches that task in a way that is rather different from most other tools. For a start – and unlike most server benchmarks – vpsb gives you true and relevant numbers. Unlike other tests vpsb starts measuring only after name resolution and http setup of the other side, i.e. vpsb measures what you are really interested in. In addition vpsb v.2.x provides both normal and web ping times.
You can read more about it in this LowEndTalk thread. You can download the software itself. He has begun analyzing hosts, and his review of NexusBytes is covered in this thread.
Related Posts:
I also enjoy using LEBs! I have an empire of little guys for VPNs, larger guys for apps, and big guys for databases and storage. When I'm not in front a screen I'm into German Shepherd dogs, quality knives, target shooting, theology, tabletop roleplaying games, and forest hiking.
I enjoy writing technical articles here on LowEndBox to help people get more out of their systems. You can find me on LowEndTalk @raindog308
- LowEndBoxTV: Are All Hetzner Locations Alike?No!And Piotr Has the Proof: “This is Clearly Not the Same Processor” - November 22, 2024
- One Week From Tomorrow…THE WORLD WILL LOSE THEIR MINDS!Lines Are Already Forming! - November 21, 2024
- Crunchbits Discontinuing Popular Annual Plans – The Community Mourns! - November 20, 2024
Leave a Reply