Just a small tips for those trying to run multiple threaded applications on OpenVZ.
Personally I code quite a few multi-threaded applications in Python (as some things are just too hard to do asynchronisely). Very often a 6MB RSS application would eat up close to 100MB of VSZ. As one of the most important metrics on OpenVZ — not “guaranteed memory” but “burstable memory”, i.e. privvmpages — is counted on how much your processes have allocated, you’ll soon find your application, which actually only use 6MB of “real memory”, quickly died on your 64MB VPS with 128MB of burstable memory.
The same applies to many other multi-threaded applications that use the default stack size when creating threads. I use pdnsd as a local DNS resolver/cache. A fresh start on Debian would show to use around 1MB RSS but 51MB VSZ!
$ ps -u pdnsd u USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND pdnsd 32521 0.0 0.3 51436 976 ? Sl 23:12 0:00 /usr/sbin/pdnsd
It turns out that for multi-threaded applications, each thread will be allocated 8MB of stack (although not actually used). The allocated stack actually counts against privvmpages in user beancounters, i.e. against your “burstable memory”. That sucks. Which is why I much prefer a Xen VPS which has a proper memory accounting!
So the trick is reducing the stack size so not much will be allocated for each of the threads. In many cases, you don’t need that much stack (applications should malloc instead of assuming a big stack is available). So let’s reduce the stack size (with ulimit -s) and see how much memory you can save.
# ulimit -s 256 # /etc/init.d/pdnsd restart ... # ps -u pdnsd u USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND pdnsd 3261 0.0 0.3 3308 916 ? Sl 23:24 0:00 /usr/sbin/pdnsd
There we go. From 51MB to 3.3MB. You should see a huge drop under the held column of privvmpages as well under /proc/user_beancounters!
You can pretty much use ulimit -s to limit the stack size of any application but I found it’s only really effectively on multi-threaded applications on OpenVZ. To find out which apps are multi-threaded, you can use `ps auxm` to list not just the processes but also threads. Add the limit before you start the service, or put the ulimit line inside the RC file. You’ll find that you can now squeeze more into your low end box :)
- 5 Reasons Why You Want a Low End Box - May 26, 2021
- Dead Pool January 2012 - February 2, 2012
- exit(0); - January 19, 2012
Thanks! My memory usage went from 101MB to 58MB :D
Thanks, I didn’t know OpenVZ monitored VSZ instead of RSS…since my VPS has 512MB I figured I’d go with apache, currently there are 6 Apache processes each taking about 4MB RSS, no big deal I thought, but VSZ they each have about 21MB, that’s about 125MB which is a big deal…
Well it is not entirely true. OpenVZ counts how much you allocated, but VSZ is a lot more than how much each process allocates. For example shared memory — that’s why for Apache it looks big, but not necessarily a lot are through memory allocation — much are in fact shared between processes, reported multiple times with ps, but only counted once on OpenVZ.
As well as changing the rc file I also add to /etc/security/limits.conf just before the end.
In which RC file do I need to add limit?
or in /etc/security/limits.conf?
I have Centos 6 on OpenVz.
I guess he talks about
/etc/init.d/rc
or
/etc/init.d/rc.local
But I am not sure. What I normally do, is to put that line “ulimit -s 128” almost at the start of an init script. In the example of this post, can be /etc/init.d/pdnsd
I insert ulimit -s 128 into /etc/init.d/rc using Debian Squeeze.
Thanks so much!
My Apache memory usage went from 800M to 65M!