LowEndBox - Cheap VPS, Hosting and Dedicated Server Deals

Reduce Stack Limit for Multi-Thread Apps

Date/Time: December 5, 2009 @ 12:32 pm, by LowEndAdmin

Just a small tips for those trying to run multiple threaded applications on OpenVZ.

Personally I code quite a few multi-threaded applications in Python (as some things are just too hard to do asynchronisely). Very often a 6MB RSS application would eat up close to 100MB of VSZ. As one of the most important metrics on OpenVZ — not “guaranteed memory” but “burstable memory”, i.e. privvmpages — is counted on how much your processes have allocated, you’ll soon find your application, which actually only use 6MB of “real memory”, quickly died on your 64MB VPS with 128MB of burstable memory.

The same applies to many other multi-threaded applications that use the default stack size when creating threads. I use pdnsd as a local DNS resolver/cache. A fresh start on Debian would show to use around 1MB RSS but 51MB VSZ!

$ ps -u pdnsd u
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
pdnsd    32521  0.0  0.3  51436   976 ?        Sl   23:12   0:00 /usr/sbin/pdnsd

It turns out that for multi-threaded applications, each thread will be allocated 8MB of stack (although not actually used). The allocated stack actually counts against privvmpages in user beancounters, i.e. against your “burstable memory”. That sucks. Which is why I much prefer a Xen VPS which has a proper memory accounting!

So the trick is reducing the stack size so not much will be allocated for each of the threads. In many cases, you don’t need that much stack (applications should malloc instead of assuming a big stack is available). So let’s reduce the stack size (with ulimit -s) and see how much memory you can save.

# ulimit -s 256
# /etc/init.d/pdnsd restart
...
# ps -u pdnsd u
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
pdnsd     3261  0.0  0.3   3308   916 ?        Sl   23:24   0:00 /usr/sbin/pdnsd

There we go. From 51MB to 3.3MB. You should see a huge drop under the held column of privvmpages as well under /proc/user_beancounters!

You can pretty much use ulimit -s to limit the stack size of any application but I found it’s only really effectively on multi-threaded applications on OpenVZ. To find out which apps are multi-threaded, you can use `ps auxm` to list not just the processes but also threads. Add the limit before you start the service, or put the ulimit line inside the RC file. You’ll find that you can now squeeze more into your low end box :)

The original owner of LowEndBox known as "LowEndAdmin" or "LEA" for short founded LowEndBox in 2008 and created the concept of hosting applications on low resource "Low End Boxes". After creating the roots of the community that we know today, "LEA" stepped aside and allowed others to carry the torch forward.

9 Comments

  1. InsDel:

    Thanks! My memory usage went from 101MB to 58MB :D

    December 5, 2009 @ 1:28 pm | Reply
  2. cling[y]:

    Thanks, I didn’t know OpenVZ monitored VSZ instead of RSS…since my VPS has 512MB I figured I’d go with apache, currently there are 6 Apache processes each taking about 4MB RSS, no big deal I thought, but VSZ they each have about 21MB, that’s about 125MB which is a big deal…

    December 5, 2009 @ 5:57 pm | Reply
  3. Well it is not entirely true. OpenVZ counts how much you allocated, but VSZ is a lot more than how much each process allocates. For example shared memory — that’s why for Apache it looks big, but not necessarily a lot are through memory allocation — much are in fact shared between processes, reported multiple times with ps, but only counted once on OpenVZ.

    December 5, 2009 @ 9:21 pm | Reply
    • Keith:

      As well as changing the rc file I also add to /etc/security/limits.conf just before the end.

      root            -       stack           256
      *               -       stack           256
      
      April 18, 2011 @ 8:01 am | Reply
  4. collaborando:

    In which RC file do I need to add limit?
    or in /etc/security/limits.conf?

    I have Centos 6 on OpenVz.

    September 11, 2011 @ 9:16 pm | Reply
    • I guess he talks about

      /etc/init.d/rc

      or

      /etc/init.d/rc.local

      But I am not sure. What I normally do, is to put that line “ulimit -s 128” almost at the start of an init script. In the example of this post, can be /etc/init.d/pdnsd

      September 11, 2011 @ 10:04 pm | Reply
      • Keith:

        I insert ulimit -s 128 into /etc/init.d/rc using Debian Squeeze.

        September 11, 2011 @ 11:17 pm | Reply
  5. Sasha:

    Thanks so much!

    My Apache memory usage went from 800M to 65M!

    April 18, 2012 @ 9:19 am | Reply

Leave a Reply

Some notes on commenting on LowEndBox:

  • Do not use LowEndBox for support issues. Go to your hosting provider and issue a ticket there. Coming here saying "my VPS is down, what do I do?!" will only have your comments removed.
  • Akismet is used for spam detection. Some comments may be held temporarily for manual approval.
  • Use <pre>...</pre> to quote the output from your terminal/console, or consider using a pastebin service.

Your email address will not be published. Required fields are marked *