LowEndBox - Cheap VPS, Hosting and Dedicated Server Deals

Tutorial - The LowEndCluster - Part 1

lowendtutorial

The tutorials are back! After a period of absence it’s time to get the tutorials back on track and I’m going to start of with a great series, and a challenging one on that.

With all these LowEndSpirit-style offers these days it’s extremely cheap to get a whole bunch of servers quite quickly. For less than $50 a year you can have at least 10 servers in two continents across 10 different cities. That’s pretty awesome, right? But what do you do with all those servers?

That’s where my (*cough*) brilliant idea comes in: we’re going to create a cluster out of them! And not just any cluster, a redundant cluster! Oh well, if we make it anyway. Let’s first get somewhere before I start bragging. So, up to part 1 of N parts…

The ingredients

From what I can think of right now, this is what kind of servers you need for this cluster (taking into account redundancy):

  • 2x VPS with IPv4 and IPv6 (BudgetVZ offers these for a low fee) for load balancers
  • 2x VPS with just IPv6 for MySQL
  • 2x VPS with just IPv6 for Web Nodes
  • 3x VPS with just IPv6 for a filesystem

The above is just an indication, I may need to adapt as we go along. More is always possible, less is making it problematic. We need the IPv4 on the load balancers in order to get dual stack easy access to whatever you’re going to host on there.

For now, my plan is as follows:

  • Host a WordPress site on the cluster
  • Use MariaDB as a database server
  • Use Apache as an application server on the web nodes
  • Use NGINX as a load balancer and eventually to server/cache static files
  • Use MooseFS or CEPH as a filesystem (depending on which one I can get to work)

So, if you’re in for the challenge, let’s get those servers (if you don’t have them already) and get cracking!

Oh, one more thing: you may want to get KVM machines or OpenVZ machines with FUSE for those filesystem nodes. I’d go for KVM as that broadens your options in case you’d like to switch in the future (to say, GlusterFS once they finally support IPv6…).

The MariaDB cluster

This is the easiest step… for me! As I’ve already written this tutorial and only had to make some tweaks. So, fire up a new tab and go to my original tutorial!

Follow this tutorial step-by-step, taking the following notes and changes into account:

bind-address

The bind-address should listen to any interface, not just to IPv4. So instead of this:

bind_address = 0.0.0.0

Use this:

bind_address = ::

As strange as this may look, but that will make the server listen on all interfaces. After having made that change, be sure to restart MariaDB:

sudo service mysql restart

IPv6 instead of IPv4

Wherever I’ve used an IPv4 address, use the corresponding IPv6 address of the server(s). Make sure they stay within the quotes, otherwise you’ll break stuff.

If you’re running these commands after the fact, flush the privileges to make sure it’s all set up correctly. From within MariaDB:

FLUSH PRIVILEGES;

And you’re good!


That was it for this week. Next time (in about two weeks) I want to focus on the web nodes and how we’re going to set those up.

Any feedback thus far, or suggestions, feel free to let me know!

mpkossen

24 Comments

  1. rosh:

    Great tutorial! :)
    What is your personal favorite vps seller?

    April 18, 2015 @ 1:34 am | Reply
  2. jbarbieri:

    What do you mean budgetvz offers those for free? Offer dual stack for free?

    Looking forward to see how you set up redundancy. I have been working on something similar with a VPS and a dedicated server. Hardest part is trying to keep the files in sync.

    April 18, 2015 @ 4:41 am | Reply
    • Probably he means that they offer free load balancers.

      April 19, 2015 @ 11:48 am | Reply
    • Jack Morton:

      He obviously meant “IPv4 and IPv6″…

      April 25, 2015 @ 4:38 am | Reply
  3. rm:

    > Use MooseFS or CEPH as a filesystem (depending on which one I can get to work)

    Try also GlusterFS, it seemed to be the easiest to set up among all of them.

    April 18, 2015 @ 2:37 pm | Reply
    • Maarten Kossen:

      It is supposed to be, but it doesn’t play with IPv6 and/or custom ports. I’ve spent hours on trying to set it up only to find out it is not possible without IPv4.

      They will support IPv6 in the future, though.

      May 3, 2015 @ 11:23 am | Reply
  4. I’d suggest swappping Apache with Nginx, and then replacing NGINX as a load balancer for Varnish, which is blazing fast. IMO performance would be much better with such combination, don’t you think?

    April 18, 2015 @ 2:52 pm | Reply
    • Maarten Kossen:

      I am going to replace Apache with NGINX indeed. I am sticking to NGINX for load balancers as well, though, as Varnish is a whole different sport :-)

      May 3, 2015 @ 11:24 am | Reply
  5. so, we need setup MariaDB cluster from begining again if one of the cluster down?

    April 19, 2015 @ 4:43 pm | Reply
    • Maarten Kossen:

      I’m not sure what you mean.

      May 3, 2015 @ 11:24 am | Reply
  6. What minimum size VPS’s do you recommend for the different server types?

    Obviously the size of file, web, and database servers will ultimately be determined by the amount of content, but to get starter, what minimum sizes are you looking at, particularly fro the load balancers?

    April 20, 2015 @ 1:58 pm | Reply
  7. sandoz:

    is it also possible with 3 or 4 vps’s?
    i don’t really need redundancy

    and, where do i get 9 vps for $50/year?

    April 21, 2015 @ 7:27 am | Reply
  8. Mathias R:

    Hi, the same question where did you get all of these with $ 50/ year

    April 25, 2015 @ 7:35 pm | Reply
  9. RyanB:

    Quite simple to get 9 VPS Servers for $50 a year.

    LowEndSpirit.com – 13 Locations @ 3 euro per year 3 * 13 = 39euro (based on todays rates) = $42.41USD – Then if needed to get a small VPS with a Dedicated IPv4 for frontend, there are a few in the ~8USD a year bracket.

    April 26, 2015 @ 3:44 am | Reply
  10. sandoz:

    these are all NAT servers

    April 30, 2015 @ 9:11 am | Reply
  11. sandoz:

    and in the tutorial hes talking about KVM servers

    April 30, 2015 @ 9:14 am | Reply
    • Maarten Kossen:

      These servers are NAT servers except for the BudgetVZ ones, these include an IPv4. There are three KVM machines which littlehappycloud.net provides.

      Calculating the cost:

      2x BudgetVZ 128MB = €8/year
      3x KVM 128MB = €16.50/year
      2x MegaVZ 512MB = €11/year
      2x LowEndSpirit 128MB = €6/year

      Grand total: €41.50/year = $46.25/year

      So, there’s room in the budget _and_ I’ve picked two instances with more than 128MB RAM for database servers.

      May 5, 2015 @ 11:10 am | Reply
      • Maarten Kossen:

        Actually, while thinking about this, let’s bring up the following:

        Clustered file systems in this setup are still a pain, mostly because they are built for enterprise environments and rely on IPv4 support. In addition to that, they require some level of access to the OS that’s hard to obtain in a commercial OpenVZ environment (something with admin privileges). That last bit is the reason I went for KVM, that first bit is why I’m now looking for a working solution.

        The easiest and most LowEnd way would be SSHFS and rsync. Manual intervention is still part of this setup and the DB only replicates and doesn’t fail-over. I call this plan B for now.

        Alternatives are working with MooseFS (which has challenges), figuring out how to work up CEPH in this environment (8 hours spent and nothing is working), or to find two KVM machines _with_ IPv4 for less than €19.50/year in total (which is what room is left in the budget).

        I’m still trying to decide.

        May 5, 2015 @ 11:21 am | Reply
  12. thanks..Great tutorial! :

    November 22, 2015 @ 1:07 am | Reply
  13. Great tutorial! Thanks :)

    April 20, 2016 @ 5:09 pm | Reply
  14. Great tutorial! :)
    Thanks you.

    April 20, 2016 @ 5:11 pm | Reply
  15. Your tutorial is very cool. Can you tell me about vps best for beginners ? Thanks

    December 21, 2016 @ 4:52 pm | Reply
  16. very helpful tutorial, many thanks

    February 22, 2017 @ 4:46 am | Reply
  17. Thanks for the tutorial !!

    July 22, 2017 @ 9:42 am | Reply

Leave a Reply

Some notes on commenting on LowEndBox:

  • Do not use LowEndBox for support issues. Go to your hosting provider and issue a ticket there. Coming here saying "my VPS is down, what do I do?!" will only have your comments removed.
  • Akismet is used for spam detection. Some comments may be held temporarily for manual approval.
  • Use <pre>...</pre> to quote the output from your terminal/console, or consider using a pastebin service.

Your email address will not be published. Required fields are marked *