The tutorials are back! After a period of absence it’s time to get the tutorials back on track and I’m going to start of with a great series, and a challenging one on that.
With all these LowEndSpirit-style offers these days it’s extremely cheap to get a whole bunch of servers quite quickly. For less than $50 a year you can have at least 10 servers in two continents across 10 different cities. That’s pretty awesome, right? But what do you do with all those servers?
That’s where my (*cough*) brilliant idea comes in: we’re going to create a cluster out of them! And not just any cluster, a redundant cluster! Oh well, if we make it anyway. Let’s first get somewhere before I start bragging. So, up to part 1 of N parts…
The ingredients
From what I can think of right now, this is what kind of servers you need for this cluster (taking into account redundancy):
- 2x VPS with IPv4 and IPv6 (BudgetVZ offers these for a low fee) for load balancers
- 2x VPS with just IPv6 for MySQL
- 2x VPS with just IPv6 for Web Nodes
- 3x VPS with just IPv6 for a filesystem
The above is just an indication, I may need to adapt as we go along. More is always possible, less is making it problematic. We need the IPv4 on the load balancers in order to get dual stack easy access to whatever you’re going to host on there.
For now, my plan is as follows:
- Host a WordPress site on the cluster
- Use MariaDB as a database server
- Use Apache as an application server on the web nodes
- Use NGINX as a load balancer and eventually to server/cache static files
- Use MooseFS or CEPH as a filesystem (depending on which one I can get to work)
So, if you’re in for the challenge, let’s get those servers (if you don’t have them already) and get cracking!
Oh, one more thing: you may want to get KVM machines or OpenVZ machines with FUSE for those filesystem nodes. I’d go for KVM as that broadens your options in case you’d like to switch in the future (to say, GlusterFS once they finally support IPv6…).
The MariaDB cluster
This is the easiest step… for me! As I’ve already written this tutorial and only had to make some tweaks. So, fire up a new tab and go to my original tutorial!
Follow this tutorial step-by-step, taking the following notes and changes into account:
bind-address
The bind-address should listen to any interface, not just to IPv4. So instead of this:
bind_address = 0.0.0.0
Use this:
bind_address = ::
As strange as this may look, but that will make the server listen on all interfaces. After having made that change, be sure to restart MariaDB:
sudo service mysql restart
IPv6 instead of IPv4
Wherever I’ve used an IPv4 address, use the corresponding IPv6 address of the server(s). Make sure they stay within the quotes, otherwise you’ll break stuff.
If you’re running these commands after the fact, flush the privileges to make sure it’s all set up correctly. From within MariaDB:
FLUSH PRIVILEGES;
And you’re good!
That was it for this week. Next time (in about two weeks) I want to focus on the web nodes and how we’re going to set those up.
Any feedback thus far, or suggestions, feel free to let me know!
Related Posts:
- How to Rapidly Install Java, OpenJDK & Oracle JDK on your VPS - December 14, 2015
- It’s been a great ride - December 14, 2015
- Cheap Windows VPS – $21/quarter 1GB KVM-based Windows VPS in 11 worldwide locations - November 30, 2015
Great tutorial! :)
What is your personal favorite vps seller?
What do you mean budgetvz offers those for free? Offer dual stack for free?
Looking forward to see how you set up redundancy. I have been working on something similar with a VPS and a dedicated server. Hardest part is trying to keep the files in sync.
Probably he means that they offer free load balancers.
He obviously meant “IPv4 and IPv6″…
> Use MooseFS or CEPH as a filesystem (depending on which one I can get to work)
Try also GlusterFS, it seemed to be the easiest to set up among all of them.
It is supposed to be, but it doesn’t play with IPv6 and/or custom ports. I’ve spent hours on trying to set it up only to find out it is not possible without IPv4.
They will support IPv6 in the future, though.
I’d suggest swappping Apache with Nginx, and then replacing NGINX as a load balancer for Varnish, which is blazing fast. IMO performance would be much better with such combination, don’t you think?
I am going to replace Apache with NGINX indeed. I am sticking to NGINX for load balancers as well, though, as Varnish is a whole different sport :-)
so, we need setup MariaDB cluster from begining again if one of the cluster down?
I’m not sure what you mean.
What minimum size VPS’s do you recommend for the different server types?
Obviously the size of file, web, and database servers will ultimately be determined by the amount of content, but to get starter, what minimum sizes are you looking at, particularly fro the load balancers?
is it also possible with 3 or 4 vps’s?
i don’t really need redundancy
and, where do i get 9 vps for $50/year?
Hi, the same question where did you get all of these with $ 50/ year
Quite simple to get 9 VPS Servers for $50 a year.
LowEndSpirit.com – 13 Locations @ 3 euro per year 3 * 13 = 39euro (based on todays rates) = $42.41USD – Then if needed to get a small VPS with a Dedicated IPv4 for frontend, there are a few in the ~8USD a year bracket.
these are all NAT servers
and in the tutorial hes talking about KVM servers
These servers are NAT servers except for the BudgetVZ ones, these include an IPv4. There are three KVM machines which littlehappycloud.net provides.
Calculating the cost:
2x BudgetVZ 128MB = €8/year
3x KVM 128MB = €16.50/year
2x MegaVZ 512MB = €11/year
2x LowEndSpirit 128MB = €6/year
Grand total: €41.50/year = $46.25/year
So, there’s room in the budget _and_ I’ve picked two instances with more than 128MB RAM for database servers.
Actually, while thinking about this, let’s bring up the following:
Clustered file systems in this setup are still a pain, mostly because they are built for enterprise environments and rely on IPv4 support. In addition to that, they require some level of access to the OS that’s hard to obtain in a commercial OpenVZ environment (something with admin privileges). That last bit is the reason I went for KVM, that first bit is why I’m now looking for a working solution.
The easiest and most LowEnd way would be SSHFS and rsync. Manual intervention is still part of this setup and the DB only replicates and doesn’t fail-over. I call this plan B for now.
Alternatives are working with MooseFS (which has challenges), figuring out how to work up CEPH in this environment (8 hours spent and nothing is working), or to find two KVM machines _with_ IPv4 for less than €19.50/year in total (which is what room is left in the budget).
I’m still trying to decide.
thanks..Great tutorial! :
Great tutorial! Thanks :)
Great tutorial! :)
Thanks you.
Your tutorial is very cool. Can you tell me about vps best for beginners ? Thanks
very helpful tutorial, many thanks
Thanks for the tutorial !!