A firewall is a security tool that blocks network traffic, with many different configuration options. For example, you could configure your firewall to block all traffic except attempts to connect to port 80 and 443 (for a web server) as well as your ssh port. This is an example of the best practice of “blocking everything except what is explicitly allowed”.
However, even with this sort of policy in place, the security gains for a basic firewall are limited. If you only have a web server and sshd server running, then there is nothing to block because connecting to a different port will fail anyway. Chiefly the benefit from this kind of simple firewall is to prevent inadvertent opening of ports. For example, let’s say you install a piece of software that has a dependency that starts up a service you hadn’t intended to run, or perhaps you change configurations an accidentally set MySQL to expose itself on your public IP. With a basic firewall, traffic to those port will still be blocked so you benefit from this kind of safety net.
Read more to continue…
However, the real power of firewalls becomes evident when we deploy an active firewall that intelligently monitors traffic and blocks traffic to/from specific clients. For example:
- If you are running an IMAP service, you have no choice but to have it publicly-facing where it may become a magnet for people trying to brute-force (guess passwords). If you deploy an active firewall, the firewall software will watch the IMAP logs and after a configured number of failed logins, the client IP trying to connect will be temporarily blocked.
- Likewise, you could have someone trying passwords on your ssh ports all day but they’ll be quickly stopped by an active firewall after a few failed logins.
- Although an active firewall cannot stop a distributed denial of service, it can end some limited denial of service attacks that are coming from a small number of attacking hosts.
Active firewalls are critical when you have less sophisticated users on your system, such as if you’re running a web host or mail server. While you may conscientiously pick good passwords, use SSH keys, etc., your more naive users might not. An active firewall helps protect you by eliminating brute-forcing.
Installing CSF
There are number of different products but one of the best known is ConfigServer Firewall.
To set it up, first install some required perl modules. On Debian-based systems:
apt-get install libwww-perl liblwp-protocol-https-perl libgd-graph-perlOn CentOS-based systems:
yum install perl-libwww-perl.noarch perl-LWP-Protocol-https.noarch perl-GDGraphIf you’re running CentOS, that distro’s native firewall (firewalld) comes pre-enabled. You’ll want to disable it before setting up CSF:
systemctl stop firewalld systemctl disable firewalld
Now download and extract the tarball:
cd /usr/src wget https://download.configserver.com/csf.tgz tar xzf csf.tgz cd csf sh install.sh
Next make sure you have all the required kernel modules:
# perl /usr/local/csf/bin/csftest.pl Testing ip_tables/iptable_filter...OK Testing ipt_LOG...OK Testing ipt_multiport/xt_multiport...OK Testing ipt_REJECT...OK Testing ipt_state/xt_state...OK Testing ipt_limit/xt_limit...OK Testing ipt_recent...OK Testing xt_connlimit...OK Testing ipt_owner/xt_owner...OK Testing iptable_nat/ipt_REDIRECT...OK Testing iptable_nat/ipt_DNAT...OK RESULT: csf should function on this server
In this case, all required kernel modules were present. On your VPS you might see an error or missing module but as long as you see the result that “csf should function on this server,” you’re good to go.
Configuring CSF
CSF starts in “TESTING” mode. This is so you do not accidentally lock yourself out.
Take a look at /etc/csf/csf.conf. You probably want to adjust the following:
- TCP_IN has a list of ports you allow. Remove any services you’re not using. For example, if you’re not running an FTP server, you can remove ports 20 and 21. If you’ve changed your SSH port, make sure it is in this list and remove port 22.
- TCP_OUT should match TCP_IN
- You can probably pare down UDP_IN and UDP_OUT, perhaps just to port 53 (DNS)
- If you’re not using IPv6, set IPV6 to 0. Otherwise, adjust TCP6_IN, TCP6_OUT, UDP6_IN, and UDP6_OUT to match the ipv4 versions.
- You will be mailed for each blocked IP. You can modify the templates in /etc/csf/alerts to set an appropriate To: address, or you can set LF_ALERT_TO to one master email address.
There are many, many more ways to customize CSF. You should take a look through the documentation on configserver.com to see the plethora of CSF capabilities.
Starting CSF
By default, CSF is in TESTING mode. Before you start it for real, it’s handy to pre-set a time to turn it off. You could use a crontab entry like this:
*/10 * * * * systemctl stop lfd ; systemctl stop csfDon’t forget to remove this crontab entry when you are ready to use CSF for real!
This will disable CSF entirely every 10 minutes on the 10s (so 1:00, 1:10, etc.) You can then fire up CSF with the knowledge that if you lock yourself out, you will have to wait a maximum of 10 minutes to get back in. Alternatively, you can login from your VPS provider’s console.
When you’re ready to go, modify /etc/csf/csf.conf and set TESTING to 0. Then:
systemctl restart csf systemctl restart lfd
Testing CSF
Here’s a test of CSF so you can see how it works. I’ve replaced the IP of the server with ‘s.s.s.s’ and the IP of the client with ‘c.c.c.c’.
I ssh’d with a bogus user to the VPS:
$ ssh nonexistant@s.s.s.s nonexistant@s.s.s.s's password: $ ssh nonexistant@s.s.s.s nonexistant@s.s.s.s's password: $ ssh nonexistant@s.s.s.s nonexistant@s.s.s.s's password: $ ssh nonexistant@s.s.s.s nonexistant@s.s.s.s's password: $ ssh nonexistant@s.s.s.s nonexistant@s.s.s.s's password:
CSF (specifically the lfd daemon) detected someone trying to brute-force a login by watching the system logs. In /var/log/lfd.log this entry appeared:
Apr 7 16:31:23 debian10 lfd[6780]: (sshd) Failed SSH login from c.c.c.c (US/United States/some.example.com): 5 in the last 3600 secs - *Blocked in csf* [LF_SSHD]CSF then created a firewall rule to block the IP. It also made a notation in /etc/csf/csf.deny (so that if the system restarts, the firewall rule is recreated):
c.c.c.c # lfd: (sshd) Failed SSH login from c.c.c.c (US/United States/some.example.com): 5 in the last 3600 secs - Tue Apr 7 16:31:23 2020
Everything after the # is a comment so you know why this IP was blocked. CSF also looks up the location of the IP address (this is not 100% perfect) and notes it.
Behind the scenes, CSF is manipulating the kernel’s firewall rules to create blocks. You can see the entire CSF ruleset at any time by issuing this command:
csf -lYou may also see them in dmesg and /var/log/messages, depending on your syslog config.
If you configured an email address as mentioned above, you’ll get an email like this one:
Time: Tue Apr 7 16:31:23 2020 -0700 IP: s.s.s.s (US/United States/some.example.com) Failures: 5 (sshd) Interval: 3600 seconds Blocked: Permanent Block [LF_SSHD] Log entries: Apr 7 16:31:07 debian10 sshd[6722]: Invalid user nonexistant from c.c.c.c port 42858 Apr 7 16:31:08 debian10 sshd[6722]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=c.c.c.c Apr 7 16:31:10 debian10 sshd[6722]: Failed password for invalid user nonexistant from c.c.c.c port 42858 ssh2 Apr 7 16:31:13 debian10 sshd[6722]: Failed password for invalid user nonexistant from c.c.c.c port 42858 ssh2 Apr 7 16:31:18 debian10 sshd[6722]: Failed password for invalid user nonexistant from c.c.c.c port 42858 ssh2
After receiving this email I tried sshing from the same client:
$ ssh nonexistant@s.s.s.s…and there was an infinite pause. Here I am blocked not just from SSH but at the IP level, so nothing from my client would be able to connect to this server (web, FTP, etc.)
Enabling CSF
To enable the product, issue these commands:
systemctl enable csf systemctl enable lfd
Again, if you’ve put any kind of temporary disabling command in cron during testing, be sure to remove it before relying on the firewall.
Living with CSF
An active firewall like CSF is a wonderful security tool but as always, security is the inverse of convenience. There are two headaches that are common: a flood of block notifications and accidental blocks.
Reducing Block Notifications
At first, you may be surprised at how many emails you get about blocked IPs. Welcome to the reality of the public Internet! Previously you were in blissful ignorance – now you can see how many attempts to attack your box are routine every day.
If you’re running a web host, then there are many ports that you are kind of stuck with because users expect FTP on port 20/21, email on 25/587, etc. However, you might consider changing your SSH port.
This advice is mildly controversial because changing your SSH port does not prevent someone from finding it and trying to login on that port. However, it will radically cut down how many automated attacks you get. Many attackers will mass-scan IPs and hone in on those that answer on port 22. If your system doesn’t respond on port 22, they will move on to another host.
Some sysadmins take the point of view that once they’ve secured the box by allowing logins only via ssh key and disabling root logins, there is little to fear from script kiddies wasting their time banging on the SSH port. While this is true, the numerous emails you’ll receive daily are tedious and may drown out alerts you truly do want to review.
You can change sshd to run on any port, but ideally an unused high port (above 1024).
For example, let’s say you wanted to use port 32222. To do this, modify /etc/ssh/sshd_config:
# Port 22 Port 32222
In this case, we’ve commented out the default and added our port.
Next, be sure to update TCP_IN and TCP6_IN in /etc/csf/csf.conf to both remove port 22 and add your custom port.
Then restart sshd:
systemctl restart sshd
Your ssh command from your client will now look something like this if your account is ‘joe’:
ssh -p 32222 -i my_ssh_key joe@myvps.example.comHandling Mistaken Blocks
When CSF blocks a client IP, it adds a firewall rule and also writes it to/etc/csf/csf.deny. If a client is accidentally are blocked by CSF, you can immediately unblock them with this CSF command (here I’m using ‘c.c.c.c’ again for the client’s IP). This will both remove the kernel’s firewall rule and clear it from /etc/csf/csf.deny:
# csf --denyrm c.c.c.cRemoving rule... DROP all opt -- in !lo out * c.c.c.c -> 0.0.0.0/0 LOGDROPOUT all opt -- in * out !lo 0.0.0.0/0 -> c.c.c.c
You can also whitelist any IP by adding it to /etc/csf/csf.allow. Note that CSF whitelists the IP you’re connected from when you first set the product up. You’ll see an entry like this in /etc/csf/csf.allow:
c.c.c.c # csf SSH installation/upgrade IP address - Tue Apr 7 22:56:37 2020
Consider whitelisting your home IP and/or any VPN IPs you use to connect.
Related Posts:
I also enjoy using LEBs! I have an empire of little guys for VPNs, larger guys for apps, and big guys for databases and storage. When I'm not in front a screen I'm into German Shepherd dogs, quality knives, target shooting, theology, tabletop roleplaying games, and forest hiking.
I enjoy writing technical articles here on LowEndBox to help people get more out of their systems. You can find me on LowEndTalk @raindog308
- Merry Christmas from LowEndBox! - December 25, 2024
- We are Social Butterflies!Check Us Out Wherever You Browse, View, or Tap! - December 23, 2024
- Let’s Celebrate the Winter Solstice with Awesome Deals and a Free Bonus Code for RackNerd’s Giveaway! - December 22, 2024
Nice security blog post, Andrew.