Today I was browsing Hacker News and reading a discussion about sites requiring age verification through photo uploads.  Down in the thread, someone posted a link to a jwz.org blog post.  That site belongs to Jamie Zawinksi, a rather famous programmer who was on the team that developed Netscape Navigator way back in the day.

He can also be cantankerous and it turns out that he doesn’t like Hacker News.

If you click on a jwz.org link from HN, you’re greeted with this lovely image:

I mean, he’s not wrong…

jwz also sets a cookie so all future visits to jwz.org – even if you type “jwz.org” into your browser location bar – result in that image.

These are both trivial to overcome.  Delete the cookie and the problem goes away, and if you want to visit a link you see on HN, copy it and paste the URL.

Who Needs HTTP Referer Anyway?

So how does this work?  How does jwz know you came from Hacker News?

The secret is HTTP Referer.  HTTP referer was introduced via RFC 1945, which introduced HTTP 1.0, a protocol that just turned 30 years old (May 1996).

It inserts a link like this into your request header:

Referer: https://example.com/page

Honestly, if a technology can’t even spell itself correctly, who needs it?  (“Referrer” is the proper spelling but historically the technology is – ahem – referred to as referer).

HTTP Referer is a privacy leak that only benefits analytics.  If I’m reading a site and click a link that takes me to example.com, it tells the example.com proprietor what other sites I read.  Why is that necessary?  As a user, this doesn’t benefit me in the least.  It’s nice for example.com’s analytics, but I’d just rather not.

How to Turn HTTP Referer Off

For the Chrome browser, there’s no native way to turn it off.  Not surprising, I guess, given who makes it.

For Firefox, go to about:config and network.http.sendRefererHeader and change it:

• 0 = never send
• 1 = send only on clicked links
• 2 = send on links and images (default)

I couldn’t find an out-of-the-box way to turn it off on Safari.

For both Chrome and Safari, you could investigate various privacy extensions and see what they provide.  Those can sometimes be a jungle of options.

Why Can’t We Just Ditch HTTP Referer?

I’m sure browser makers will say they’re just conforming to standards.  Maybe…but maybe we should ditch that one.

In theory you could break things by disabling this.  For example, your bank might say “wait, you’re going to /SomeSecurePage and the only legit way to get there is to come from /LoginPage and so we’re going to deny that.”  But today, most sites use CSRF tokens and cookies instead of HTTP Referer.

Historically, HTTP Referer was used to prevent hotlinking.  That’s when you have an image on your site that the Internet likes, so everyone does an img src with your URL.  You pay the bill, but the image shows up on other web sites.  Today, everyone uses CloudFlare anyways.

HTTP Referer sucks.

raindog308

raindog308 is a longtime community LETizen, technical writer, and self-described techno polymath. With deep roots in the *nix world, he has a passion for systems both modern and vintage, ranging from Unix, Perl, Python, and Golang to shell scripting and mainframe-era operating systems like MVS. He’s equally comfortable with relational database systems, having spent years working with Oracle, PostgreSQL, and MySQL.

As an avid user of LowEndBox providers, raindog308 runs an empire of LEBs, from tiny boxes for VPNs, to mid-sized instances for application hosting, and heavyweight servers for data storage and complex databases. He brings both technical rigor and real-world experience to every piece he writes.

Beyond the command line, raindog308 has a life-long love of German Shepherd Dogs, high-quality knives, target shooting, theology, tabletop RPGs, playing guitar, and hiking in deep, quiet forests.

His goal with every article is to help users, from beginners to seasoned sysadmins, get more value, performance, and enjoyment out of their infrastructure.

You can find him daily in the forums at LowEndTalk under the handle @raindog308.