DoT w/ Stubby & dnsmasq on macOS

OK, let’s start with DoT definition:

DNS over TLS (DoT) is a security protocol for encrypting and wrapping Domain Name System (DNS) queries and answers via the Transport Layer Security (TLS) protocol. The goal of the method is to increase user privacy and security by preventing eavesdropping and manipulation of DNS data via man-in-the-middle attacks.

There are, of course, multiple ways to use DoT on macOS.1 I took the approach of dnsmasq + Stubby simply because I had already planned playing with the former — editing /etc/hosts has bitten me so many times, that I finally gave up and decided to use some tiny DNS instead. Additionally I wanted to get rid of at least some trackers, counters, analytics and, potentially, ads.

Read more “DoT w/ Stubby & dnsmasq on macOS”

Load Balancing PHP-FPM with HAProxy and FastCGI →

The better alternative is to run the script in its own process and leave the task of receiving HTTP requests to the web server. FastCGI allows you to separate the web server (or proxy) and the running script by defining the communication protocol between the two. Performance benchmarks indicate that separating scripts into their own process equates to a boost in performance.

That was always one of the key differences between Apache and NGINX. Doesn’t really apply these days, but while mod_php is still doing well, NGINX has never adopted anything like it and kept being simply reverse-proxy and cache things. I much prefer NGINX’s approach and it’s great to see that HAProxy can now be used as a viable alternative.

There’s still value using web server for this kind of task, but now it really depends on the software stack already in place and needs. While caching and additional, web server specific, functionality makes more sense in production environment, for testing or maybe even staging purposes HAProxy may be more than enough.

Kernel 5.6 →

As mentioned the last time, I’m still tracking and building latest kernel releases. 5.6 was on my radar for some time and I’m going to stick to it for longer period. One of the most important features1 of this release is addition of WireGuard. I was building it in for the older releases but now this step and patch is no longer necessary as it’s part of mainline 🎉

One of the differences though is the place where it should be enabled:

-> Device Drivers
  -> Network device support
    -> WireGuard secure network tunnel

Latest bcc (version 0.14.0) released recently added support for kernels up to 5.6 as well.

TLS version and ciphers from NGINX in ELK →

Claudio Kuenzler:

In the past 20 years we have moved from SSL to TLS (yet we’re still talking about SSL certificates, funny isn’t it). According to Wikipedia’s Transport Layer Security page, this is the release history:

SSL 1.0 -> Never officially released due to security flaws
SSL 2.0 -> 1995, deprecated in 2011
SSL 3.0 -> 1996, deprecated in 2015
TLS 1.0 -> 1999, deprecated in 2020
TLS 1.1 -> 2006
TLS 1.2 -> 2008
TLS 1.3 -> 2018

It’s crazy to think that the modern TLS 1.2 is already 12 years old. Not long ago I dropped everything older than 1.2 and introduced support for 1.3 + I’m using only modern ciphers. But this is easily done on a blog with almost no traffic.


Building kernel w/ ZFS & perf on Ubuntu

One of my New Year’s resolution was to get back a bit closer to the lower level parts of Linux. And what’s there lower than the kernel itself? I always preferred vanilla kernel, even when I was fooling around with Gentoo, and this hasn’t changed. In December I started with preparing first builds. Nothing too fancy as it’s only for my personal usage, but I still find it worth going through. I hit four issues I needed to fix in order to be able to use these kernels on my machines:

  1. ZFS has to be supported (as I use it on my servers).
  2. Wireguard has to be built-in (cause vanilla releases occur way more often than the distro provided ones and re-running dkms each time makes no sense to me).1
  3. perf has to be part of the build.2
  4. Entire process has to be streamlined and possibly handled by some kind of CI.

Read more “Building kernel w/ ZFS & perf on Ubuntu”

NGINX Extended Security Update #2 →

NGINX before 1.17.7, with certain error_page configurations, allows HTTP request smuggling, as demonstrated by the ability of an attacker to read unauthorized web pages in environments where NGINX is being fronted by a load balancer.

That’s the essence of the CVE-2019-20372. Yet again, as I mentioned in my NGINX Extended post I was not going to work on 1.14.x branch any more with the exception of security updates — this is the case for such exception. Both 1.14.x for Xenial (16.04 LTS) and 1.16.x for Bionic (18.04 LTS) were patched against this vulnerability and are available from my PPA. On Docker Hub I bumped up only the 1.16.x branch as usage for 1.14.x is pretty much non-existent.

Backporting BCC & bpftrace

I’m following Brendan Gregg’s performance-related content for years now. I started when he was still in Joyent, later on I bought his Systems Performance book and I get back to it whenever I’m doing any profiling. Now I follow closely all of the latest work he’s doing on BPF front. There’s a small problem though. On the dependency tree of production-grade systems, I’ve got the following:

  • stable OS (Debian 10 “Buster” or Ubuntu 18.04 “Bionic Beaver”)
  • latest kernel (5.0+)
  • latest tracing tools (BCC & bpftrace)

Two first are easy — both Debian and Ubuntu are now providing ways for having relatively new kernel up and running — former with buster-backports1 and latter with HWE.2 The third one becomes a small hassle, at least in Ubuntu world.

bpftrace is currently not available for bionic at all. There’s no technical limitation here — it’s just the package was never built for it.3 I decided to backport it then.

Read more “Backporting BCC & bpftrace”

Dynamic upstreams in NGINX w/ Consul

I already briefly wrote about the idea of having dynamically discoverable upstreams in NGINX when I covered the topic of NGINX Extended. With the boom of microservices and containers scattered all over the place there was suddenly a need for something that would serve as a single source of truth. When solutions like Mesos/Marathon or Kubernetes kicked in, notion of having services statically assigned to particular address and/or port went straight to the trash. That’s exactly where Consul comes into play. I first crossed my paths with it years ago when it was both relatively new concept and software. These days I think it’s safe to say that, along etcd, it became industry’s standard. But even with its mature state, it solves only half of the problem — it registers and allows services to discover each other for variety of connection purposes, but if there’s anything that needs to serve as an application for HTTPs reverse-proxy, it has to be relatively static. Or does it?

Read more “Dynamic upstreams in NGINX w/ Consul”

Restic 0.9.6 →

Backups are one of those things that are usually afterthought. Maybe reason for that was a bit too much of necessary configuration or not enough sensible default choices to fit the bill in the older apps I’ve been trying. Either way — this small, single-binary go application simply nails it. All backups are automagically encrypted and deduplicated. These days I tend to setup backups and then just forget about them — they are taken over by restic and it performs its magic on my behalf. Saved my butt few times already.

This release brings no spectacular changes nor features. Which is good, I don’t want my backup solution to have exciting releases — I want bug fixes and small changes and minor enhancements to its matured, stable state. 0.9.6 bring exactly that 🎉