Backporting BCC & bpftrace

Posted on Published

I’m following Brendan Gregg’s performance-related content for years now. I started when he was still in Joyent, later on I bought his Systems Performance book and I get back to it whenever I’m doing any profiling. Now I follow closely all of the latest work he’s doing on BPF front. There’s a small problem though. On the dependency tree of production-grade systems, I’ve got the following:

  • stable OS (Debian 10 “Buster” or Ubuntu 18.04 “Bionic Beaver”)
  • latest kernel (5.0+)
  • latest tracing tools (BCC & bpftrace)

Two first are easy — both Debian and Ubuntu are now providing ways for having relatively new kernel up and running — former with buster-backports1 and latter with HWE.2 The third one becomes a small hassle, at least in Ubuntu world.

bpftrace is currently not available for bionic at all. There’s no technical limitation here — it’s just the package was never built for it.3 I decided to backport it then.

Read more “Backporting BCC & bpftrace”

NGINX Extended Security Update #2 →

Posted on Published

NGINX before 1.17.7, with certain error_page configurations, allows HTTP request smuggling, as demonstrated by the ability of an attacker to read unauthorized web pages in environments where NGINX is being fronted by a load balancer.

That’s the essence of the CVE-2019-20372. Yet again, as I mentioned in my NGINX Extended post I was not going to work on 1.14.x branch any more with the exception of security updates — this is the case for such exception. Both 1.14.x for Xenial (16.04 LTS) and 1.16.x for Bionic (18.04 LTS) were patched against this vulnerability and are available from my PPA. On Docker Hub I bumped up only the 1.16.x branch as usage for 1.14.x is pretty much non-existent.

Dynamic upstreams in NGINX w/ Consul

Posted on Published

I already briefly wrote about the idea of having dynamically discoverable upstreams in NGINX when I covered the topic of NGINX Extended. With the boom of microservices and containers scattered all over the place there was suddenly a need for something that would serve as a single source of truth. When solutions like Mesos/Marathon or Kubernetes kicked in, notion of having services statically assigned to particular address and/or port went straight to the trash. That’s exactly where Consul comes into play. I first crossed my paths with it years ago when it was both relatively new concept and software. These days I think it’s safe to say that, along etcd, it became industry’s standard. But even with its mature state, it solves only half of the problem — it registers and allows services to discover each other for variety of connection purposes, but if there’s anything that needs to serve as an application for HTTPs reverse-proxy, it has to be relatively static. Or does it?

Read more “Dynamic upstreams in NGINX w/ Consul”

Restic 0.9.6 →

Posted on Published

Backups are one of those things that are usually afterthought. Maybe reason for that was a bit too much of necessary configuration or not enough sensible default choices to fit the bill in the older apps I’ve been trying. Either way — this small, single-binary go application simply nails it. All backups are automagically encrypted and deduplicated. These days I tend to setup backups and then just forget about them — they are taken over by restic and it performs its magic on my behalf. Saved my butt few times already.

This release brings no spectacular changes nor features. Which is good, I don’t want my backup solution to have exciting releases — I want bug fixes and small changes and minor enhancements to its matured, stable state. 0.9.6 bring exactly that 🎉

NepTunes 1.7.0 →

Posted on Published

One of the things I really love about Mac is the availability of the nifty small utilities that aim at solving one problem and doing it right. Maybe it’s simply my admiration of UNIX philosophy.1 I’m an avid Last.fm user2 and I stumbled upon NepTunes while looking for something integrating iTunes with it. With the release of Catalina and introduction of the new Music.app, NepTunes stopped working and my scrobbles failed to be noted on Last.fm. With 1.7.0 release this is now fixed and everything is back to normal! 🎉

Ansible 2.9.0 →

Posted on Published

Ansible 2.9 “Immigrant Song” has just been released. Back in May I already mentioned how slowly but surely Ansible won my heart and I must admit that I’m constantly impressed with what can be done with this tool. It’s a huge release yet again, but there are no major changes — which is good. It means that the project focused on bug fixing and improving on things already present.

It also coincide with release of my new Ansible Role: Rsyslog. I’ve been working on it for past few days and found it good enough for shipping today. Should you look for something handling rsyslog on Ubuntu/Debian, give it a spin! 😊

Ni No Kuni Wrath of the White Witch →

Posted on Published

I’ve been playing Ni no Kuni II: Revenant Kingdom for a while now and I still think it’s the best RPG I played in years. I’m also a big fan of Studio Ghibli and while NNKII wasn’t done in collaboration, the style and inspiration is clear.

Some time ago I received as a gift Ni No Kuni Wrath of the White Witch DS version. Reason is simple — as I have 3DS I can play old DS games just fine. The problem was with… Translation. First Ni No was published with PlayStation 3 in mind, but due to large popularity of DS system in Japan at that time, it was ported there as well. I have a wonderful deluxe edition, with book inside and what not, but everything is in… Japanese.

I’ve been following this fan translation attempt1 for quite some time, but recently I noticed that there is a new, remastered version of Ni No Kuni coming to the PC! And now it’s available on Steam, fully translated! 🎉

Rolling out containers w/ Ansible

Posted on Published

I’m not huge on containers, but I can see and appreciate their value in rolling things out fast for testing purposes. I have my own server(s) running here and there and I use Ansible for handling pretty much everything on them. Until not long ago, Docker containers were among notable exceptions from that rule. But then I finally discovered1 docker_container module. The only thing I was still missing was better handling of defining multiple containers.2

Read more “Rolling out containers w/ Ansible”

NGINX Extended Security Update →

Posted on Published

There were three vulnerabilities discovered by Netflix in NGINX: CVE-2019-9511, CVE-2019-9513 and CVE-2019-9516. Both current stable1 and mainline2 were patched and point releases had been issued. As I mentioned in my NGINX Extended post I was not going to work on 1.14.x branch any more with the exception of security updates. Canonical backported patches to their nginx package3 with the following changelog:

  • SECURITY UPDATE: HTTP/2 Data Dribble issue
    – debian/patches/CVE-2019-9511.patch: limited number of DATA frames in src/http/v2/ngx_http_v2.c, src/http/v2/ngx_http_v2.h, src/http/v2/ngx_http_v2_filter_module.c.
    – CVE-2019-9511
  • SECURITY UPDATE: HTTP/2 Resource Loop / Priority Shuffling issue
    – debian/patches/CVE-2019-9513.patch: limited number of PRIORITY frames in src/http/v2/ngx_http_v2.c, src/http/v2/ngx_http_v2.h.
    – CVE-2019-9513
  • SECURITY UPDATE: HTTP/2 0-Length Headers Leak issue
    – debian/patches/CVE-2019-9516.patch: reject zero length headers with PROTOCOL_ERROR in src/http/v2/ngx_http_v2.c.
    – CVE-2019-9516

I also took these patches and rebuilt my NGINX Extended version.4

Having this opportunity I thought it’s worth mentioning that there’s also Docker container available with my NGINX version. Dockerfile is available on GitHub and image itself on Docker Hub. Internally it’s also using my PPA to provide the package so it has exactly the same version as the one provided there.