Use GUIX to fix an upstream package issue

I am using a Talos II PowerPC (ppc64le) machine as my daily computer. This poses some challenges every now and then as the support for that architecture is less ubiquitous than the default x86-64. It's the price to pay for having a completely open, documented machine.

On this system I run archpower distribution to get a linux kernel onto the machine, but I prefer to run GUIX on top of it for package management. I still need to run the native pacman package manager though, some packages are either not in GUIX yet (not so much of a problem in practice) or do not support ppc64le at all which is a bigger problem.

There's a third category, which I'd like to write about in this post. Packages that do support the architecture, but it's not a first class citizen. With this I mean that support is there, but the PowerPC package gets less attention and testing. This is somewhat inevitable as far less people use these machines rather than a normal pc, so it's likely they won't get as much eyes.

Depending on your distribution and package manager you can get stuck on a certain version for a package if there are limited options to upgrade or building the package locally is an effort out of your reach or time availability. Often the only option, for me anyways, is waiting for upstream to fix the situation. For this reason I still can't have packages that depend on rust in GUIX.

Because GUIX is basically a scheme library, you can use it to define how packages get into your machine and there is usually a better option to create a solution without having to figure out all the build details of the package itself.

One example of this is a recent ppc64le specific issue with OpenSSH.

The problem was the release (9.6p1) which was packaged for GUIX, and got eventually into my system on a GUIX pull command. However, the package build failed, apparently only on ppc64le machines due to this issue.

Because OpenSSH is used by many other packages, its failing build affected all those packages.

Here's what I did in GUIX to get to the new OpenSSH version which contained a number of security fixes which I wanted to have.

First, I created a package definition called openssh-next which takes the existing openssh package in GUIX and inherit from it in such a way that the fix for the problem outlined above will be included. In this case, take the commit just after the release from upstream.

 1  ;; Define openssh-next package which takes openssh from upstream which has the fix applied
 2  ;; See https://github.com/openssh/openssh-portable/commit/1036d77b34a5fa15e56f516b81b9928006848cbd
 3  (define-public openssh-next
 4    (let ((xcommit "1036d77b34a5fa15e56f516b81b9928006848cbd"))
 5      (package
 6        (inherit openssh)
 7        (name "openssh-next")
 8        (version "9.6p1-1")
 9        (native-inputs
10         (list autoconf
11               automake
12               pkg-config))
13        (source
14         (origin
15           (method git-fetch)
16           (uri (git-reference
17                 (url "https://github.com/openssh/openssh-portable.git")
18                 (commit xcommit)))
19           (file-name (git-file-name name version))
20           (patches (search-patches "openssh-trust-guix-store-directory.patch"))
21           (sha256
22            (base32 "1sary1ig972l4zjvpzncf9whfp5ab8snff2fw9sy5a8pda5n2a7w")))))))

The crux in the above snippets is the inherit line which hides all the package definition complexity and the adapted source block which takes a specific git commit from the OpenSSH repository. Another way would be to add an extra patch line in the source block which contains just the upstream fix for ppc64.

With this new package definition, the new version can be installed, but all packages which depend on openssh are still using the original version. We want some way to go over everything that depends on openssh and replace its dependency with the new openssh-next package.

This is where GUIX can make your life easier. The GUIX api provides a couple of ways to do this. I built it up around the functions package-input-rewriting/spec and package-mapping.

The first takes a list of replacements, where each element of the list is a pair of a package spec and a procedure passed with a package to replace it with.

The second is a function to apply a function to a package to apply the defined replacement to a package. I wrapped both functions to make the call for the relevant packages a bit simpler.

 1  ;; Given a package spec, Replace input `old` with `new` for that package incl. its dependents
 2  ;; Return a procedure which takes the package as parameter
 3  (define (package-input-replace old new)
 4    (package-input-rewriting/spec
 5     `((,old . ,(const (specification->package new))))))
 6
 7  ;; Apply the input replace for openssh
 8  ;; pass each package which fails to build due to openssh as dependency
 9  (define (openssh-fix package)
10    ((package-mapping
11      (package-input-replace "openssh" "openssh-next"))
12     (specification->package package)))
13
14  ;; Two examples on what to put in the manifest
15  (openssh-fix "gvfs")
16  (openssh-fix "remmina")

With this solution in place, the whole local package set builds again (takes a while though, as it often does with GUIX) and I can take advantage of the new OpenSSH release. There is a bit of 'keeping an eye on it' involved from this point on though. If upstream fixes the issue I want to take the above scheme code out of my manifest again and start using the upstream openssh package again instead of my openssh-next definition.

Jaguar XKR leather interior restoration

I had the leather interior of my Jaguar XKR restored last week and the supplier provided me with a video of it; in the "before/after" style. Rob did an amazing job on the whole of the interior.

Process DMARC reports with sieve

I get a lot of DMARC reports because I host mail for a couple of domains. Most of these mails require no attention as they are just notifications that others use one of our domains. I want to separate these mails from my normal mail workflow and auto archive them if I haven't looked at them within, say, 2 weeks.

Doing this with sieve server-side has my preference, but apparently it's not trivial to determine the age of a message, which is the core logic needed here. Also, the processing of sieve rules is normally only during reception of messages, not ad-hoc or on some other event, although dovecot and pigeonhole have some options for this, among others the sieve-filter tool.

I really only found one implemenation online which roughly solves the same problem I was having, but this involved more than needed I think.

My solution consists of 3 parts:

1. the sieve script that handles DMARC reports on reception and age-ing;
2. use of an extension that calls an external program to evaluate expressions to determine age;
3. a daily job that runs the sieve script in the scope of the designated folder.

Here's the sieve script which deals with DMARC reports both in the normal INBOX flow and a special treatment after 14 days. The latter part is not automatic by dovecot on reception of emails, but triggered by a run of the sieve-filter program.

 1    require ["date","fileinto","relational","variables","environment","imap4flags",
 2             "vnd.dovecot.execute", "vnd.dovecot.environment"];
 3
 4    # Parameters
 5    set "dmarc_folder" "Folder.for.dmarc-reports";
 6    set "purge_days" "14";
 7
 8    # Move DMARC notifications when received
 9    if environment :is "vnd.dovecot.default-mailbox" "INBOX" {
10      if anyof (
11        header :contains "From" "dmarcreport@microsoft.com",
12        header :contains "From" "noreply-dmarc-support@google.com",
13        header :contains "From" "opendmarc@mail.arctype.co",
14        header :contains "From" "opendmarc@box.euandre.org"  )
15      {
16        addflag "\\Seen";
17        fileinto "${dmarc_folder}";
18        stop;
19      }
20    }
21
22    # When running in the dmarc_folder, archive when age is <purge_days>
23    if environment :is "vnd.dovecot.default-mailbox" "${dmarc_folder}"
24    {
25      if currentdate :matches "julian" "*"
26      {
27        # Run a simple bc expresssion to get <purge_days> ago from todays julian day
28        execute :output "purge_date" "bc" "${1} - ${purge_days}";
29
30        # Compare this with Date header and archive when age reached
31        if date :value "le" "Date" "julian" "${purge_date}"
32        {
33          fileinto "Trash";
34          stop;
35        }
36      }
37    }

The first part of the sieve script just moves the mails into the dmarc-reports folder and is a normal sieve processing rule. The second part runs if the default folder is the dmarc-reports folder. If so, it uses the ext_program extension of the sieve interpreter to let the bc program evaluate the expression for the age of the message.

This uses a tiny script in the configured sieve execute bin directory of the ext_programs extension

1  #!/bin/sh
2  echo ${1} | /usr/bin/bc

which just pipes the input given by the sieve line into the bc program. On returning, stdout is put into the purge_date variable. I'm using execute because I do not need to pipe the whole message into the external program, but specify input specifically.

With the above configuration I can set a cron job in the crontab of the vmail user to run

1  sieve-filter -We -u <mymailaccount> \
2               /path/to/vmail/mymailaccount/sieve/dmarc-archiver.sieve \
3               Folder.for.dmarc-reports

which executes the sieve script mentioned above in the IMAP folder <dmarc_folder> only.

I'm not sure why sieve makes it so difficult to get the age of an email (unless I'm missing something). Protonmail solves this by having a custom extension 'vnd.proton.eval' which does something similar like the above, but in the scope of the sieve language itself without having to shell out to an external program explicitly. (I think; I have not seen their implementation)

My approach above obviously has some drawbacks:

• the bc external program is called for every mail that matches, fine for 10 or 20 I guess, but rather inefficient if the amount of matched messages is big. For now, not a problem.
• unsure what sort of security consequences this has, the execution scope and environment is very limited, but we're still giving control to a script calling other programs.

Sunset on dutch beach

Hike: Katwijk aan Zee - De Zilk LAW 05-2 (04)

This post is the result of a little hike along one of the Long Distance walks in the Netherlands: "LAW 05-2 (04) Katwijk aan Zee - de Zilk", part 4 of "Nederlands Kustpad"

I recorded the hike with my Suunto Traverse watch and exported this as a GPX file. I used that gpx file as example data to create a little gpx viewer which can be embeded in html pages. With Leaflet this is almost trivial to do.

The details of the hike are on my Movescount page for this move.

If you zoom in on the map, you'll see that the recorded path is not an exact match with the followed path (the dotted lines). This is because I used a long interval of GPS fixes to record the path. I think it was a 10 second interval, so there's some tuning to do for this in future.

The exported GPX has information about temperature, height, energy, speed and possibly some other stuff. My intention is to have this information incorporated in the map later on, similar to the way this is displayed on the movescount site.

If you're interested, the GPX file is here:

Move_2016_11_06_09_16_34_Hiking.gpx