Planet Redpill Linpro

19 March 2018

Bjørn Ruberg

X-Forwarded-For DDoS

A discussion forum of one of Redpill Linpro‘s customers has been under attack lately, through a number of DoS and DDoS variants. Today’s attack strain was of the rather interesting kind, as one of its very distinctive identifiers was a suspicious, not to say ridiculous, amount of IP addresses in the incoming X-Forwarded-For HTTP header. […]

by bjorn at Mon 19 Mar 2018, 21:55

25 February 2018

Redpill Linpro Techblog

Layer 3 routing on Cumulus Linux MLAG

We build our network in order to simultaneously achieve high availability and maximum utilisation of available bandwidth. To that end, we are using Multi-Chassis Link Aggregation (MLAG) between our data centre switches running Cumulus Linux and our firewall cluster

Sun 25 Feb 2018, 23:00

16 February 2018

Tore Anderson

IPv6 roaming in the United States

I spent some time in the United States last month. Equipped with SIM cards from both Tele2 Sweden (MCCMNC 24007) and Telenor Norway (MCCMNC 24201), I set out to test IPv6 while roaming, as I usually do while abroad.

The previous posts in this series are:

There were two mobile networks that I was able to register in and test: AT&T (3G/4G, no 2G) and T-Mobile (2G/3G/4G). The test results are found below.

I could also see three other 4G networks show up in a network scan (Caprock Cellular, Sprint and Verizon), but none of those were available to me. Presumably neither Tele2 nor Telenor have roaming arrangements with any of those operators.

AT&T - MCCMNC 310410

Home PLMN Tech IPV6 PDP context IPV4V6 PDP context
Tele2 SE 3G Fails (cause code 33) IPv4-only (cause code 50)
Tele2 SE 4G Works perfectly Works perfectly
Telenor NO 3G Fails (cause code 33) IPv4-only (cause code 50)
Telenor NO 4G Works perfectly Works perfectly

T-Mobile - MCCMNC 310260

Home PLMN Tech IPV6 PDP context IPV4V6 PDP context
Tele2 SE 2G Fails (cause code 33) IPv4-only (cause code 50)
Tele2 SE 3G Fails (cause code 33) IPv4-only (cause code 50)
Tele2 SE 4G Works perfectly Works perfectly
Telenor NO 2G Fails (cause code 33) IPv4-only (cause code 50)
Telenor NO 3G Fails (cause code 33) IPv4-only (cause code 50)
Telenor NO 4G Works perfectly Works perfectly

3GPP cause code #33 means «requested service option not subscribed», while cause code #50 means «PDP type IPv4 only allowed». The latter doesn’t indicate a fatal error, as I do automatically get a working IPv4-only connection to the Internet.

The fact that IPv6 consistently fails on 2G/3G is in all likelihood due to Tele2/Telenor’s HLR removing the IPv6 capabilities from my subscriber profile before transmitting it to AT&T/T-Mobile’s vSGSN.

Tele2 and Telenor do this to forestall the possible IPv6-related failures described in RFC 7445 sections 3 and 6. In this case, it is in all likelihood an unnecessary precaution, considering that both AT&T and T-Mobile are known to have deployed IPv6 to their own customers.

Fri 16 Feb 2018, 00:00

07 February 2018

Magnus Hagander

What does it mean to be on the board of PostgreSQL Europe

With the upcoming elections in PostgreSQL Europe, I'm excited to see that we have more candidates than ever before. But during the FOSDEM conference we just finished in Brussels, that also lead to a fairly large number of people who asked me the simple question "what does it actually mean to be on the board of PostgreSQL Europe". So I think it's time to summarize that for both those standing for election, and for the members of the organisation in general.

For a TL; DR; version, being on the board basically means a lot of administrative work :) But read on for some details.

by (Magnus Hagander) at Wed 07 Feb 2018, 16:49

01 February 2018

Redpill Linpro Techblog

Mule 4 SDK custom connector example

Mule 4 will be released soon. Along with Mule 4 a new Mule SDK is relased which can be used to extend the functionality of Mule with custom modules. The Mule SDK replaces Devkit for developing connectors.

Documentation on the Mule SDK can so far ...

Thu 01 Feb 2018, 23:00

25 January 2018

Redpill Linpro Techblog

Jenkinsfile to Anypoint platform and Artifactory

Jenkins Pipeline is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins. I will be using this to deploy a Mule application to Anypoint Platform Runtime Manager and store the delivery in ...

Thu 25 Jan 2018, 23:00

23 January 2018

Sigurd Urdahl

Making it right all the time - or that time I disagreed with a distro package about file permissions

This was originally posted to my employer Redpill Linpro's annual pre-Christmas Sysadvent calendar of tech blogging

Distro packages are a blessing that most of us take for granted (thank you and sorry package maintainers everywhere!). They make installation, maintenance and even removal of both simple and complex software a breeze.

Sometimes you disagree

But sometimes you disagree with a decision made in the distro package. Or maybe just have a use-case that is specific enough that the generalized setup provided by the distro package collides with your needs.

A sometime

A while ago I had one such “sometime”. Let me explain: a lot of our customers here at Redpill Linpro Managed Services use Mysql. Among them is one of my customers, - the official Norwegian national education and career portal. From time to time their developers need to export data from Mysql to CSV file (that is comma seperated values). The best way to do get CSV from Mysql is to use SELECT .. INTO OUTFILE. The resultset from such an SELECT is dropped as a CSV file in the import/export directory configured with the variable secure_file_priv. runs on Ubuntu, and in Ubuntu the mysql-server-5.5 package creates a directory intended for this usage - /var/lib/mysql-files - in it’s ‘‘postinst’’ script:
# Lets extract the scripts (and then some) from the mysql-server-5.5 package:
root@dbserver:/tmp/mysql-server-5.5_control# dpkg --control /var/cache/apt/archives/mysql-server-5.5_5.5.58-0ubuntu0.14.04.1_amd64.deb
# And pick put the parts relevant to the import/export directory from the post-install script
root@dbserver:/tmp/mysql-server-5.5_control# grep mysql_filesdir DEBIAN/postinst
if [ ! -d "$mysql_filesdir" -a ! -L "$mysql_filesdir" ]; then mkdir "$mysql_filesdir"; fi
chown -R mysql:mysql $mysql_filesdir
chmod 700 $mysql_filesdir
This code ensures a directory like this exist:
drwx------ 2 mysql mysql 4096 nov.  23 10:09 /var/lib/mysql-files/
Since my customer wants to use SELECT .. INTO OUTFILE to export data it is not surprising that they also want to access the produced files with another user than mysql. (And as a sysadmin I strongly prefer them to not use “my” system users, but rather “their” application users. And since they are smart they want me to prefer them to not use “my” system users.) Basically the customer wants their developers to ‘just be able to access the files’, and I want to facilitate that in a simple, effective and non-convoluted way.
Non-convoluted is important since we work in teams in Redpill Linpro Managed Services. The easier it is for my co-workers to see how something works on a server, the better services we can provide. So we prefer to avoid implementing things in complicated ways with possibly suprising behaviour.

Chmod as a solution

For our customer and us the preferred solution is to have the directory accesible by their application user. An easy way to do that is to add the application user to the mysql group and chmod it so that it is accessible for members of the mysql group.
chmod 0770 /var/lib/mysql-files

Chmod reverted

But as you probably noticed above - the postinst script will revert this. After every run of the postinst script - thus efter every time the mysql-server-5.5 package is upgraded - the directory will revert to being only accessible to the mysql user.

Sudo as a non-solution

Giving the customer sudo access to copy the files acting as the mysql user is easy, but it forces the developers to remember that they can’t just copy the files as they are used to, but have to use sudo. It also opens us up to security problems with globbing arguments in our sudo rules, unless we force them to decide on filenames in advance. A helper script to which we provide sudo access would remove the security concerns, but still leaves us with a cumbersome way of getting where we want. And for all we know maybe they don’t really want to copy the file, but to do something to it? The developers just want to access their exported data, and since the developers are our friends I want to help them do just what they want - without hassle.

Different ways to get our will

We just need to make sure that it is we, not the postinst script, that get our will when it comes to these permissions. Preferably in an elegant way.
Luckily there are lots of ways to solve this on a Unix-based system. Some ways not as good as others. Some not so elegant.

An ensured failure

The most straighforward solution is to remember to run the chmod command every time the package is upgraded. That is of course not going to happen, it will be forgotten. And with all these computers around there isn’t really any reason why we should handle this manually.

Be a puppet master

At Redpill Linpro Managed Services we use Puppet for configuration management, so we could let Puppet handle this. Most likely noone will try to “SELECT .. INTO OUTFILE” in the time between the upgrade and the next Puppet run, right? Well, probably not, but I bet that the one time when it is important to get access those data really quickly will be between a package upgrade and a Puppet run.
Cron is is the daemon who execute scheduled commands on your Linux system.
So we can just make it right all the time (well, once a minute that is) with the help of cron
File: /etc/cron.d/fix_permissions_mysql-files

# Ensure correct permissions for /var/lib/mysql-files directory
* * * * * root /bin/chmod 0770 /var/lib/mysql-files
This of course is overkill, in the sense that a package upgrade will set the wrong permissions a handful times a year and this cron job will try to fix it a little over half a million times a year. The strain this puts on the server should be negliable, but the strain such a solution puts on the mental health and well being of many sysadmins probably isn’t. And that is not really elegant, is it?
rm  /etc/cron.d/fix_permissions_mysql-files

Immediate (i)notification and reaction

A more elegant, and thus attractive, solution would be if we could trigger a chmod when there is a change to the directory, not when there is a change to what time it is. And the Linux kernel provides us with a way to do just that through the inotify subsystem. Inotify is a facility that makes it possible to do things when something changes in a filesystem. Luckily, since throwing together small C programs using kernel system calls isn’t modus operandi for most of us, there are user space tools that makes it possible to utilise the inotify functionality quite easily in shell scripts (and if your weapon of choice is C, Haskell, ocaml, python, ruby or basically anything else, most likely there is a library to help you getting inotified).
One of the tools available is incron:
[..] the “inotify cron” system. It consist of a daemon and a table manipulator. You can use it a similar way as the regular cron. The difference is that the inotify cron handles filesystem events rather than time periods
So this gives ut a method to revert the postinst script’s revert of our chmod immediately. This script will do the actual chmod (and leave a trace of this in the syslog)
File: /usr/local/bin/


logger -t incron -p daemon.notice "Chmodding /var/lib/mysql-files back to 0770"
/bin/chmod 0770 /var/lib/mysql-files
And this is the configuration for incron
File: /etc/incron.d/RL_fix_permissions_mysql-files

# Ensure correct permissions for /var/lib/mysql-files directory
# chmod if metadata changes, but disable while doing this, so we don't loop.

/var/lib/mysql-files IN_ATTRIB,IN_NO_LOOP /usr/local/bin/
The IN_NO_LOOP ensures that our chmodding of the file doesn’t trigger a new inotification and thus a new chmod and a new inotification and a new chmod and ….
But every time a file inside the directory is changed we also change the metadata for the directory, and then we trigger a chmod. A completely unecessary chmod. And that is not really elegant, is it?
rm /usr/local/bin/
rm /etc/incron.d/RL_fix_permissions_mysql-files

React instead of random act

It would be better if we could react only when our chmod is reverted by the postinst script. It would be neat if we could hook into the package handling. And we can!
We can configure apt to trigger a script after running dpkg:
File: /etc/apt/apt.conf.d/98RL_fix_permissions_mysql-files

# Ensure correct permissions after dpkg invocations
DPkg::Post-Invoke { "/bin/chmod -v 0770 /var/lib/mysql-files"; };
Now every time dpkg runs we will make sure the permissions are as we want them. Let me install an old favourite to demonstrate
root@host:~# aptitude install cowsay
The following NEW packages will be installed:
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Setting up cowsay (3.03+dfsg1-6) ...
mode of ‘/var/lib/mysql-files’ changed from 0700 (rwx------) to 0770 (rwxrwx---)
You noticed that last line, right?
“But this too is a sub-optimal solution, isn’t it?”, you’re probably thinking, and you are absolutely right. It will run chmod every time dpkg has been invoked from apt, which is quite often. And if we for some reason or not download the package and install it directly with dpkg our chmod will not be ran. The latter could be handled if we configured this with a ‘‘post-invoke’’ hook in dpkg.conf instead of treating it as apt configuration, but we would still only solve this for ourselves, and only on the systems where we add this configuration. And that is not really elegant, is it?
It is tempting to just
rm /etc/apt/apt.conf.d/98RL_fix_permissions_mysql-files
but I’m not doing that until we have a better solution in place.

Isn’t this really a bug?

The elegant and community oriented solution is to remove the cause for this behaviour, and make sure we don’t need to go in and chmod the directory ever again.
Rembemer, the problem comes from this code in the postinst script:
if [ ! -d "$mysql_filesdir" -a ! -L "$mysql_filesdir" ]; then mkdir "$mysql_filesdir"; fi
chown -R mysql:mysql $mysql_filesdir
chmod 700 $mysql_filesdir
and could be resolved if we move the chmod (and that chown that could lead to the same kind of problems) inside the if statement:
if [ ! -d "$mysql_filesdir" -a ! -L "$mysql_filesdir" ]; then
mkdir "$mysql_filesdir"
chown -R mysql:mysql $mysql_filesdir
chmod 700 $mysql_filesdir
To that end I have proposed this as a bug to the Ubuntu package maintainers, and hopefully I can soon
rm /etc/apt/apt.conf.d/98RL_fix_permissions_mysql-files
You can track the progress of the bug and my patch at Ubuntu Launchpad

Closing remarks

I hope you have enjoyed this little walk through my sometime episode. If you learned something new that’s great. All of the approaches above can prove useful for solving problems completely different from this one. You mighe have suggestions to other ways of solving this, and that’s great - one of my favourite things with Unix-based systems is that there are so many ways you can solve a problem, and thus so many new things you still can learn even after having been in the game for a couple of decades.
From all of us (including my old friend) to all of you:
< A very merry Christmas! >
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||

by Sigurd Urdahl ( at Tue 23 Jan 2018, 09:55

22 January 2018

Redpill Linpro Techblog

Mulesoft Runtime on Raspberry with Kubernetes

The Raspberry Pi 3 is the third generation Raspberry Pi, on this I will be installing Mulesoft enterprise runtime with latest Java 8 running inside Kubernetes. The pods will register themselves with Anypoint platform runtime manager. This ...

Mon 22 Jan 2018, 23:00

21 January 2018

Redpill Linpro Techblog

yum shell - bat out of dependency hell

There’s evil in the air and there’s thunder in sky – Meatloaf “Bat out of hell”

yum install foo [..] Error: foo conflicts with bar 

Again I have had the dubious pleasure of having dependencies between RPM-packages ending my attempt to install a single package because of a deep-rooted fear of removing core packages from production ...

Sun 21 Jan 2018, 23:00

17 January 2018

Sigurd Urdahl

yum shell - bat out of dependency hell

There's evil in the air and there's thunder in sky
(Meatloaf "Bat out of hell")

# yum install foo
Error: foo conflicts with bar

Again I have had the pleasure of having dependencies between RPM-packages ending my attempt to install a single package with a suggestion of removing core packages. I think this most often happen with Mysql or Percona packages, but I am sure MariaDB will be able to give you the same situation too. It's not the first time I have been here..

[root@ftp01-prod ~]# yum install Percona-Server-client-57
Loaded plugins: fastestmirror, priorities
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package Percona-Server-client-57.x86_64 0:5.7.20-19.1.el7 will be installed
--> Processing Dependency: Percona-Server-shared-57 for package: Percona-Server-client-57-5.7.20-19.1.el7.x86_64
--> Running transaction check
---> Package Percona-Server-shared-57.x86_64 0:5.7.20-19.1.el7 will be installed
--> Processing Dependency: Percona-Server-shared-compat-57 for package: Percona-Server-shared-57-5.7.20-19.1.el7.x86_64
--> Running transaction check
---> Package Percona-Server-shared-compat-57.x86_64 0:5.7.20-19.1.el7 will be installed
--> Processing Conflict: Percona-Server-shared-compat-57-5.7.20-19.1.el7.x86_64 conflicts Percona-Server-shared-56
--> Finished Dependency Resolution
Error: Percona-Server-shared-compat-57 conflicts with Percona-Server-shared-56-5.6.38-rel83.0.el7.x86_64
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
[root@ftp01-prod ~]#

So if I want to install Percona-Server-client-57 I have to install Percona-Server-shared-compat-57 too, and that I can't because of the already installed Percona-Server-shared-56. OK, so I will just remove Percona-Server-shared-56 and then install Percona-Server-shared-compat-57 before doing the install I first tried to do:

[root@ftp01-prod ~]# yum remove Percona-Server-shared-56
Dependencies Resolved

 Package                           Arch            Version                      Repository                 Size
 Percona-Server-shared-56          x86_64          5.6.38-rel83.0.el7           @percona-release          3.4 M
Removing for dependencies:
 MySQL-python                      x86_64          1.2.5-1.el7                  @centos_os                284 k
 fail2ban                          noarch          0.9.7-1.el7                  @epel                     0.0
 fail2ban-sendmail                 noarch          0.9.7-1.el7                  @epel                      11 k
 perl-DBD-MySQL                    x86_64          4.023-5.el7                  @centos_os                323 k
 postfix                           x86_64          2:2.10.1-6.el7               @centos_os                 12 M
 redhat-lsb-core                   x86_64          4.1-27.el7.centos.1          @anaconda                  45 k

Transaction Summary
Remove  1 Package (+6 Dependent packages)

Installed size: 16 M
Is this ok [y/N]:

Very much not OK. I'd like to at least keep things with descriptions like this

Description : The Linux Standard Base (LSB) Core module support
: provides the fundamental system interfaces, libraries,
: and runtime environment upon which all conforming
: applications and libraries depend.

The problem seems to be that Postfix needs which is provided by both Percona-Server-shared-56 and Percona-Server-shared-compat-57. So I just need to swap the former for the latter, and then I can run my original install.

OK, so I both want to remove a package and install a package. And I want to do it at the same time, so that I don't have to remove things like redhat-lsb-core. Did you notice the use of the word transaction in "Transaction Summary" from yum? A transaction is actually what I want. Luckily yum provides a way of doing this, and have probably done since forever, but I didn't learn about it till today. And as so many times before, it is a shell that solves our problems:

[root@ftp01-prod ~]# yum shell
Loaded plugins: fastestmirror, priorities
> remove Percona-Server-shared-56
> install Percona-Server-shared-compat-57
Loading mirror speeds from cached hostfile
> run
--> Running transaction check
---> Package Percona-Server-shared-56.x86_64 0:5.6.38-rel83.0.el7 will be erased
---> Package Percona-Server-shared-compat-57.x86_64 0:5.7.20-19.1.el7 will be installed
--> Finished Dependency Resolution

 Package                             Arch       Version                Repository            Size
 Percona-Server-shared-compat-57     x86_64     5.7.20-19.1.el7        percona-release      1.2 M
 Percona-Server-shared-56            x86_64     5.6.38-rel83.0.el7     @percona-release     3.4 M

Transaction Summary
Install  1 Package
Remove   1 Package

Total download size: 1.2 M
Is this ok [y/d/N]:

Yes, very much thank you! And then finally:

[root@ftp01-prod ~]# yum install Percona-Server-client-57
Dependencies Resolved

 Package                        Arch         Version                  Repository             Size
 Percona-Server-client-57       x86_64       5.7.20-19.1.el7          percona-release       7.2 M
Installing for dependencies:
 Percona-Server-shared-57       x86_64       5.7.20-19.1.el7          percona-release       747 k

Transaction Summary
Install  1 Package (+1 Dependent package)

Total download size: 7.9 M
Installed size: 41 M
Is this ok [y/d/N]:y

Done and done :-)

by Sigurd Urdahl ( at Wed 17 Jan 2018, 23:00

29 December 2017

Ingvar Hagelund

J.R.R. Tolkien: The Silmarillion

I read Tolkien’s “canon”, that is, The Hobbit, The Lord of the Rings, and The Silmarillion, around Christmas every year. So also this year.

One of the most fascinating stories in The Silmarillion is of course the story of Túrin Turambar. He is regarded as one of the major heroes of his age. At the Council of Elrond, Elrond himself lists the great men and elf-friends of old, Hador and Húrin and Túrin and Beren. But while reading through the Silmarillion, there are few among mortal men that have also added so much pain and disaster to the elves. While a great war hero, Húrin was also responsible for the slaying of the greatest hunter of the elves, Beleg Cúthalion, the strong bow. Being the war hero, he turned the people of Nargothrond away from the wisdom of their history, and even their king, and made the hidden kingdom available for the enemy. How many elves were cruelly slain or taken to captivity in Angband because of Turin’s pride? Thousands! Perhaps even tens of thousands? So how come the elves, ages later, still reckoned Túrin son of Húrin as one of the great elf-friends?

In a Nordic saga style stunt, Túrin finally slew his greatest enemy, Glaurung the great fire-breathing dragon. Glaurung had been a continous danger to all peoples of Middle-Earth, and the end of that worm was of course a great relief to all the elves, even Elrond’s ancestors, the kings of Doriath and Gondolin. Also, we must remember that the lives of the elves are different from that of men. When the elves’ bodies die, their spirits go to Mandos, where they sit in the shadow of their thought, and from where they may even return, like Glorfindel of both Gondolin and Rivendell. But when men die, they go to somewhere else, and are not bound to the world. It seems that elves are more willing to forgive and let grief rest for wisdom over time, than are men’s wont. Even the Noldor who survived the passing of the Helcaraxë forgave and united the Noldor of Fëanor’s people that left them at the burning of the ships at Losgar.

Perhaps that is one of the lessons learned from the tragic story of Túrin. From all his unhappy life, good things happened, and afterwards, the elves forgave and even mourned him and his family.

by ingvar at Fri 29 Dec 2017, 16:27

25 December 2017

Ingvar Hagelund

J.R.R. Tolkien: The Lord of the Rings

I read Tolkien’s “canon”, that is, The Hobbit, The Lord of the Rings, and The Silmarillion, around Christmas every year. So also this year.

2017 was a great year for Tolkien fans. It was the 125th anniversary of the Professor’s birth, and the 80th anniversary for the Hobbit. We also got the magnificent news that Amazon will produce a TV series based on “previously unexplored stories based on J.R.R. Tolkien’s original writings“. So what storylines would that be? A reboot of the 2001-03 trilogy is out of the question, as Peter Jackson explored and extended more than enough already. So, what do we have left? A lot! Let’s have a look.

The Lord of the Rings and its appendices tells stories in several different timelines. Long before (as in hundreds, and even thousands of years) before the main story, just before the main story (like a few decennials), parallel to the main story, and after.

One storyline could follow the ancient history of Gondor and Arnor. There are lots and lots of substories there. If I should pick one I would like to see, it would be the stories of the kings Arvedui of Arnor and  Eärnil II of Gondor, perhaps started with the Firiel incident. There are lots of exciting points to pick up there. Gondor throne heiritage politics, the war against, and the prediction of the downfall of the Witch King, the flight to Forochel, with the disastrous ship’s wreck in the ice, and the loss of the palantiri.

For the “near history” before The War of the Ring, the obvious choice would be a “The young Aragorn” series, where we could follow Aragorn in his many guises, riding with the Rohirrim, going on raids with Gondor against Harad, in and in constant conflict with Denethor. And his love life, of course, with his meeting and very long-term relationship with Arwen. And speaking of Arwen, her family story is a good storyline, with the love of Celebrían and Elrond, travelling from Lorien to Rivendell, and her abduction, and Elladan and Elrohir’s rescue of her from the orcs. Parallel to that, the story I would most love to see, would be, the story of Denethor. His tragic life is worth a season alone. Another storyline from the years just before The War of the Ring, could be Balin’s attempt to retake Moria, and build  a colony of dwarves. Lots of gore and killing of goblins to depict!

Parallel to the War of the Ring, there are a lot of things going on, that are merely mentioned in the book, and completely forgotten in the movies. The fight in Dale. The Ents’ war against the orcs after the capture of Isengard, the loss of Osgiliath and Cair Andros, to name just a few.

And of course, even after the the War of the Ring, and the Return of the King, there are stories to follow up. Aragorn’s “negotiations” for peace with his neighbouring peoples, with armed battle as alternative, supported by Eomer of Rohan. The sweet but bitter death of Aragorn and Arwen. The reign of King Eldarion.

I’m optimistic! This is going to be great!

by ingvar at Mon 25 Dec 2017, 07:00

24 December 2017

Redpill Linpro Sysadvent

Thank you for visiting our SysAdvent Blog!

We hope you have enjoyed the articles in our third SysAdvent season!

This is the last post in this years sysadvent. If you want to read more, we have other blog entries at our main site, our techblog, our employees have personal blogs that are aggregated at

Sun 24 Dec 2017, 23:00

Ingvar Hagelund

J.R.R. Tolkien: The Hobbit

I read Tolkien’s “Canon”, that is, The Hobbit, The Lord of the Rings, and The Silmarillion, every year about Christmas. These year, it’s even The Hobbit’s 80th Anniversary, and to celebrate, I have of course read through The Hobbit again.

So many have said so much about this book, so I’d rather show off my newest addition to my Tolkien bookshelf. This is the Swedish 1962 edition of The Hobbit, Bilbo, En Hobbits Äventyr (Bilbo, A Hobbit’s Adventure), and it has quite an interesting history.

In the 50s and 60s, Astrid Lindgren, maybe most famous for her children’s books about Pippi Longstocking, worked as an editor at the department for Children’s literature at Rabén & Sjögren, who published Tolkien’s works in Sweden. Lindgren was very interested in Tolkien’s work, and while she later denied Tolkien as an inspiration for it, she published the quite Lord of the Rings reminiscing Mio my Son in 1954, and later the world beloved classic children’s fantasy novels The Brothers Lionheart and Ronia, the Robber’s daughter.

In the early 60s Lindgren was not content* with the current Swedish translation of The Hobbit, Hompen (translation by Tore Zetterholm, 1947), and wanted to better it. So she opted for a new translation and got hold of Britt G. Hallqvist for the job. For illustrations, she contacted her friend Tove Jansson, now World famous for her Moomin Valley universe. Jansson had already had success with her Moomintrolls, and had previously made illustrations for a Swedish edition of Lewis Carrol’s classic poem Snarkjakten (The Hunting of the Snark, 1959), so a successful publication seemed likely.

Hallqvist translated, Jansson drew, Lindgren published it, and it flopped! Tolkien fans didn’t enjoy Jansson’s drawings much, and the illustrations were not used** again before 1994. By then, the 1962 version was cherished by Tove Jansson fans and Tolkien collectors over the World, and it had become quite hard to find. The 1994 edition was sold out in a jiffy. The illustrations were finally “blessed” by the Tolkien Estate, when they were used for the 2016 Tolkien Calendar.

Jansson’s illustrations were also used in the 2016 Tolkien calendar, which I’m, afraid to say, have not acquired (yet).

I was lucky and found a decent copy of the 1962 edition in a Japanese(!) bookstore on the Net. Now I LOVE this book. Its illustrations are absolutely gorgeous.




The destruction of Lake Town and the death of Smaug are my personal favourites

The destruction of Lake Town and the death of Smaug is my personal favourite

It makes a great additon to my ever growing list of Hobbits.

This book makes a great additon to my ever growing list of Hobbits.

It would be a pity to let this book stay alone without decent Janssonic company, so I searched a few weeks, was lucky again and found a nice copy of the mentioned Snarkjakten by Lewis Carrol, and an almost mint copy of the absolutely fantastic (in all meanings of that word) Swedish 1966 edition of Alice i underlandet (Alice in Wonderland). If you enjoy Alice, you will love Janssons’ illustrations, even outshining her work on The Hobbit.

For an intensely interesting read about Jansson’s artistic work on these classics: Read Olga Holownia’s essay at

That’s it. Merry Christmas and happy Youletide everybody!

*) Neither was Tolkien himself. He specially disliked the translation of Elvish names into Swedish, like Esgaroth -> Snigelby (ie. Snail Town!!!). Also interesting: Svensson, Louise, Lost in Translation? – A Comparative Study of Three Swedish Translations of J.R.R. Tolkien’s ‘The Hobbit’, Lund University 2016

**) Actually, there were other versions with Jansson’s illustrations; the Finnish Hobbit Lohikäärme-vouri (The Dragon mountain) from 1973, and the updated Finnish translation in 2003. The illustrations were also used in this year’s Finnish 80th Anniversary edition of The Hobbit.

by ingvar at Sun 24 Dec 2017, 07:00

23 December 2017

Redpill Linpro Sysadvent

Using Ansible for system updates

As mentioned in the previous ansible post, we use ansible quite a lot for day to day operations. While we prefer Puppet for configuration management, ansible is excellent for automation of maintenance procedures.

One such procedure is gracefully applying package upgrades, including any required reboot, of application servers. In ...

Sat 23 Dec 2017, 23:00

22 December 2017

Redpill Linpro Sysadvent

Vagrant for the lazies

Personally, I consider the script parameter in a Vagrantfile to be a feature that is not abused enough. It’s got a lot of potential - every script can have a parameter (or several). Modifying your Vagrant use to include this gives you a more flexible and reliable ...

Fri 22 Dec 2017, 23:00

21 December 2017

Redpill Linpro Sysadvent

Tiny virtual firewalls with IncludeOS

Sometimes you need just a little something on your network to do a simple task. Perhaps you need a small router, a firewall or load balancer. Currently, the most popular option is to deploy a little Linux server. There is a downside though, a Linux is rather heavy. It requires ...

Thu 21 Dec 2017, 23:00

Bjørn Ruberg

Control code usernames in telnet honeypot

By running a Cowrie honeypot, I’m gathering interesting information about various kinds of exploits, vulnerabilities, and botnets. Upon a discovery of a new Linux-based vulnerability – often targeting network routers, IoT devices, and lately many IP camera products – the botnets will usually come in waves, testing the new exploits. The honeypot logs everything the […]

by bjorn at Thu 21 Dec 2017, 08:19

20 December 2017

Redpill Linpro Sysadvent

Making it right all the time - or that time I disagreed with a distro package about file permissions

Distro packages are a blessing that most of us take for granted (thank you and sorry package maintainers everywhere!). They make installation, maintenance and even removal of both simple and complex software a breeze.

Sometimes you disagree

But sometimes you disagree with a decision made in the distro package. ...

Wed 20 Dec 2017, 23:00

19 December 2017

Redpill Linpro Sysadvent

S2I hooks

S2I, Source-To-Image, is a toolkit for building Docker images with minimum effort. The S2I project description describes itself like this:

Source-to-Image (S2I) is a toolkit and workflow for building reproducible Docker images from source code. S2I produces ready-to-run images by injecting source code into a Docker container and letting ...

Tue 19 Dec 2017, 23:00

Ingvar Hagelund

12 days of Varnish

While Varnish is most famous for its speedy caching capabilities, it is also a general swiss army knife of web serving. In the spirit of Christmas, here’s Twelve Days of Varnish Cache, or at least, twelve Varnish use cases. Read the rest of this post on Redpill Linpro’s Sysadvent calendar.

by ingvar at Tue 19 Dec 2017, 22:24

18 December 2017

Redpill Linpro Sysadvent

12 days of Varnish

While Varnish is most famous for its speedy caching capabilities, it is also a general swiss army knife of web serving. In the spirit of Christmas, here’s Twelve Days of Varnish Cache, or at least, twelve use cases.


Mon 18 Dec 2017, 23:00

17 December 2017

Redpill Linpro Sysadvent

Investigating performance problems through snapshot logging

So, the database is slow - why?

There can be several reasons for this. Perhaps a few very heavy queries are bogging down the database. In this case, you’d typically set up slow query logging and find them in the slow.log. However, sometimes the reason is simply lots of ...

Sun 17 Dec 2017, 23:00

16 December 2017

Redpill Linpro Sysadvent

Threat intelligence: Sharing is caring

In today’s threat landscape, with botnets, ransomware, and unpatched and unprotected IoT toasters and garbage bins in every home, the ability to quickly and easily identify suspicious activities and artifacts is probably more important than ever before. If your company’s web server is being probed from a particularly malicious IP ...

Sat 16 Dec 2017, 23:00

15 December 2017

Redpill Linpro Sysadvent

A different approach to log rotation

Logrotation is a key for running a stable server, but removing log files is often an anathema to security, traceability, and server history. In reality, you want a perfect rotation setup in order to maximise the retention of logs.

Instead of trying to continuously trying to balance the number of ...

Fri 15 Dec 2017, 23:00

14 December 2017

Redpill Linpro Sysadvent

Using Let's Encrypt with OpenShift

When installing OpenShift, the default certificates that are being installed are self-certified. Although this gives you functional encryption, this is in no way best practice and is especially annoying for the route being exposed for the Hawkular metrics, which is integrated within the Web console.

Luckily there is a ...

Thu 14 Dec 2017, 23:00

13 December 2017

Redpill Linpro Sysadvent

Using ssh_config(5) and FoxyProxy for fun and profit

The other day, as I just had updated my workstation to Fedora 27, I realized maybe the Include statement in ssh_config(5) had been implemented. And indeed it had.

So it’s time to reorganize my ssh-config-generate script, FoxyProxy browser plugin for tunneling web traffic through ssh, and maybe even setting up ...

Wed 13 Dec 2017, 23:00

12 December 2017

Redpill Linpro Sysadvent

iPXE and automated provisioning

Provisioning of new servers can be a daunting experience. Back in days it meant booting the machine with a CD or a DVD and doing manual choices. Automation of the installation process makes the process faster and less prone to human errors.

Network installation helps the process, but you still ...

Tue 12 Dec 2017, 23:00

11 December 2017

Redpill Linpro Sysadvent

Care and feeding of SMTP honeypots

In parallel with an SSH/telnet honeypot, I’m also running an SMTP honeypot using INetSim. The SMTP honeypot is only one of many functions of INetSim; this article will cover the SMTP component only.

The SMTP part of INetSim has been configured with the following settings in inetsim.conf:

Mon 11 Dec 2017, 23:00

10 December 2017

Redpill Linpro Sysadvent

Allow backup sysadmins to gain access through a "Break the Glass"-solution

I want backup sysadmins to have login access to some systems, with said access rarely (if ever) used. To prevent abuse I’d like strong audit logging, logging that stands out from the rest of all the logging, logging that cannot be tampered with, and that can easily be followed up ...

Sun 10 Dec 2017, 23:00