Planet Redpill Linpro

25 December 2021

Ingvar Hagelund

Sam the Spy (J.R.R. Tolkien: The Lord of the Rings)

I read Tolkien’s “Canon”, that is, The Hobbit, The Lord of the Rings, and The Silmarillion, every year about Christmas. So also this year.

If you are among the lucky readers that get to immerse yourself in The Lord of the Rings regularly, you may have wondered about Sam’s thoughts and reactions in The Shadow of the Past. After Sam’s exchange with Ted Sandyman at The Green Dragon inn at Bywater, we learn that Sam had a good deal to think about (…) He would have a busy day tomorrow (…) But Sam had more on his mind than gardening. After a while he sighed, and got up and went out. When I read this, I used to pause and consider what Sam was thinking about. For some years I thought it was Rose Cotton that was on his mind. But she does not enter the story until the very end. So what is it that bothers Sam so much?

We learn that this is the same time as Gandalf is visiting Frodo. And their exchange about The Ring must be the next morning. When Sam is discovered by Gandalf, eavesdropping outside Frodo’s windows in Bag End, Sam first try to bluff Gandalf, producing his garden shears. Then he quakes and begs mercy and talks like a waterfall. Finally, he shouts of joy, before bursting into tears. Anyone may feel a bit intimidated under Gandalf’s bristling beard and brow, but isn’t this reaction a bit much? Sam is a bit of an emotional type, but shouting of joy, and then crying his eyes out?

We know from A Conspiracy Unmasked that Sam, Merry and Pippin are conspiring against Frodo leaving The Shire alone, and have been for years. Sam is presented as the chief investigator of the group. Here it all comes together. Merry and Pippin has talked Sam into spying on Frodo and Gandalf. It is not strange that he is thinking a lot and planning how to get through with this, even cooking up an alibi of mowing the lawn, and trimming the grass outside exactly the window where Frodo and Gandalf are discussing The Ring. He is even almost caught at one point, where he appears to coincidentally pass along the garden path whistling. Let us repeat that: He actually passes by, whistling innocently. When I read this again, I almost can’t believe Gandalf not seeing through this! When Sam finally is discovered, he actually tricks Frodo and Gandalf into believing that he only coincidentally heard what they were talking about. It is not strange that he first babbles and begs before finally shouts of joy and bursts into tears. He cries in relief of not disclosing the conspiracy. He is not revealed as a spy yet – and luckily, not by Gandalf, or he might actually been turned into a spotted toad.

In Crickhollow, after the conspiracy is finally unmasked, Sam says that Frodo ought to take the Elves advice. Gildor said you should take them as was willing, and you can’t deny it. Frodo’s answer is a bit remarkable unless you have figured out the connection: I’ll never believe you are sleeping again. Here, Frodo is of course pointing to the fact that in Three is a company, while Gildor has a conversation with Frodo, and that while these words fall, Sam sat curled up at Frodo’s feet, where at least he nodded and closed his eyes. But Sam is here still the spy in the group. He only pretends to sleep, and is actually eavesdropping as hard as he can all the time. This is taken up again by Merry in The Palantir: Now Pippin my lad, don’t forget Gildor’s saying – the one Sam used to quote: “Do not meddle in the affairs of Wizards, for they are subtle and quick to anger.” Gildor said this to Frodo while Sam was apparently sleeping.

At the end of The Council of Elrond we hear that Sam again is spying and eavesdropping. He suddenly jumps up from the corner where he had been quietly sitting on the floor, and Elrond remarks that the council was secret, and that Sam was not invited.

In Flight to the Ford, Frodo says about Sam that First he was a conspirator, now he’s a jester. He’ll end up by becoming a wizard – or a warrior! And Sam answers: I hope not (…) I don’t want to be neither!. But at least his career as a conspiring spy was rather successful.

Merry Christmas, and a happy new year!

 

 

With great thanks to The Tolkien Professor and his Exploring the Lord of the Rings project, where the role of Sam has been more than thoroughly discussed

by ingvar at Sat 25 Dec 2021, 22:19

11 December 2021

Redpill Linpro Techblog

Log4Shell and the importance of an outbound firewall

The most common firewall setups reject inbound traffic initiated from the internet as such, but let all traffic pass through as long as the connection was initiated from the intranet. I strongly believe that such firewalls are overrated and that it makes as much sense (maybe even more sense) to filter outbound traffic.

The Log4Shell case

Yesterday some of us got quite busy with patching things and searching for vulnerable software due to the Log4Shell 0-day exploit. This ...

Sat 11 Dec 2021, 00:00

26 October 2021

Redpill Linpro Techblog

Zimbra and Outlook

TLDR: Jump to the end for the quick explanations of how to do this.

At Redpill Linpro we use Zimbra for our email hosting.

It has the same email and calendar functionality as Microsoft Exchange, so it’s a great alternative to Office 365. Especially if you don’t want to have your data stored in Microsoft’s datacenters.

But if you like to use Microsoft Office, it’s not directly intuitive to get Outlook fully integrated with Zimbra. If you try to ...

Tue 26 Oct 2021, 00:00

31 May 2021

Redpill Linpro Techblog

Better Bluetooth headset audio quality with mSBC

Pipewire, the new default audio subsystem in Fedora 34, introduces support for the mSBC audio codec for the Bluetooth Headset Profile (HSP), which is used for transmitting bi-directional audio to/from Bluetooth headsets. The mSBC codec provides greatly improved audio quality over the CVSD, the default codec used with HSP. If you are using a Bluetooth headset to participate in teleconferences, you will most definitively want to use mSBC if your headset supports it. Your colleagues will sound better ...

Mon 31 May 2021, 00:00

17 May 2021

Redpill Linpro Techblog

Introduction to GraphQL with Vert.X

GraphQL is a modern approach to APIs that simplifies integrations. This is an introduction to what GraphQL is, and we build a simple GraphQL service with the help of Vert.X.

What you’ll need to follow along

This introduction will barely scratch the surface of the potential and power that GraphQL together with Vert.X offers. Understanding of programming and the REST protocol is assumed.

Make sure to have the following installed:

  • JDK 8+
  • Maven
  • IDE

Source code for ...

Mon 17 May 2021, 00:00

07 May 2021

Magnus Hagander

Getting random rows faster. Very much faster.

Getting a single random row, or a few rows, from a table in order to get representative data for example is a frequent need. The most common way to do this in PostgreSQL is using ORDER BY random() like:

SELECT id FROM data ORDER BY random() LIMIT 1

But when run on a large table this can be very slow because it will have to scan the entire table to find the rows. Jonathan Katz mentioned a different way to do it on Twitter, which reminded me that people keep coming up with different (and sometimes very complicated) ways of trying to solve this problem.

And while Jonathan's method (he has the super simple sample code and results up on a gist) is still about twice as fast as ORDER BY random() on my test (with his data), it comes with some problems. For example, it requires a contiguous set of id values, that have to be integers. And it still takes about a second to run on my machine with his sample of 5 million rows -- and will keep getting slower as the table grows.

And it turns out, if you don't need your row to be perfectly random, just mostly random, and can deal with some caveats, PostgreSQL has built-in functionality that does the job about 20,000 times faster than Jonathan's version and 40,000 times faster than ORDER BY random(). Enter TABLESAMPLE.

by nospam@hagander.net (Magnus Hagander) at Fri 07 May 2021, 18:30

Redpill Linpro Techblog

Getting random rows faster. Very much faster.

Getting a single random row, or a few rows, from a table in order to get representative data for example is a frequent need. The most common way to do this in PostgreSQL is using ORDER BY random() like:

SELECT id FROM data ORDER BY random() LIMIT 1 

But when run on a large table this can be very slow. Jonathan Katz mentioned a ...

Fri 07 May 2021, 00:00

28 April 2021

Redpill Linpro Techblog

Backup verification for Veeam Agent for Windows

As mentioned in my previous post, we have some interesting projects involving Hyper-V. One of them being a way to automatically verify that our Windows server backups are functional.

This post is about how we’ve set that up.

TLDR: check the script on github

Windows backup on OpenStack

Our main virtualization platform is OpenStack, which is awesome. And while we try to mostly utilize open-source solutions, there are times when that’s just not the best way to ...

Wed 28 Apr 2021, 00:00

07 April 2021

Redpill Linpro Techblog

Nested Virtualization - Hyper-V 2019 in qemu-kvm

TLDR: Jump to the end for the XML tags we found necessary to get this working.

We have a few projects coming up where we for different and exciting reasons need to use Hyper-V. But running Hyper-V on bare-metal would in this case take up more resources than reasonable for us, so we’re nesting it inside one of our kvm clusters.

But getting Hyper-V 2019 to actually run steadily using nested virtualization inside of qemu-kvm provided some challenges that turned ...

Wed 07 Apr 2021, 00:00

31 March 2021

Redpill Linpro Techblog

Finalizing the VPC template

I want to wrap up the VPC template from the previous blog entry “Moving forward with Cloudformation templates”

What we ended up with there was a VPC with a Private and a Public Subnet in 3 Availability Zones.

Now I want to start to use the Outputs section of the template.

And when that has been introduced, I want to use Nested Stacks

Why Outputs?

When we create a VPC with the template from the last ...

Wed 31 Mar 2021, 00:00

Faster Clojure with GraalVM

Clojure is a really nice, dynamic programming language on the Java virtual machine. It gives you the expressivity of a lisp, full interop with the whole Java/Kotlin/Scala ecosystem of libraries, a battle-hardened VM, and it’s elegant.

The downside is that it runs on the JVM and has to pay a heavy cold start penalty. We can fix that, using GraalVM, and have our cake, eat it, and space-time fold it too.

We will be using a few containers, podman and GraalVM’s native-image ...

Wed 31 Mar 2021, 00:00

19 February 2021

Redpill Linpro Techblog

Free Software and Open Source: Get involved

Contributing to Free Software using Open Source methodics may look like intimidating deep expert work. But it doesn’t have to be that. Most Free Software communities are friendly to newcomers, and welcome all kind of contributions

...

Fri 19 Feb 2021, 00:00

18 February 2021

Ingvar Hagelund

Free Software and Open Source: Get involved

Contributing to Free Software using Open Source methodics may look like intimidating deep expert work. But it doesn’t have to be that. Most Free Software communities are friendly to newcomers, and welcome all kind of contributions.

Reporting bugs

Hitting a bug is an opportunity, not a nasty problem. When you hit a bug, it should be reported, and with a bit of luck, it may even be fixed. Reporting the bug in an open forum also makes other users find the bug, give attention to it, and they may in turn be able to help out working around or fixing it. Reporting bugs is the most basic, but still of the most valuable contributions you may do. Finding bugs are finding real problems. Reporting bugs are helping fixing them, for you, and for other users. You may not complain to your coworker on a bug unless it is reported upstream.

While reporting bugs, remember to collect as much information as possible on the issue, including logs, runtime envionment, hardware, operating system version, etc. While collecting this information, make sure you don’t send any traceable private information that may be used by rouge parties, like ip adresses, hostnames, passwords, customer details, database names, etc.

Bugs in operating system packages

Bugs in components delivered by a Linux distribution (Ubuntu, Debian, Fedora, Red Hat, SuSE, etc), should be reported through their bug reporting interface. Remember to search for the bug before posting yet another duplicate bug. Perhaps a workaround already exists.

So the next time something strange happens to your haproxy, nginx, varnish, or your firefox browser crashes or has unexpected behaviour, collect data from your logs, and open a bug report.

  • Red Hat / EPEL / Fedora users should report bugs through https://bugzilla.redhat.com/
  • Similarly, OpenSuSE users may search for and report bugs at https://bugzilla.opensuse.org
  • Ubuntu users may have luck looking at https://help.ubuntu.com/community/ReportingBugs
  • As Ubuntu’s upstream is Debian, you may search for bugs, fixes and workarounds using their tools at https://www.debian.org/Bugs/Reporting

    These tools have detailed guidelines on the details on how to search, report, and follow up the bugs.

    For an example of an end user bug report with an impressive follow up from a dedicated package maintainer, have a look at https://bugzilla.redhat.com/show_bug.cgi?id=1914917

    Reporting upstream bugs

    Using software directly from the upstream project is growing more usual, specially as container technology has matured, enabling developers to use software components without interfering with the underlying operating system. Reporting and follow up bugs becomes even more important, as such components may not be filtered and quality assured by operating system security teams.

    Find your component’s upstream home page or project development page, usually on Github, Savannah, Gitlab, or similar code repo service. These services have specialised issue trackers made for reporting and following up bugs and other issues. Some projects only has good old mailing lists. They may require you to subscribe to the list before you are allowed to report anything.

    Following up the report, you may be asked for test cases and debugging. You will learn a lot in the process. Do not be shy to ask for help, or admitting that you don’t understand or need guidance. Everybody started somewhere. Even you may learn to use the GNU debugger (gdb) in time.

    Non code commits

    Similarly to reporting bugs, non code commits may be low-hanging fruit to you, but may be crucial to a project’s success. If you can write technical documentation, howtos, or do translations to your native language, such contributions to Free Software are extremely welcome. Even trivial stuff like fixing typos in a translated piece of software should be reported. No fix is too small. I once did a single word commit to GPG: A single word typo fix in their Norwegian translation. Also, write blog posts. Don’t have a blog yet? Get one. Free blog platforms are thirteen to a dozen.

    Use source code tools

    Admit it: You already use git in your day job. Using it for documentation or translation should be trivial. If you have not done so already, learn how to clone a project on github (just google it), grep through the source for what you like to fix or add, make a branch with your contribution, and ask for a pull request (again, just google it). If you changes are not merged at once, be patient, ask for the maintainer’s advice, and listen to their guidelines. Be proud of your contribution, but humble in your request.

    Feature requests

    Usage of a piece of software is not given from the start. Perhaps you have ideas to how a piece of code may be used in some other way, or there is some piece missing that is obvious to you, though not reported in the project’s future roadmap. Don’t be shy to ask. Report a feature request. Usually this is done the same way as reporting a bug. The worst you can get is that they are not interested, or a request for you to produce the missing code. Which you may do.

    Join a project

    If your work require it, and/or your interests and free time to spend allows for it, join a Free Software project.

    Distribution work

    Upstream distributions like Fedora, Debian, and OpenSuse (not to mention Arch and Gentoo) are always looking for volunteers, and have sub projects for packagers, documentation, translation, and even marketing. As long time players in the field, they have great documentation for getting started. Remember to be patient, ask for advice, follow guidelines. Be proud of your contributions, but humble in your requests.

    Upstream projects

    If you want to join a project, show your interest. Join the project’s social and technical forums. Subscribe to their development email lists. Join their IRC channels. Lurk for a while, absorbing the project’s social codes. Some projects are technoraties, and may seem hostile to newbie suggestions without code to back them up. Others are welcoming and supportive. Do some small work showing what you are capable of. Fix things in their wiki documentation. Create pull requests for simple fixes. Join in their discussion. Grow your fame. Stay humble. Listen the long time players.

    Release your own

    Made a cool script at work? A build recipe for some special case? An Ansible playbook automating som often-visited task? A puppet module? Ask your manager for permission to release it as Free Software. Put GPLv3 or some other OSS license on it, and put it on Github. Make a blog post about it. Tell about it in social media. Congratulations, you are now an open source project maintainer. Also, Google will find it, and so will other users.

  • by ingvar at Thu 18 Feb 2021, 13:21

    11 February 2021

    Redpill Linpro Techblog

    Documentation with terraform-docs

    There are many steps to take on the path to automation. Some are easier, some not so much.

    One less popular step is documentation. Yes - the documentation you haven’t written. You know what I mean.

    No matter how well written your code is, there usually is a gap between today’s YOU and the future YOU, who has to work with and pick up what you leave behind.

    This is usually also the reason for writing the documentation last. Better ...

    Thu 11 Feb 2021, 00:00

    30 January 2021

    Redpill Linpro Techblog

    Bonding SR-IOV ports with OpenStack

    All our customers have an online presence. A subset of these have higher demands when it comes to latency and reliability than others. Sometimes this is purely because of high amount of real end-user traffic - and sometimes it’s more malicious; A DDOS-attack.

    In most OpenStack-configurations, you have the concept of «port security». This is a firewall enforced on the network interface of the virtual instance. It is also there to prevent a malicious self-service user from spoofing their IP ...

    Sat 30 Jan 2021, 00:00

    03 January 2021

    Magnus Hagander

    A surprising? lesson in the speed of languages

    Scenario: I had the need for a small tool that would parse logfiles of approximately 6 million lines. To each line apply two regexps to extract a few values, and using two separate dictionaries/hashmaps (choose your poison wrt terminology) calculate how many times each capture group int he regexp shows up.

    All in all, this is not going to be a very performance critical run, as it will only parse about 4 times these 6 million lines per day, so even at worst case we're talking less than 100M regexp matches per day. Piece of cake for any language. And since it's a weekend and I don't have any time when I have to deliver this finished script, I set out to prove to myself that python is going to be fast enough for this.

    by nospam@hagander.net (Magnus Hagander) at Sun 03 Jan 2021, 14:38