New – Amazon Linux WorkSpaces


Over two years ago I explained why I Love my Amazon WorkSpace . Today, with well over three years of experience under my belt, I have no reason to return to a local, non-managed desktop. I never have to worry about losing or breaking my laptop, keeping multiple working environments in sync, or planning for disruptive hardware upgrades.

SysAdmin1138: Systemd dependencies


There is a lot of hate around Systemd in unixy circles. Like, a lot . There are many reasons for this, a short list: For some reason they felt the need to reimplement daemons that have existed for years. And are finding the same kinds of bugs those older daemons found and squashed over a decade ago.

Source: SysAdmin1138: Systemd dependencies

Useful man pages in your browser


A new useful *nix tool popped up in my Twitter timeline a while back.

Lede says Simplified and community-driven man pages and it does what it says on the tin.

If you’re a *nix admin you know the drill of looking up man pages for *nix tools. You’re solving some problem and need to grok an *nix command options and/or refer to a sample of how the tool is used. man {toolname} is the way to do it.

Frequently though the result is usually page and pages of esoteric information about the tool most of which you will never learn and will take you a lot of time to wrap your brain around. Sometimes there will be examples, waaay at the bottom of the page and often those usage examples are pretty light on information.

This is where comes in. Put your *nix tool name into the sample at and it will display useful help. Example:

But it doesn’t end there. The page also has many community contributed clients. Scroll down the page at for the full list.

My favourite use of it is to add the as a search provider in your browser. Chrome in my case. Open your Chrome settings and add a search engine with the config as below.

Screen Shot 2017-12-21 at 11.36.16

Now, in any search bar in Chrome you can type tldr {toolname} eg tldr tar and it will display the results right there for you. A convenient way to get useful information about *nix tools.

JIRA, Confluence and Lets Encrypt


I recently had to move a JIRA and Confluence environment to a new infrastructure stack. During the move, we also changed the TLS Certificates and instead of using one of the paid-for incumbents we decided to give Lets Encrypt a go.

Everything with the migration went smoothly. The first hurdle we hit was when we were checking the Application Integration between the two systems. The integration wasnt functioning and no amount of delete, change, recreate would fix it. The admin pages in JIRA and Confluence were both reporting SSL errors. When I dug into the actual Tomcat logs for each instance, the following errors were appearing:


Caused by: PKIX path building failed:
: unable to find valid certification path to requested target


Caused by: PKIX path building failed: unable to find valid certification
path to requested target

Some quick googlephoo found a few items on the internet about this, not specific to JIRA and Confluence though.


The root cause of the problem is that the JRE thats included with our version JIRA and Confluence is too old and doesn’t include the Lets Encrypt root keychain in its included keystore. The above articles had references and code snippets to help get the Lets Encrypt certificates into the JRE keystore but they were all very ugly.

Its worth mentioning that Oracle JAVA JRE 1.8.0_101 DOES include the Lets Encrypt certificates.

Options at this point were:

  • Find a way to get the required certificates into the JRE keystore (the CLI method to do this is described in the Lets Encrypt community post above).
  • Install a new JRE on the servers and make JIRA and Confluence work with that. Most likely putting us out of support with Atlassian.
  • Find out if current JIRA and Confluence include the required JRE version and then upgrade JIRA and Confluence. This would need another round of testing to properly do the upgrade.

Moving to unsupported configuration was undesirable and I didnt have the time to properly dive into a new round of testing to see if newer JIRA and Confluence had the required JRE version. I did look for some detail on the Atlassian pages to determine the answer to this and wasn’t able to locate anything.

What I did find on an Atlassian page was this article which shows how to use a free JIRA and Confluence plugin to get third party root certificates into the JRE keystore using a simple web page GUI. Some small CLI steps (a cp) are still required after the plug-in does its thing but it does make the fix less likely to fail.


Introduction to strace | The Road to Elysium


HT to @patickkelso for this. Adding to the stash for future reference.

Source: Introduction to strace | The Road to Elysium

Internet Performance Measurement Tools


This post contains a catalogue of useful Internet performance testing tools.

ICSI Netalyzr

sivel/speedtest-cli: Command line interface for testing internet bandwidth using

A nifty tool for the CLI jockeys to test the internet performance of their machine.

speedtest-cli – Command line interface for testing internet bandwidth using

Source: sivel/speedtest-cli: Command line interface for testing internet bandwidth using

Combining PTP with NTP to Get the Best of Both Worlds – Red Hat Enterprise Linux Blog


There are two supported protocols in Red Hat Enterprise Linux for synchronization of computer clocks over a network. The older and more well-known protocol is the Network Time Protocol (NTP). In it…

Source: Combining PTP with NTP to Get the Best of Both Worlds – Red Hat Enterprise Linux Blog


Install vSphere CLI on Ubuntu 14.04 LTS


ARGGGH. What a painful experience this was.

Mainly due to unresolved dependancies in the script.

After a half a day shaving this yak, the method to fix the dependancies is:

sudo apt-get install libxml-libxml-perl libdevel-stacktrace-perl libclass-data-inheritable-perl libconvert-asn1-perl libcrypt-openssl-rsa-perl libcrypt-x509-perl libexception-class-perl libarchive-zip-perl libpath-class-perl libtry-tiny-perl libclass-methodmaker-perl libdata-dump-perl libnet-inet6glue-perl

Lastly, you need to install UUID and UUID::Random from CPAN. There doesnt appear to be suitable packages in 14.04.

  • install cpan (find your own link to an article on doing this on Ubuntu)
  • Then:
perl -MCPAN -e 'install UUID'
perl -MCPAN -e 'install UUID::Random'

You can then run your

Home Internet Monitoring Appliance


My home internet has been pretty flakey of late and I needed a way to monitor the performance of the connection over long periods of time so that I could gather evidence to escalate to my ISP for support and troubleshooting.

Due to the absence of a suitable machine at home to run the tools on I decided to build an Ubuntu VM with smokeping and Cacti. This would allow me to build the environment quickly and then move the VM to a temporary laptop running VMware Workstation at home.

The following steps describe the sequence of events and references I used to complete the work.

Upgrading Request Tracker


I recently upgraded a Request Tracker (RT) environment from version 3.2.2 to version 3.8.7. These are my learnings and thoughts from the experience.

The environment that RT 3.2.2 was running on was an old hand rolled from source deployment on Fedora Core 3. The database was on a separate Debian 4 host running a backported MySQL 4.1.

A new web hosting and database back end was available based on Ubuntu 10.04 LTS and MySQL 5.1. This was the target environment for the upgraded RT.

Based on the available target environment the simplest approach to deliver this upgrade was to:

  • Install and configure RT 3.8.7 from packages on Ubuntu 10.04 LTS. Instructions here.
  • Dump the database using mysqldump from the old MySQL server.
  • Load the DB into the new database server.
  • Run the upgrade steps.
  • Validate the data and functionality.

As I tested and read up on the upgrade steps I soon found that this wasn’t going to be the big hassle I first imagined it to be. The details of the migration went like this:

  • Turn off all inbound SMTP email to the primary mail server. This is necessary as I want any emails bound for our ticketing system to queue up at the senders end as my migration involved a change of host for the ultimate destination for ticket emails.
  • Shutdown the old RT apache instance on the old web server and its MTA as well.
  • Shutdown the new RT apache instance on the new web server and its MTA as well.
  • Use mysqldump to copy the prod RT database to a .sql file on the new database server.

mysqldump –user={rtuser} –password={rtuserpasswd} –host={oldrtdbhost} –opt –skip-lock-tables –single-transaction –default-character-set=binary –databases {rtdbname} > {pathto .sql file}

  • Use mysql to load the .sql file into the new database server.

mysql –user={rtuser} –password={rtuserpasswd} –host={newrtdbhost} < {pathto .sql file}

  • Use the rt-setup-database script on the new RT web server to upgrade the DB from 3.2.2 to 3.7.87.

rt-setup-database –action upgrade –dba {rtuser} –prompt-for-dba-password

  • Build the schema upgrade script.

perl /usr/share/request-tracker3.8/etc/upgrade/ {rtdbname}:{dbhost} {rtusername} {rtuserpasswd} > {path to schema .sql}

  • Apply the schema upgrade script.

mysql –user {rtuser} –password={rtuserpasswd} -h {newrtdbhost} {rtdbname} < {path to schema .sql}

  • Use the rt-setup-database script on the new RT web server to upgrade the DB from 3.7.87 to 3.8.7.

rt-setup-database –action upgrade –dba {rtuser} –prompt-for-dba-password

  • Start the web server and MTA on the new RT web server
  • Logged in to the new RT instance and made sure it all looked okay. I had to tweak settings in the following locations:
    • The rights all needed redoing. I blew them all away and reassigned rights to my groups from scratch.
    • Because I had started using Exim4 integration I had to go through each queues config and update its correspondence and comment address.
    • I tuned the default dashboard to look more like our original 3.2.2 dashboard. All the new stuff can be turned on later. I didnt want to move the agents cheese too much at first.
  • I then had to jump on our primary MX and fix up all its forwarder configs for our queue email addresses so that emails addressed to our queues would be delivered to the new host.
  • Turned in bound email on the primary mail server back on.
  • Sent and received test messages.

That was it. Of course there were teething problems from the migration. These included:

  • I hadn’t checked the $Organization value in the config of our new RT to make sure it matched the value in the old config. I ended up having to fix it.
  • I had a few problems with the change of the queue correspondence and comment address. The exim4 integration expects a comment address of {queuename} Our comment addresses didn’t match this and I had to quickly update our forwarder configs to deliver to the correct address on the RT host.
  • I needed to give all the agents the ModifyCustomField right.
  • In my original exim4 reconfigure I hadnt configured exim to rewrite outbound emails from the host so all emails sent from RT left with an address of when what I wanted was A quick `dpkg–reconfigure exim4-config` allowed me to turn on the rewriting of sent emails and the address they should be rewritten as.
  • RT will truncate ticket transactions with large bodies. See the MaxInlineBody RTConfig option or have your agents adjust their settings in their preferences.
  • Agents didnt have the right to modify requestor details. Had to grant them the AdminUsers and ShowConfigTab rights.

Somethings I learnt and knowledge I want to retain:

  • To test how exim4 will handle an email address on a host use this command

exim -d -bt {email address}

That’s it. So far things have been running really well.