• Moving Kubernetes resources across namespaces

    If you have a series of yaml files without namespaces in their metadata, and add one and reapply the files, an entirely new series of resources will be created in the new namespace. Moving a series of resources, then, involves spinning up a new set of resources, migrating the data across, and taking down the old set.

  • Fixing a hanging git pull

    Occasionally git will hang on a push or pull using an SSH key until it times out. Using HTTP(S) mode instead is a workaround but undesired because it requires faffing about with passwords.

  • Moving from Docker Compose to Kubernetes

    We were running a Task Definition generated from a Docker Compose file in AWS’ ECS when we encountered an issue: we needed to run 11 containers in a single task (as they all needed to talk to each other), but the limit was 10. Looking at the documentation, it seemed that putting in a network they could communicate on in the ECS infrastructure was about as hard as migrating everything to Kubernetes, which would offer other benefits, so we decided to go for that instead.

  • Statement injection in C#

    We have a system that allows a user to write a single statement to be evaluated with a subset of C#: something like

  • Fridays, after the Sprint Ends

    Our sprints start on Monday and last for two weeks, and we release at the end of each one. This means we have some dead time on Friday: we want to start the release early to give us some time to recover if it goes wrong or we find an issue after release, but if it all goes well we don’t want to start the next sprint early, to give us some time off the treadmill.

  • Show complicated filtered data in EntityFramework Core

    We had a requirement for an API to, for a given user, return only records that user had permissions to see. We didn’t want to manually filter in every case, because the filters were quite complicated, and there was no guarantee the developers would remember to call them.

  • GitHub Copilot

    GitHub has recently released a product called Copilot, an advanced form of autocomplete that can complete blocks of code when prompted with a filename, comments, and a function contract (name, inputs and outputs), and potentially any other code above or even below the item being completed.

  • Storing encrypted Ansible vault keys safely with GPG

    Ansible vault keys are required when running Ansible playbooks to decrypt vault secrets. Because of this, it’s useful to store them somewhere accessible from the machine running the playbooks (on disk or in an environment variable as a secret, for example). If you don’t have such a centralised machine, one possibility is to store the keys in the same git repository as the playbooks, encrypted so that only the dev team can access them.

  • COVID and the Manchurian Plague

    After being linked a book describing the response to the Manchurian plague in Moukden, I read a few chapters and found that there are parallels between the Manchurian plague and COVID. Some paragraphs could easily have been written in the modern day. I took the book from the Internet Archive; the Google Books link didn’t work for me.

  • Mathematics at Warwick

    I got my degree (MMath) from the University of Warwick, which I attended from 2010 to 2014. I decided on there because having looked around Cambridge I thought I probably wouldn’t do too well there (in retrospect, likely correctly) and it had a good reputation for maths, in my mind.

  • Collapse EntityFramework Migrations

    On every major version release (or any other time you don’t think you’re ever going to want to roll back to a given version), it’s nice to collapse migrations into a single file so they all run faster and don’t do needless work (e.g. deleting and recreating data, creating and dropping tables, etc.).

  • Migrating an IE7 App

    I recently had the experience of updating a web app designed for IE7 to additionally work in modern browsers (users use predominantly Internet Explorer or Edge). Here I note some changes I needed to make – I couldn’t find any solid documentation on differences in old versions of IE, and much was trial and error.

  • Ansible Patterns

    I’ve been using Ansible for a while now, and there are certain common practices I’ve observed, recorded here in the spirit of Design Patterns.

  • Running PHP scripts targeting localhost for MySQL in Docker

    When targeting localhost as the host for a MySQL connection, PHP will attempt to connect using the configured socket instead of TCP. This can prove a problem when the socket does not exist, such as when you’re running inside a container. A simple solution is to target instead (or alternate appropriate hostname), but this is can involve a big change to an existing codebase, particularly if it wasn’t written to use external or obvious configuration files.

  • Running SQL scripts in dockerized MySQL

    You can redirect a SQL script through a mysql client by running:

  • Debugging ancient PHP scripts

    Modern browsers have a “feature” where if a page takes too long to send anything (headers / body) it times out. I can’t find a way to stop this on the frontend, but it’s annoying for debugging old, slow, half-broken scripts designed for IE7 (in PHP 5.4/Apache, where I am at the moment).

  • Reducing the Number of Colors in a PNG

    My dad recently asked that I reduce the number of colors in a clip-art icon of a server, so he could recolor it to red (downloading the image shows it’s made of stripes of slightly different colors, probably because it’s been saved as a JPG).

  • Continuing to Use Flash in 2021

    Adobe Flash becomes end-of-life in Janurary 2021. Despite that, it’s still used for business apps and tools. Most have upgrades, but perhaps you weren’t able to convince your boss to let you schedule one. This post gives some options on what you can do.

  • Typing Express' res.locals in TypeScript

    TypeScript is a good tool to use to prevent bugs in the inconsistent use of variables by ensuring that they’re all of the same type, at least. By default, this restriction does not apply to Express’ res.locals, which is a map of string to any. This can lead to issues, as res.locals can be set in middleware or earlier routers quite distant from where it is used, which can be hard to notice, and units tests often mock the data, and so it can’t be noticed that way either. This post shows a way to add types to res.locals.

  • Interpreting 'Us' Metaphorically

    Having enjoyed Jordan Peele’s “Get Out”, I checked out his next film “Us”, which I was rather less impressed with. I found it had too many plot holes, required the characters behave foolishly to propel the plot, and like “Get Out” delivered its alternative messages (e.g. on privilege) in a way that was undermined by other parts of the film.

  • Digital Drill Data Systems vs Petrolink: A Hacking Case

    I occasionally enjoy reading legal judgements for interesting interpretations of the law and crazy behaviour companies and people manage to get up to. Digital Drilling Data Systems, LLC v. Petrolink Services, Inc., No. 19-20116 (5th Cir. 2020) does not disappoint. I first looked into this after it was linked off the youtube-dl GitHub DMCA reversal by the EFF.

  • A Successful Dogwhistle, or Inciting Outrageous Outrage

    A billboard was recently put up in Vancouver, Canada with the message “I ❤️ JK Rowling”

  • Getting code coverage for manual or integration tests in NodeJS

    It can be useful to get code coverage for a manual test set in order to check that you’ve covered all the cases for the code you’ve written. Getting this is similar to the process for getting coverage for an integration test.

  • In the COVID lockdown without a smartphone

    Not having a smartphone, I’m more aware of the changes brought about by COVID that assumes everyone has one (that’s fairly recent, and that’s on a data plan or can access local Wi-Fi). My local bus stop no longer posts times, instead having a QR code / NFC which presumably sends you to a website with the times. This is going to be more certain to be accurate – I expect they’re changing them more frequently due to lower use – and can be enhanced with the standard “when will the bus get here” approximations which are common online or on the text screens in more central bus stops.

  • Testing without Dependency Injection: NodeJS

    NodeJS offers the ability to overwrite methods and values in imported modules. This can be useful for testing without dependency injection, for example if you have a large amount of legacy code, and you don’t want to refactor the architecture before the unit tests are in place.

  • Testing without Dependency Injection

    Dependency Injection is a software pattern where you pass dependencies to an object, instead of creating them inside the object. Around a decade ago, it became very popular to use dependency injection for everything – not only for swapping out dependencies at runtime (the problem it was the solution for initially) but for creating a dependency with a particular lifecycle (e.g. singleton or once per session for a webapp), replacing dependencies for testing, and even for instantiating dependencies which were always going to be the same, which were “injected” as the concrete class and “registered” as the same.

  • Importing modules into a browser REPL

    It can be useful to import a module for a single session without saving it as a scratchpad, or to be able to use it against a webpage.

  • Using JavaScript's Proxy to add methods to a data object

    Assume we have a document obtained from a database as a JSON object, and we want to add additional properties, or override existing properties, preferably without modifying the original object.

  • Changes in the Definition of Racism

    From until the 1970s until about the 2010s, the “right” way to oppose racism was to be colour-blind – to avoid considering race as much as possible, and perhaps to even argue that it didn’t exist. Nowadays, the concern is systemic racism: the observation that even our attempts at colour-blindness haven’t lead to equal outcomes, and so the institutions (and the people in them) may be inherently anti-black in a way we didn’t notice before. Recently, “anti-racist” has become suggestive of “pro-black” – treatment like affirmative action, or quota systems, or treating minority students in a way that reflects their unique life experiences are the ways to go. Unfortunately, this conflicts with the legacy definition of racism as “treating people differently depending on their race”.

  • Using RxJS's switchMap to display a loading spinner after a delay

    The problem is to display a loading spinner while requests are in-flight. The complexity is that everything’s event-driven, so the components themselves don’t know when this happens. A solution to this complexity is to use RxJs or some other event management system.

  • Removing boilerplate in REST API tests

    While converting our backend REST API integration tests from Cucumber to JUnit, I saw the chance to simplify our rollback of test data.

  • Notes on Jenkins Declarative Pipelines

    At Lhasa on the Kaptis project we used Jenkins declarative pipelines to build, test and create artifacts after every commit. Here I note down some things I’ve learned.

  • Useful snippets from a reusable Angular Typescript input component

    Angular allows creation of reusable components to simplify development. I created an input component in order to simplify how the HTML form pages looked (it allowed a large block of code to be abstracted and replaced; later it allowed all the error texts to be put in one place instead of being repeated for each field).

  • Notes on Spring Data

    I find the Spring documentation generally comprehensive, but most useful after you already know the right answer to your questions. Here I note down some things I’ve learned this past project.

  • Aborting a Cucumber rerun of failed tests if too many failed

    Cucumber allows you to rerun failed tests by defining a test runner that specifies the features to run to be those with failed last time, as output by the built-in rerun plugin. We wanted to fail the build entirely if too many tests failed the first run, as we found tests weren’t being fixed (they would always fail the first time, refresh the database, and pass the second time: some earlier tests weren’t properly cleaning up after themselves).

  • Using JUnit5 Extensions to allow pushing tests for code not yet implemented

    Edit 18th May 2020: I’d now recommend just using @Disabled("Ticket XX-111") instead of having an entirely separate annotation with different behaviour. This means that you can search your codebase for the presence of disabled tags with closed tickets, and I think the tradeoff of this not being automated is worth the gain of not having another annotation and not running tests every build that aren’t expected to pass. You could automate this with a nightly job, for example: run through, find all references to tickets, look on JIRA to see if they’re closed, maybe make some metrics as to how long tests stay in this state.

  • Incomplete documentation is better than no documentation

    I’ve encountered many systems that were poorly documented (or entirely undocumented) that I’ve had to figure out in the course of my work. Sometimes I get lucky and there’s somebody still in the company who understands what’s going on and can explain it to me. Sometimes there isn’t.

  • Getting DataTables to map to classes in Cucumber version 4 and above

    In Cucumber 2 and below, you could reference classes containing primitive elements in step definitions, and data tables would be mapped automatically.

  • Testing an Angular AsyncValidatorFn using Jasmine

    The Angular documentation on validation mentions that there exists an AsyncValidatorFn counterpart to ValidatorFn, but doesn’t give any details in implementing it other than the function prototype. You can guess that it’s similar to ValidatorFn in the same way that AsyncValidator is similar to Validator, and this turns out to be correct as far as I can see.

  • Debugging by example: Build log liquibase error

    At work on Thursday, we encountered a particularly thorny issue. I think it would be beneficial to write up a log of how I attempted to solve the problem, what went right, what went wrong, and what I could have done better.

  • I lost a day to relative URLs

    We’d been having problems with this feature branch already. It was supposed to change the behaviour on logging in to redirect you to where you’d been just before. It worked fine locally, but failed on the build server.

  • Web Development upskilling: Creating a minimap for an SVG

    I find it easier to learn new skills if I have a project to work on. I knew that for my current project, I want to learn more CSS, vanilla JavaScript and SVG. I thought a project that could involve all three was creating a minimap: take a large SVG image, duplicate it in miniature, and allow the user to drag a square around the miniature replica to move the view around the larger one. Somewhat like the minimap in an RTS like Age of Empires (although that involves a simplification in the smaller map I don’t think I’d go for).

  • On Paul Graham's Bus Ticket Theory of Genius

    I read Paul Graham’s essays when they get linked on the other blogs I read, which is semi-regularly. The Bus Ticket Theory of Genius has just showed up on Hacker News, wherein Graham argues that to do great work requires natural ability, determination, and an obssessive interest in a particular topic, with the last one serving as a good proxy for either of the first two.

  • Basic CSS: Opacity on overlapping components

    Last week, I designed an icon using two separate overlapping SVG icons (a flask with a line through it). This was to represent whether to show or hide assays (on a graph of assays and key event relations), and I wanted to fade out the icon if assays were hidden. I tried this using color: rgba(255, 255, 255, 0.5) for both icons. This didn’t work:

  • Using Git over HTTPS through an SSL briged proxy

    I’ve recently joined an organisation that uses Forcepoint to monitor and SSL bridge all HTTPS connections. This means that connections to GitHub over HTTPS fail, as the GitHub certificate is replaced with Forcepoint’s, with the error message

  • Using Firefox at work

    I use Firefox at work. Unlike the prime focus of their advertising campaigns, I’m not interested in the privacy protections. I use it primarily for two features that haven’t yet made it to Chrome – containers (cookie isolation) and tab hiding.

  • A Use-Case for Monkeypatching Python's stdlib

    Some languages, such as C# and Kotlin, allow the declaration of extension methods – new methods callable on existing classes, without needing access to the original source code. Some languages, such as Python, go further and allow you to modify the behaviour of existing function calls (known as “monkeypatching”).

  • Learning other people's productivity tools

    One of the most important things I learn from demos and show & tells are frequently not what the presenter meant to put across, but all the minor things that show up when they’re setting up their machine, and moving around it to show what they need to. Windows has a way to disable notifications? What’s that editor? Woah, how’d you format that file? Outlook has a dark mode??

  • GitHub tech tests

    Many companies put tech challenges (to be completed when applying for the job) on the public web. If there’s a challenge on GitHub, there’s a good chance that people will put their solutions on GitHub as well, which can be useful to compare your own solution against / take inspiration / blatantly cheat.1

subscribe via RSS