Posts
-
Add Directory Read Permissions to a Managed Identity in Azure
Normal roles can be added to a managed identity using the
az role assignment createCLI command. The “Directory Reader” role (and other similar roles) cannot be assigned this way as they are Graph roles and not normal roles (I assume, I can’t find a way to add them using this command). -
Dude is Gender Neutral
When I use a word, it means exactly what I want it to mean, no more and no less.
-
Solve Specific Problems
Suppose you have a particular problem. When solving it, you can look into things which might solve only that problem, or try to see it as a case of a more general problem, which you already know how to solve. This is common, but the solution to the general problem can be much more complicated than a solution to the specific problem.
-
Movie Review: I Saw the TV Glow
I Saw the TV Glow is a movie with heavy queer themes, commonly interpreted as being about coming out (and not doing so). I felt it said a lot about masking and escapism.
-
Restricting user permissions in AWS
For most projects, restricting permissions to the level of “power user” or “admin” (with the occasional “read only” user) will suffice. Other projects want more control.
-
Foreshadowing in Whodunnit Slashers
Slasher movies are a type of film where a costumed killer murders a group of friends in close succession. Whodunnit slashers are where the killer is one of the friend group, instead of being a supernatural force or human escapee.
-
Local LLMs with llama.cpp
Most local LLM use is through some high-level application such as SillyTavern. These offer advantages such as automatically templating input appropriately, setting context sizes, and potentially offering other features, such as image generation. They’re often focused on chatbots, and giving certain personalities to the characters chatted with.
-
Strange Illness Symptoms: Enhanced Taste
I recently caught a respiratory illness, and was hit by a symptom I haven’t had before: an enhanced sense of taste. People talk about COVID taking away the sense of taste, and I guess this is some sort of the opposite of that.
-
Dependency Injection and Lifetimes
Dependency Injection is useful in the case where you want to manage lifetimes of instances of classes you request. Using the Microsoft DI terminology, we can call these “Singleton”, “Scoped” and “Transient”. Singleton is where you want the same instance every time you ask for one. Transient is where you always want a new instance of a class. Scoped is where it gets interesting – under some condition, you want the same instance, under different circumstances you want a different one. For web applications, this tends to be a single request – inside a request, you want the same instance; otherwise you want a new one.
-
Ubuntu Pro is surprisingly cheap
Ubuntu LTS releases provide security updates in general support for 5 years, after which you are encouraged to upgrade to a newer release (which comes every 2 years). If you don’t want to, Canonical offer “Extended Security Maintenance” for another 5 years of security updates.
-
A Year of Z
Z is a replacement to
cd, that allows you to jump directly to a certain directory by providing a set of regexes. For example,z kolmtakes me to the directory~/thirdparty/kolmafia/, and at work,z les2 mgttakes me to~/ctg-twc/infrastructure-modules2/mgt. -
Moving from Vim to Neovim
I recently installed Neovim (nvim) after about a decade of vim, hoping it would fix a problem I had with an extension. That turned out to be unrelated, but I kept with it – coc.nvim focuses on neovim, and having access to all features by default is helpful.
-
Performance
Performance is one of the properties of a program that the learned developer has a quote at hand to discount – as premature optimisation is the root of all evil (carefully omitting the second part of the quote). The natural consequence is that code is written to be more readable (as code is written once, but read many times), with better variable names (good code is self-documenting) and more reusability (Don’t Repeat Yourself), but as performance is not a concern, it is ignored (or occasionally worked against, preferring developer experience over user experience).
-
Flies and Smoke
A guard looks out into the distance, and sees a black cloud. “Ah, smoke!” he thinks, “There’s a fire there.”. He remarks as much to his friend, who, slightly more eagle-eyed, identifies the cloud as flies. “Ah,” he thinks, “Then there is not a fire there after all.”
-
Avoiding blank lines in Terraform templates
You can use Terraform templates to include or omit lines of configuration (or scripts) based on some condition:
this is always present %{ if condition } this is present %{ endif } this is also always present -
Migrating deprecated aws_s3_bucket in Terraform AWS Provider 4 and above
Terraform AWS Provider 4.0 introduced a series of new, smaller resources intended to replace the monolithic
aws_s3_bucketresource, which was deprecated. While it hasn’t been removed even in 5.0, it’s nice to migrate to the new subcomponents under the assumption it will eventually prevent major version updates. -
Reflections on a Back Injury
I injured my back on Wednesday afternoon, picking up a tennis ball from the ground too quickly. This is some notes about the experience.
-
Fixing broken characters in WSL
By default in WSL, certain characters used by particular applications (e.g. ⚠, the warning sign), show up as a question mark in a box. This is because the default font used, Consolas, does not support these characters.
-
Sometimes language does affect performance
A common refrain is that if rewriting your application in a different language improves the performance, this is mostly due to being able to rearchitect to remove technical debt, instead of due to the language itself.
-
Choosing the language for the job
If you’re a developer who knows multiple languages and you’re working on a new task, you may, depending on how modular the codebase is, have the flexibility to pick a new language for it. I think this is something which should be strongly considered.
-
Looking for subdomains
It’s useful to have a tool that can find subdomains of a given domain. I expect this is generally used for security testing, but I use it to look for development sites that other developers haven’t documented for projects I’m working on.
-
Using Containers for development without DevContainers
DevContainers are a formal specification for adding JSON config to a Dockerfile to run applications and editors inside a container, and to be able to share the configuration that allows you to do this with others. But even without that, it’s still valuable (and somewhat easier or more flexible) to use containers in your local development.
-
Classifying ChatGPT Exploits
ChatGPT was recently released to the general public and promptly exploited in a variety of ways. This post serves a brief overview of exploits known to me, sorted into particular categories. Exploits may belong to multiple categories. As OpenAI trains the bots, a broken exploit might be “fixed” by adding a technique from another category.
-
Curling ElasticSearch
When trying to debug ElasticSearch index issues, it’s generally nicest use Kibana to provide a graphical interface to assist. But sometimes you can’t forward the Elastic instance to a place where you can run Kibana. Sometimes you just have a CLI.
-
Variable shadowing is good
Variable shadowing is when you give a variable the same name as a variable that already exists. Linters and tools generally complain when you do this (e.g. no-shadow on eslint, mypy or TypeScript will report errors), and the language may forbid in entirely. Despite that, I find that it often makes code easier to read, instead of harder.
-
Azure App Service can fail to deploy due to free plan
I recently failed to deploy a project to an Azure App Service with a variety of mysterious errors. The project ran fine locally, but:
-
Deliberate Forgetting
In my third year of university, I tried to forget a specific thing, and succeeded.
-
Coding in Wiki Templates
My first major programming projects, before university, involved coding using wiki templates.
-
Moving from Json.NET (Newtonsoft) to System.Text.Json
In mid-2019, Microsoft released the
System.Text.Jsonlibrary, intended to be a faster and lower memory JSON serializer / deserializer than the existing and still most popularNewtonsoft.Jsonlibrary. At the time, there were many features missing compared toNewtonsoft.Json, and it was difficult to migrate. Now, 3 years on, it is easier. -
Installing private GitHub packages from CI
While developing microservices or other projects, it can be useful to extract shared functionality into a library that can be used across all projects. To avoid copying or vendoring, it’s convenient to put this library somewhere external where it can be installed – often an external registry. In the absence of a registry, a library can be installed from source from a GitHub repository.
-
100% coverage is not sufficient
I recently posted a pull request that I was certain would work correctly as I had 100% code coverage. I was mistaken.
-
Two Factor is more secure than One Factor
HSBC sent me an email at the start of February stating that they were simplifying the login process, and that I no longer needed to enter a password (which they call “an answer to a memorable question”), only my Secure Key (a probably time-based one time password).
-
Azote
Those who have adopted these opinions, represent the atom of azote by the number 1.75. We consider the 5 compounds of azote and oxygen, as composed of 1 atom azote, united with 1, 2, 3, 4, and 5, atoms of oxygen.
1831, Thomas Thomson, A System of Chemistry of Inorganic Bodies, Volume 1, page 133 -
Questing through the tax office
Last year, I signed up to file self assessment. I wanted to claim tax relief on charitable donations and VCT investments. The gov.uk website is tells you what to do in those cases: for charitable donations, either file self assessment or ask HMRC directly, and for VCTs, file self assessment. I knew a few people who filed self-assessment already, so I was expecting the process to be straightfoward. I registered as someone who isn’t self-employed, filled in form SA1 mentioning VCTs and charity in the “other reasons” box, and waited.
-
Moving Kubernetes resources across namespaces
If you have a series of
yamlfiles withoutnamespaces in their metadata, and add one and reapplythe files, an entirely new series of resources will be created in the new namespace. Moving a series of resources, then, involves spinning up a new set of resources, migrating the data across, and taking down the old set. -
Fixing a hanging git pull
Occasionally
gitwill hang on apushorpullusing an SSH key until it times out. Using HTTP(S) mode instead is a workaround but undesired because it requires faffing about with passwords. -
Moving from Docker Compose to Kubernetes
We were running a Task Definition generated from a Docker Compose file in AWS’ ECS when we encountered an issue: we needed to run 11 containers in a single task (as they all needed to talk to each other), but the limit was 10. Looking at the documentation, it seemed that putting in a network they could communicate on in the ECS infrastructure was about as hard as migrating everything to Kubernetes, which would offer other benefits, so we decided to go for that instead.
-
Statement injection in C#
We have a system that allows a user to write a single statement to be evaluated with a subset of C#: something like
-
Fridays, after the Sprint Ends
Our sprints start on Monday and last for two weeks, and we release at the end of each one. This means we have some dead time on Friday: we want to start the release early to give us some time to recover if it goes wrong or we find an issue after release, but if it all goes well we don’t want to start the next sprint early, to give us some time off the treadmill.
-
Show complicated filtered data in EntityFramework Core
We had a requirement for an API to, for a given user, return only records that user had permissions to see. We didn’t want to manually filter in every case, because the filters were quite complicated, and there was no guarantee the developers would remember to call them.
-
GitHub Copilot
GitHub has recently released a product called Copilot, an advanced form of autocomplete that can complete blocks of code when prompted with a filename, comments, and a function contract (name, inputs and outputs), and potentially any other code above or even below the item being completed.
-
Storing encrypted Ansible vault keys safely with GPG
Ansible vault keys are required when running Ansible playbooks to decrypt vault secrets. Because of this, it’s useful to store them somewhere accessible from the machine running the playbooks (on disk or in an environment variable as a secret, for example). If you don’t have such a centralised machine, one possibility is to store the keys in the same git repository as the playbooks, encrypted so that only the dev team can access them.
-
COVID and the Manchurian Plague
After being linked a book describing the response to the Manchurian plague in Moukden, I read a few chapters and found that there are parallels between the Manchurian plague and COVID. Some paragraphs could easily have been written in the modern day. I took the book from the Internet Archive; the Google Books link didn’t work for me.
-
Mathematics at Warwick
I got my degree (MMath) from the University of Warwick, which I attended from 2010 to 2014. I decided on there because having looked around Cambridge I thought I probably wouldn’t do too well there (in retrospect, likely correctly) and it had a good reputation for maths, in my mind.
-
Collapse EntityFramework Migrations
On every major version release (or any other time you don’t think you’re ever going to want to roll back to a given version), it’s nice to collapse migrations into a single file so they all run faster and don’t do needless work (e.g. deleting and recreating data, creating and dropping tables, etc.).
-
Migrating an IE7 App
I recently had the experience of updating a web app designed for IE7 to additionally work in modern browsers (users use predominantly Internet Explorer or Edge). Here I note some changes I needed to make – I couldn’t find any solid documentation on differences in old versions of IE, and much was trial and error.
-
Ansible Patterns
I’ve been using Ansible for a while now, and there are certain common practices I’ve observed, recorded here in the spirit of Design Patterns.
-
Running PHP scripts targeting localhost for MySQL in Docker
When targeting
localhostas the host for a MySQL connection, PHP will attempt to connect using the configured socket instead of TCP. This can prove a problem when the socket does not exist, such as when you’re running inside a container. A simple solution is to target127.0.0.1instead (or alternate appropriate hostname), but this is can involve a big change to an existing codebase, particularly if it wasn’t written to use external or obvious configuration files. -
Running SQL scripts in dockerized MySQL
You can redirect a SQL script through a mysql client by running:
-
Debugging ancient PHP scripts
Modern browsers have a “feature” where if a page takes too long to send anything (headers / body) it times out. I can’t find a way to stop this on the frontend, but it’s annoying for debugging old, slow, half-broken scripts designed for IE7 (in PHP 5.4/Apache, where I am at the moment).
-
Reducing the Number of Colors in a PNG
My dad recently asked that I reduce the number of colors in a clip-art icon of a server, so he could recolor it to red (downloading the image shows it’s made of stripes of slightly different colors, probably because it’s been saved as a JPG).
-
Continuing to Use Flash in 2021
Adobe Flash becomes end-of-life in Janurary 2021. Despite that, it’s still used for business apps and tools. Most have upgrades, but perhaps you weren’t able to convince your boss to let you schedule one. This post gives some options on what you can do.
-
Typing Express' res.locals in TypeScript
TypeScript is a good tool to use to prevent bugs in the inconsistent use of variables by ensuring that they’re all of the same type, at least. By default, this restriction does not apply to Express’
res.locals, which is a map ofstringtoany. This can lead to issues, asres.localscan be set in middleware or earlier routers quite distant from where it is used, which can be hard to notice, and units tests often mock the data, and so it can’t be noticed that way either. This post shows a way to add types tores.locals. -
Interpreting 'Us' Metaphorically
Having enjoyed Jordan Peele’s “Get Out”, I checked out his next film “Us”, which I was rather less impressed with. I found it had too many plot holes, required the characters behave foolishly to propel the plot, and like “Get Out” delivered its alternative messages (e.g. on privilege) in a way that was undermined by other parts of the film.
-
Digital Drill Data Systems vs Petrolink: A Hacking Case
I occasionally enjoy reading legal judgements for interesting interpretations of the law and crazy behaviour companies and people manage to get up to. Digital Drilling Data Systems, LLC v. Petrolink Services, Inc., No. 19-20116 (5th Cir. 2020) does not disappoint. I first looked into this after it was linked off the youtube-dl GitHub DMCA reversal by the EFF.
-
A Successful Dogwhistle, or Inciting Outrageous Outrage
A billboard was recently put up in Vancouver, Canada with the message “I ❤️ JK Rowling”
-
Getting code coverage for manual or integration tests in NodeJS
It can be useful to get code coverage for a manual test set in order to check that you’ve covered all the cases for the code you’ve written. Getting this is similar to the process for getting coverage for an integration test.
-
In the COVID lockdown without a smartphone
Not having a smartphone, I’m more aware of the changes brought about by COVID that assumes everyone has one (that’s fairly recent, and that’s on a data plan or can access local Wi-Fi). My local bus stop no longer posts times, instead having a QR code / NFC which presumably sends you to a website with the times. This is going to be more certain to be accurate – I expect they’re changing them more frequently due to lower use – and can be enhanced with the standard “when will the bus get here” approximations which are common online or on the text screens in more central bus stops.
-
Testing without Dependency Injection: NodeJS
NodeJS offers the ability to overwrite methods and values in imported modules. This can be useful for testing without dependency injection, for example if you have a large amount of legacy code, and you don’t want to refactor the architecture before the unit tests are in place.
-
Testing without Dependency Injection
Dependency Injection is a software pattern where you pass dependencies to an object, instead of creating them inside the object. Around a decade ago, it became very popular to use dependency injection for everything – not only for swapping out dependencies at runtime (the problem it was the solution for initially) but for creating a dependency with a particular lifecycle (e.g. singleton or once per session for a webapp), replacing dependencies for testing, and even for instantiating dependencies which were always going to be the same, which were “injected” as the concrete class and “registered” as the same.
-
Importing modules into a browser REPL
It can be useful to import a module for a single session without saving it as a scratchpad, or to be able to use it against a webpage.
-
Using JavaScript's Proxy to add methods to a data object
Assume we have a document obtained from a database as a JSON object, and we want to add additional properties, or override existing properties, preferably without modifying the original object.
-
Changes in the Definition of Racism
From until the 1970s until about the 2010s, the “right” way to oppose racism was to be colour-blind – to avoid considering race as much as possible, and perhaps to even argue that it didn’t exist. Nowadays, the concern is systemic racism: the observation that even our attempts at colour-blindness haven’t lead to equal outcomes, and so the institutions (and the people in them) may be inherently anti-black in a way we didn’t notice before. Recently, “anti-racist” has become suggestive of “pro-black” – treatment like affirmative action, or quota systems, or treating minority students in a way that reflects their unique life experiences are the ways to go. Unfortunately, this conflicts with the legacy definition of racism as “treating people differently depending on their race”.
-
Using RxJS's switchMap to display a loading spinner after a delay
The problem is to display a loading spinner while requests are in-flight. The complexity is that everything’s event-driven, so the components themselves don’t know when this happens. A solution to this complexity is to use RxJs or some other event management system.
-
Removing boilerplate in REST API tests
While converting our backend REST API integration tests from Cucumber to JUnit, I saw the chance to simplify our rollback of test data.
-
Notes on Jenkins Declarative Pipelines
At Lhasa on the Kaptis project we used Jenkins declarative pipelines to build, test and create artifacts after every commit. Here I note down some things I’ve learned.
-
Useful snippets from a reusable Angular Typescript input component
Angular allows creation of reusable components to simplify development. I created an input component in order to simplify how the HTML form pages looked (it allowed a large block of code to be abstracted and replaced; later it allowed all the error texts to be put in one place instead of being repeated for each field).
-
Notes on Spring Data
I find the Spring documentation generally comprehensive, but most useful after you already know the right answer to your questions. Here I note down some things I’ve learned this past project.
-
Aborting a Cucumber rerun of failed tests if too many failed
Cucumber allows you to rerun failed tests by defining a test runner that specifies the features to run to be those with failed last time, as output by the built-in rerun plugin. We wanted to fail the build entirely if too many tests failed the first run, as we found tests weren’t being fixed (they would always fail the first time, refresh the database, and pass the second time: some earlier tests weren’t properly cleaning up after themselves).
-
Using JUnit5 Extensions to allow pushing tests for code not yet implemented
Edit 18th May 2020: I’d now recommend just using
@Disabled("Ticket XX-111")instead of having an entirely separate annotation with different behaviour. This means that you can search your codebase for the presence of disabled tags with closed tickets, and I think the tradeoff of this not being automated is worth the gain of not having another annotation and not running tests every build that aren’t expected to pass. You could automate this with a nightly job, for example: run through, find all references to tickets, look on JIRA to see if they’re closed, maybe make some metrics as to how long tests stay in this state. -
Incomplete documentation is better than no documentation
I’ve encountered many systems that were poorly documented (or entirely undocumented) that I’ve had to figure out in the course of my work. Sometimes I get lucky and there’s somebody still in the company who understands what’s going on and can explain it to me. Sometimes there isn’t.
-
Getting DataTables to map to classes in Cucumber version 4 and above
In Cucumber 2 and below, you could reference classes containing primitive elements in step definitions, and data tables would be mapped automatically.
-
Testing an Angular AsyncValidatorFn using Jasmine
The Angular documentation on validation mentions that there exists an AsyncValidatorFn counterpart to ValidatorFn, but doesn’t give any details in implementing it other than the function prototype. You can guess that it’s similar to ValidatorFn in the same way that AsyncValidator is similar to Validator, and this turns out to be correct as far as I can see.
-
Debugging by example: Build log liquibase error
At work on Thursday, we encountered a particularly thorny issue. I think it would be beneficial to write up a log of how I attempted to solve the problem, what went right, what went wrong, and what I could have done better.
-
I lost a day to relative URLs
We’d been having problems with this feature branch already. It was supposed to change the behaviour on logging in to redirect you to where you’d been just before. It worked fine locally, but failed on the build server.
-
Web Development upskilling: Creating a minimap for an SVG
I find it easier to learn new skills if I have a project to work on. I knew that for my current project, I want to learn more CSS, vanilla JavaScript and SVG. I thought a project that could involve all three was creating a minimap: take a large SVG image, duplicate it in miniature, and allow the user to drag a square around the miniature replica to move the view around the larger one. Somewhat like the minimap in an RTS like Age of Empires (although that involves a simplification in the smaller map I don’t think I’d go for).
-
On Paul Graham's Bus Ticket Theory of Genius
I read Paul Graham’s essays when they get linked on the other blogs I read, which is semi-regularly. The Bus Ticket Theory of Genius has just showed up on Hacker News, wherein Graham argues that to do great work requires natural ability, determination, and an obssessive interest in a particular topic, with the last one serving as a good proxy for either of the first two.
-
Basic CSS: Opacity on overlapping components
Last week, I designed an icon using two separate overlapping SVG icons (a flask with a line through it). This was to represent whether to show or hide assays (on a graph of assays and key event relations), and I wanted to fade out the icon if assays were hidden. I tried this using
color: rgba(255, 255, 255, 0.5)for both icons. This didn’t work: -
Using Git over HTTPS through an SSL briged proxy
I’ve recently joined an organisation that uses Forcepoint to monitor and SSL bridge all HTTPS connections. This means that connections to GitHub over HTTPS fail, as the GitHub certificate is replaced with Forcepoint’s, with the error message
-
Using Firefox at work
I use Firefox at work. Unlike the prime focus of their advertising campaigns, I’m not interested in the privacy protections. I use it primarily for two features that haven’t yet made it to Chrome – containers (cookie isolation) and tab hiding.
-
A Use-Case for Monkeypatching Python's stdlib
Some languages, such as C# and Kotlin, allow the declaration of extension methods – new methods callable on existing classes, without needing access to the original source code. Some languages, such as Python, go further and allow you to modify the behaviour of existing function calls (known as “monkeypatching”).
-
Learning other people's productivity tools
One of the most important things I learn from demos and show & tells are frequently not what the presenter meant to put across, but all the minor things that show up when they’re setting up their machine, and moving around it to show what they need to. Windows has a way to disable notifications? What’s that editor? Woah, how’d you format that file? Outlook has a dark mode??
-
GitHub tech tests
Many companies put tech challenges (to be completed when applying for the job) on the public web. If there’s a challenge on GitHub, there’s a good chance that people will put their solutions on GitHub as well, which can be useful to compare your own solution against / take inspiration / blatantly cheat.1
subscribe via RSS