https://iambenzo.com Zola en Fri, 11 Jul 2025 00:00:00 +0000 Fri, 11 Jul 2025 00:00:00 +0000 Ben Burbage https://iambenzo.com/posts/k8s-kind-cr-ssl/ https://iambenzo.com/posts/k8s-kind-cr-ssl/ <h1 id="context"><a class="zola-anchor" href="#context" aria-label="Anchor link for: context">Context</a></h1> <p>Often, I want to play with a Kubernetes cluster without having to pay a cloud provider for compute, or by setting up a home lab cluster with <code>kubeadm</code>. In these times, I reach for <a rel="noopener" target="_blank" href="https://kind.sigs.k8s.io/">K8s Kind</a> (although I'd love to have a home lab cluster).</p> <p>Simply put, Kind allows me to run a small Kubernetes cluster within Docker/Podman/You Choose. Whilst the number of pods I can run is limited to the power of my machine, it's great for running small proof-of-concepts (PoCs) and exploring the configuration of a product before deploying it to a <em>real</em> cluster.</p> <p>I do have a small tip that I can share with you now to increase that pod limit if you're on a Debian-based machine:</p> <pre data-lang="sh" class="language-sh z-code"><code class="language-sh" data-lang="sh"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> sysctl fs.inotify.max_user_watches=524288</span> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> sysctl fs.inotify.max_user_instances=512</span> </span></code></pre> <p>These commands boost the number of file watchers available for the duration of your session, returning those values to their defaults once you reboot. More file watches = more pods.</p> <p>If Docker/Others were already running before you ran these commands, you might want to restart those services.</p> <p>But that's not what I really want to share...</p> <h1 id="problem"><a class="zola-anchor" href="#problem" aria-label="Anchor link for: problem">Problem</a></h1> <p>Recently, I decided to spend too much time writing YAML files, in playing with the <a rel="noopener" target="_blank" href="https://argoproj.github.io/">Argo Project</a>. Mostly to explore some features and generally play with GitOps type processes. As part of my test-bed setup, I deployed <a rel="noopener" target="_blank" href="https://forgejo.org/">Forgejo</a> to replicate a scenario where there's a Git server and container registry hosted alongside the GitOps tooling.</p> <p>My goals and reasons aren't really important here. The problem I came up against was <em>after</em> I had code repos set up, events firing off workflows to build and push container images, and finally automatic deployment to a target Kubernetes namespace.</p> <p>All of that worked, as defined in the glorious YAML files I had painstakingly created. The problem was at deployment time: <code>ImagePullBackOff</code> - Kubernetes couldn't pull my container images because it couldn't verify the self-signed SSL certificate I'd used for the Forgejo Ingress rule.</p> <p><em>How did I get around that problem?</em> Well, first, I destroyed my Kind cluster.</p> <p><em>Was I having a tantrum?</em> Maybe. <em>Would I have needed to nuke everything I created anyway?</em> Unfortunately so.</p> <h1 id="resolution"><a class="zola-anchor" href="#resolution" aria-label="Anchor link for: resolution">Resolution</a></h1> <p>The reason I had to destroy my Kind cluster is that the container image used by Kind as a virtual Kubernetes node doesn't seem to contain a text editor. No Vim, no Vi, not even Nano. So I couldn't simply shell into the node, make a few edits and <code>service containerd restart</code> my way to happiness.</p> <p>The first modification I needed to make is in the ContainerD config file. This is typically located at <code>/etc/containerd/config.toml</code> on a Kubernetes node.</p> <p><a rel="noopener" target="_blank" href="https://kind.sigs.k8s.io/docs/user/private-registries/#use-a-certificate">A bit of reading</a> pointed me to a way to modify this file upon cluster creation. Cool. I modified my Kind cluster definition (another YAML file) to look a little like this:</p> <pre data-lang="yaml" class="language-yaml z-code"><code class="language-yaml" data-lang="yaml"><span class="z-source z-yaml"><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">kind</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">Cluster</span> </span><span class="z-source z-yaml"><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">apiVersion</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">kind.x-k8s.io/v1alpha4</span> </span><span class="z-source z-yaml"><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">name</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">plaything</span> </span><span class="z-source z-yaml"><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">containerdConfigPatches</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-keyword z-control z-flow z-block-scalar z-literal z-yaml">|</span><span class="z-storage z-modifier z-chomping-indicator z-yaml">-</span> </span><span class="z-source z-yaml"><span class="z-string z-unquoted z-block z-yaml"> [plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry] </span></span><span class="z-source z-yaml"><span class="z-string z-unquoted z-block z-yaml"> config_path = &quot;/etc/containerd/certs.d&quot; </span></span><span class="z-source z-yaml"><span class="z-string z-unquoted z-block z-yaml"></span><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">nodes</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">role</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">control-plane</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">extraPortMappings</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">containerPort</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-constant z-numeric z-integer z-decimal z-yaml">80</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">hostPort</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-constant z-numeric z-integer z-decimal z-yaml">80</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">protocol</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">TCP</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">containerPort</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-constant z-numeric z-integer z-decimal z-yaml">443</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">hostPort</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-constant z-numeric z-integer z-decimal z-yaml">443</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">protocol</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">TCP</span> </span></code></pre> <p>The magic is under the <code>containerdConfigPatches</code> key. It's basically telling ContainerD that some configuration exists under the defined directory.</p> <p>The next thing I needed to do was to create some config under that <code>config_path</code> for my self-hosted container registry. I had my Forgejo instance running under the <code>git.benzo.test</code> hostname, thanks to some <a rel="noopener" target="_blank" href="https://dnsmasq.org/doc.html">Dnsmasq</a> magic that could have also been achieved with an extra entry in my machine's <code>hosts</code> file.</p> <blockquote> <p>Note: <code>git.benzo.test</code> is a local-only hostname, which isn't internet facing - it won't work for you.</p> </blockquote> <p>I'm telling you the hostname because it defines the location and the contents of the next file we need to create: <code>/etc/containerd/certs.d/&lt;hostname&gt;/hosts.toml</code>. Because I wanted this configuration to be repeatable, I chose to create this file locally and then mount it to the Kind node at cluster creation time.</p> <p>I fired up my text editor and smashed the following keys:</p> <pre data-lang="toml" class="language-toml z-code"><code class="language-toml" data-lang="toml"><span class="z-source z-toml"><span class="z-meta z-tag z-key z-toml"><span class="z-entity z-name z-tag z-toml">server</span></span> <span class="z-punctuation z-definition z-key-value z-toml">=</span> <span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>git.benzo.test<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span> </span><span class="z-source z-toml"> </span><span class="z-source z-toml"><span class="z-punctuation z-definition z-table z-begin z-toml">[</span><span class="z-meta z-tag z-table z-toml"><span class="z-entity z-name z-table z-toml">host</span><span class="z-punctuation z-separator z-table z-toml">.</span><span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>git.benzo.test<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span></span><span class="z-punctuation z-definition z-table z-end z-toml">]</span> </span><span class="z-source z-toml"> <span class="z-meta z-tag z-key z-toml"><span class="z-entity z-name z-tag z-toml">capabilities</span></span> <span class="z-punctuation z-definition z-key-value z-toml">=</span> <span class="z-punctuation z-definition z-array z-begin z-toml">[</span><span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>pull<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span><span class="z-punctuation z-separator z-array z-toml">,</span> <span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>resolve<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span><span class="z-punctuation z-definition z-array z-end z-toml">]</span> </span><span class="z-source z-toml"> <span class="z-meta z-tag z-key z-toml"><span class="z-entity z-name z-tag z-toml">skip_verify</span></span> <span class="z-punctuation z-definition z-key-value z-toml">=</span> <span class="z-constant z-language z-toml">true</span> </span></code></pre> <p>This config tells ContainerD, <em>"Hey! There's this server...you can pull and resolve container images there...but keep your eyes off my dodgy self-signed SSL certificate. It's good, I promise."</em></p> <p>The last thing I needed to do was to place that config file in the right location, via an additional modification to my Kind cluster definition:</p> <pre data-lang="yaml" class="language-yaml z-code"><code class="language-yaml" data-lang="yaml"><span class="z-source z-yaml"><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">kind</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">Cluster</span> </span><span class="z-source z-yaml"><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">apiVersion</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">kind.x-k8s.io/v1alpha4</span> </span><span class="z-source z-yaml"><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">name</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">plaything</span> </span><span class="z-source z-yaml"><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">containerdConfigPatches</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-keyword z-control z-flow z-block-scalar z-literal z-yaml">|</span><span class="z-storage z-modifier z-chomping-indicator z-yaml">-</span> </span><span class="z-source z-yaml"><span class="z-string z-unquoted z-block z-yaml"> [plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry] </span></span><span class="z-source z-yaml"><span class="z-string z-unquoted z-block z-yaml"> config_path = &quot;/etc/containerd/certs.d&quot; </span></span><span class="z-source z-yaml"><span class="z-string z-unquoted z-block z-yaml"></span><span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">nodes</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">role</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">control-plane</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">extraMounts</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">hostPath</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">/home/ben/code/kind/my-cr.toml</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">containerPath</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">/etc/containerd/certs.d/git.benzo.test/hosts.toml</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">extraPortMappings</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">containerPort</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-constant z-numeric z-integer z-decimal z-yaml">80</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">hostPort</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-constant z-numeric z-integer z-decimal z-yaml">80</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">protocol</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">TCP</span> </span><span class="z-source z-yaml"> <span class="z-punctuation z-definition z-block z-sequence z-item z-yaml">-</span> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">containerPort</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-constant z-numeric z-integer z-decimal z-yaml">443</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">hostPort</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-constant z-numeric z-integer z-decimal z-yaml">443</span> </span><span class="z-source z-yaml"> <span class="z-string z-unquoted z-plain z-out z-yaml"><span class="z-entity z-name z-tag z-yaml">protocol</span></span><span class="z-punctuation z-separator z-key-value z-mapping z-yaml">:</span> <span class="z-string z-unquoted z-plain z-out z-yaml">TCP</span> </span></code></pre> <p>Focusing on the <code>extraMounts</code> section, I'm instructing Kind to take a file from my local filesystem (<code>hostPath</code>) and place it in a specific location on the Kubernetes node (<code>containerPath</code>).</p> <p>With all that done, I spooled up my cluster with <code>just go</code>. Thankfully, I had configured all the deployments as ArgoCD applications, which meant that once ArgoCD was deployed, everything else I had previously configured and deployed just came back to life, without any additional command-line input from myself. The container images hosted in Forgejo pulled without any issues this time around.</p> <h1 id="for-a-real-cluster"><a class="zola-anchor" href="#for-a-real-cluster" aria-label="Anchor link for: for-a-real-cluster">For a <em>Real</em> Cluster</a></h1> <p>I would <strong>not</strong> recommend using the above <code>hosts.toml</code> for a production deployment. Blindly accepting a connection and trusting the SSL certificates can open you up to a number of exploits.</p> <p>It's quite common for enterprises to have their own SSL CA certificate and require it to be used for signing your Ingress SSL certs, instead of those you might get from a public CA like Let's Encrypt.</p> <p>For this scenario, I'd suggest creating:</p> <ul> <li>A Secret <ul> <li>Containing the Client SSL certificate, which needs to be trusted by ContainerD</li> </ul> </li> <li>A ConfigMap containing a shell script to: <ul> <li>Modify the ContainerD <code>config.toml</code> if necessary</li> <li>Copy over the SSL cert to a location on a Kuberneres Node</li> <li>Create the required <code>hosts.toml</code> file (<code>echo 'blah' &gt; /path/to/hosts.toml</code>)</li> </ul> </li> <li>A DaemonSet (Runs on each Node) <ul> <li>With the above Secret and ConfigMap mounted</li> <li>Runs script defined in above ConfigMap</li> <li>Note: You may need to configure the DaemonSet with <code>tolerations</code> if your nodes have <code>taints</code> (<a rel="noopener" target="_blank" href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/">link</a>)</li> </ul> </li> </ul> <p>Finally, I would be using a <code>hosts.toml</code> file that looks a little like this:</p> <pre data-lang="toml" class="language-toml z-code"><code class="language-toml" data-lang="toml"><span class="z-source z-toml"><span class="z-meta z-tag z-key z-toml"><span class="z-entity z-name z-tag z-toml">server</span></span> <span class="z-punctuation z-definition z-key-value z-toml">=</span> <span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>hostname<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span> </span><span class="z-source z-toml"> </span><span class="z-source z-toml"><span class="z-punctuation z-definition z-table z-begin z-toml">[</span><span class="z-meta z-tag z-table z-toml"><span class="z-entity z-name z-table z-toml">host</span><span class="z-punctuation z-separator z-table z-toml">.</span><span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>hostname<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span></span><span class="z-punctuation z-definition z-table z-end z-toml">]</span> </span><span class="z-source z-toml"> <span class="z-meta z-tag z-key z-toml"><span class="z-entity z-name z-tag z-toml">capabilities</span></span> <span class="z-punctuation z-definition z-key-value z-toml">=</span> <span class="z-punctuation z-definition z-array z-begin z-toml">[</span><span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>pull<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span><span class="z-punctuation z-separator z-array z-toml">,</span> <span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>resolve<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span><span class="z-punctuation z-definition z-array z-end z-toml">]</span> </span><span class="z-source z-toml"> <span class="z-meta z-tag z-key z-toml"><span class="z-entity z-name z-tag z-toml">client</span></span> <span class="z-punctuation z-definition z-key-value z-toml">=</span> <span class="z-string z-quoted z-double z-basic z-toml"><span class="z-punctuation z-definition z-string z-begin z-toml">&quot;</span>path/to/cert.pem<span class="z-punctuation z-definition z-string z-end z-toml">&quot;</span></span> </span></code></pre> <p>This file tells ContainerD, <em>"Hey! There's this server...you can pull and resolve container images there...it has an SSL certificate you might not know about but don't worry, here's a pem file you can use for validation."</em></p> <p>The path to the <code>cert.pem</code> file can be relative to the <code>hosts.toml</code> file. I would recommend storing the certificate alongside the <code>hosts.toml</code> file, in the same directory, to keep things visible.</p> <p>A little more reading on the use of this file can be <a rel="noopener" target="_blank" href="https://github.com/containerd/containerd/blob/main/docs/hosts.md">found here</a>.</p> Fri, 21 Feb 2025 00:00:00 +0000 Ben Burbage https://iambenzo.com/posts/wasm-will-be-your-backend/ https://iambenzo.com/posts/wasm-will-be-your-backend/ <p>For those of you who don't know me too well, I've spent a lot of time around Kubernetes clusters. Managing Kubernetes clusters, writing applications for Kubernetes clusters, architecting solutions that centre around Kubernetes, planning my escape to a sun-bathed island with my own Kubernetes cluster in-hand...a fair bit of time.</p> <p>A microservice architecture is typical for systems targeting a Kubernetes deployment - a number of loosely-coupled services which communicate frequently over the network, often via a message or event queue. The big reason for this is that Kubernetes can scale individual components of a system in response to a peak or trough of usage. <em>Why scale an entire application when only parts of it are under stress, right?</em></p> <p>Not so long ago, <a rel="noopener" target="_blank" href="https://www.cncf.io/blog/2024/12/23/navigating-platform-engineering-pitfalls-with-webassembly-components/">an article</a> made it's way into my RSS reader about a CNCF sandbox project called <a rel="noopener" target="_blank" href="https://wasmcloud.com/docs/intro/">wasmCloud</a>. On the surface, wasmCloud looks like an opinionated but robust way to design and deliver a microservice application architecture.</p> <p>It's a Kubernetes-like orchestrator for applications that have been compiled to WASM binaries instead of Containers.</p> <p>Without regurgitating the documentation, there's components for your business logic, capabilities for pluggable functionality (such as messaging via Kafka) and interfaces for defining communication boundaries. As a bonus, you can link WASM runtimes via a "lattice", which is a mesh built upon NATS.</p> <p>wasmCloud looks like it may be a Kubernetes replacement, running WASM applications instead of Containers. Whilst very similar, in that it provides a level of application orchestration, it can harness a Kubernetes cluster to run WASM applications as well as Containers. You get the benefits of Kubernetes' hardware management <em>and</em> the features of wasmCloud.</p> <p>wasmCloud doesn't <em>need</em> a Kubernetes cluster to run though. It simply requires a WASM runtime to be installed on a host machine. This WASM runtime can be thought of as a JVM for Java applications, or a .NET runtime for dotnet applications. Thanks to this, the small binary size of WASM applications, and the NATS-based lattice, wasmCloud could be useful in edge computing scenarios - somewhere a Kubernetes deployment doesn't particularly shine.</p> <h1 id="containers-vs-wasm"><a class="zola-anchor" href="#containers-vs-wasm" aria-label="Anchor link for: containers-vs-wasm">Containers vs WASM</a></h1> <p><em>Why would you use WASM instead of Containers?</em> - Well, let's look at the similarities first:</p> <ul> <li>Both are good options for polyglot dev teams, though WASM is a little more restrictive in programming language options.</li> <li>Both can be stored in an OCI compatible registry for distribution.</li> <li>Both enable applications to be run repeatably across multiple hosts. <ul> <li>Containers aren't portable across CPU architectures though.</li> </ul> </li> </ul> <p>Now, some differences:</p> <ul> <li>Sandboxing <ul> <li>Containers run a stripped down OS.</li> <li>WASM is purely application logic.</li> </ul> </li> <li>Startup Times <ul> <li>Containers take a few seconds to start running.</li> <li>WASM applications can be running in under a second.</li> </ul> </li> <li>Multi-threading <ul> <li>Containers support mutli-threaded applications.</li> <li>WASM applications are single-threaded but support asynchronous applications.</li> </ul> </li> <li>Attack Surface <ul> <li>For Containers, this depends on the Container OS and the host OS configuration.</li> <li>WASM applications run in a sandbox, with protected memory.</li> </ul> </li> </ul> <p>The above are just some highlights which I find interesting, <a rel="noopener" target="_blank" href="https://wasmedge.org/wasm_linux_container/">there's more</a> to take a look at.</p> <h1 id="digging-in"><a class="zola-anchor" href="#digging-in" aria-label="Anchor link for: digging-in">Digging In</a></h1> <p>After doing a some digging, I found an <a rel="noopener" target="_blank" href="https://github.com/wasmCloud/wasmCloud/tree/main/examples/rust/composition">example</a> of a Rust project for using wasmCloud. It looks a little convoluted.</p> <p>There seems to be a swath of tooling required, that isn't so visible in the project's documentation right now. This can be remediated quickly through the bolstering of the project's technical documentation, and through providing some tutorials that go below skin deep.</p> <p>I think there's an opportunity for refinement to improve the developer experience here. Taking a look at some programming language tooling, like the Cargo ecosystem may prove useful. The wasmCloud CLI could include the features of the WebAssembly Compositions CLI and the WASI Virt CLI, perhaps via a Cargo-like plugin system.</p> <p>Looking a little further into the codebase, I notice that there's a lot of <code>toml</code>, <code>yaml</code> and <code>wac</code> configuration files required for gluing everything together.</p> <p>This takes me back to the days of writing Oracle SOA applications using SOAP, where you needed WSDL and XSD definitions for your components to work together - it wasn't an experience that I particularly enjoyed as a developer but I do understand that, whilst verbose, it's a way of defining and maintaining an explicit contract of communication.</p> <p>For the various configuration files that are required, is there a generator available? One that may enable the generation of these files at compile time? My thinking for this would be around providing small language-specific libraries where a developer could implement an interface, or trait. Then the composition part could live in whatever the language's equivalent of <code>Cargo.toml</code> is.</p> <p>Maybe a stupid question/idea but there must be a way of consolidating them to reduce the number of steps required to get components communicating with each other.</p> <h1 id="looking-forwards"><a class="zola-anchor" href="#looking-forwards" aria-label="Anchor link for: looking-forwards">Looking Forwards</a></h1> <p>Whilst I feel there is room for improvement when it comes to the developer experience with wasmCloud, I do think that WASM + WASI has to be considered seriously for server-side workloads.</p> <p>If your application doesn't require multiple CPU threads, there's too many benefits to choosing WASM over a Container: Smaller OCI artifacts <a href="https://iambenzo.com/posts/sustainability-starts-at-the-codebase/">have a large impact</a> on the sustainability of a solution, faster start-up times increase developer velocity and decrease scale-out times, application sandboxing reduces the surface of attack available to bad actors...</p> <p>I don't think that WASM will replace Containers entirely. But I believe we'll certainly see a huge move towards WASM for a lot of use-cases server-side.</p> <p>I'm certainly going to be keeping an eye on the wasmCloud, WASM and WASI projects.</p> Thu, 28 Nov 2024 21:00:03 +0000 Ben Burbage https://iambenzo.com/posts/ktr/ https://iambenzo.com/posts/ktr/ <h2 id="inspiration"><a class="zola-anchor" href="#inspiration" aria-label="Anchor link for: inspiration">Inspiration</a></h2> <p>Recently, I started reading the enlightening <a rel="noopener" target="_blank" href="https://amzn.to/3VJoekN">book by Bob Doto</a> on adopting the Zettelkasten methodology for capturing ideas and thinking creatively.</p> <p>In the opening chapters, Bob describes the different types of notes you could take, one of them being reference notes.</p> <blockquote> <p>Reference notes are single, long notes of brief citations or "references" to what caught your attention whilst reading a book, or consuming other forms of media.</p> </blockquote> <p>I do the majority of my reading on my beloved Kindle and, after reading the section of Bob's book on reference notes, I wondered how I could optimally capture passages of interest from my Kindle and turn them into something tangible to revisit and store in my Zettelkasten.</p> <!-- The idea of switching focus from my book to a notepad or phone app to capture my thoughts presents me with an opportunity to lose focus through task switching. --> <p>I do appreciate that a notepad is sometimes best, as there's a little cognitive magic that happens when we humans put pen to paper - but I'm using an electronic device for reading and I'm lazy enough to want to take advantage of that through some form of automation.</p> <p>I shelved (virtually) Bob's book to ponder.</p> <h2 id="stepping-through"><a class="zola-anchor" href="#stepping-through" aria-label="Anchor link for: stepping-through">Stepping Through</a></h2> <p>The Kindle has a feature that allows you to highlight passages of text as you're reading. You can also add notes to these highlights whilst using the device.</p> <p>These highlights and notes are stored on the Kindle in a plain text file found at: <code>&lt;kindle_root&gt;/documents/My Clippings.txt</code>. This file can be accessed by plugging the Kindle into your computer via USB.</p> <p>We have a data source.</p> <p>I've opted to maintain a digital Zettelkasten under a fresh "vault" in <a rel="noopener" target="_blank" href="https://obsidian.md">Obsidian</a> - a program I've been using for quite some time for note taking, and my own personal [chaotic] brand of journalling.</p> <p>Obsidian stores every note as a markdown file on your computer's filesystem.</p> <p>My thinking here was to find a way to parse the clippings file found on the Kindle and process it into a markdown file, which I can save to the part of my filesystem that is used by Obsidian.</p> <p>Ideally the contents of the output would be uniform, conforming to some template that makes sense to me when revisiting the highlights and notes. Bob illustrates a potential format in his book, I decided to try and emulate that.</p> <h2 id="enter-kindle-to-references-ktr"><a class="zola-anchor" href="#enter-kindle-to-references-ktr" aria-label="Anchor link for: enter-kindle-to-references-ktr">Enter Kindle to References (KTR)</a></h2> <p>I've created a small Command Line Interface (CLI) application, "Kindle to References" or KTR, to address my requirements.</p> <p>I chose to write it in Rust because it's a language I've been yearning to build something in for a little while, since going through <a rel="noopener" target="_blank" href="https://rust-exercises.com/100-exercises/">100 Exercises in Rust</a>.</p> <p>I've had a wonderful experience with Rust, though you could argue that what I've built has almost no complexity to it and would be a simple creation in any language.</p> <p>As a user of the CLI application, you supply the path to your clippings file and a path to an output directory. Hitting enter will result in a set of markdown files, one for each book you have highlights and/or notes for, in your output directory.</p> <p>These files can then be read by/imported into Obsidian:</p> <p><img src="https://iambenzo.com/posts/ktr/assets/obsidian_output.png" alt="Preview of output file format" /></p> <p>The above image shows an example of the output format in Obsidian. KTR's default template will create a table listing the page number that a highlight was captured in.</p> <p>The description column is where any related notes made on the Kindle will appear, or provide a space to add thoughts after reading a book. This column will always contain a footnote reference link to the highlighted content.</p> <p>The final column will always be empty, no matter what you do on your Kindle - it's a space intended for tagging and/or linking thoughts to others in your Zettelkasten.</p> <p>At the bottom of the page will be a numbered list of the highlighted content. These are the footnotes that are linked to in the table - Obsidian has a nice feature where you can hover your mouse over an internal link and a small window showing the content will pop up. I've tried to demonstrate that in the screenshot above - the mouse is missing, so you'll have to take my word that I'm hovering it over the <code>[1]</code> link in the first row of the table.</p> <p>Whilst the application captures any notes that were added to highlights, I generally don't take notes on my Kindle when highlighting text. This output file grants me an opportunity to revisit passages of text I found notable, add thoughts to them and also take advantage of Obsidian's tagging functionality for linking references, thoughts and other notes together.</p> <p>I feel that forcing myself to review the content I've highlighted might help embed my thoughts further - there's no science behind that thinking though.</p> <p>As a treat, you can optionally supply a path to a template of your own to alter the structure of the output files to your liking. I chose to use <a rel="noopener" target="_blank" href="https://github.com/Keats/tera">Tera</a> as the template engine because <a rel="noopener" target="_blank" href="https://hachyderm.io/@LGUG2Z">@LGUG2Z</a> was raving about it to me on Mastodon.</p> <h3 id="going-a-little-further"><a class="zola-anchor" href="#going-a-little-further" aria-label="Anchor link for: going-a-little-further">Going a Little Further</a></h3> <p>I decided that the CLI wasn't quite enough of a project and so built a Graphical User Interface (GUI) application too. It's essentially a wizard style utility - <em>next, next, next...</em></p> <p>The <a rel="noopener" target="_blank" href="https://guide.elm-lang.org/architecture/">ELM Architecture</a> was only something I had read about. Building this small application let me dip my toes into the heated pool of State, Messages, Update logic and View logic.</p> <p>My application isn't going to win any design or UX awards but it was fun to create. I got to dig into the (<a rel="noopener" target="_blank" href="https://iced.rs/">Iced</a>) library's documentation and source code to understand how things hang together, giving me experience of reading through a much more complicated Rust project. Again, I had a wonderful time.</p> <p>The GUI application has an additional feature over the CLI - you can choose which books to process, rather than processing them all each time.</p> <p><img src="https://iambenzo.com/posts/ktr/assets/ktr_gui_book_selection.png" alt="KTR GUI Book Selection screenshot" /></p> <p>Whether using the CLI, or the GUI - the end result is the same: A collection of one or more markdown files in an output directory - ready for use in Obsidian, or your favourite Markdown editor.</p> <h2 id="download-it-if-you-dare"><a class="zola-anchor" href="#download-it-if-you-dare" aria-label="Anchor link for: download-it-if-you-dare">Download It, If You Dare</a></h2> <p>Obviously, I've built this tool for myself. I have chosen to make the code available <a rel="noopener" target="_blank" href="https://github.com/iambenzo/ktr">over on my GitHub</a> though - just in case anyone else finds it useful too.</p> <p>In the code repository, there are <a rel="noopener" target="_blank" href="https://github.com/iambenzo/ktr/releases/latest"><em>unsigned</em> binaries available</a> for download. Your computer will likely complain about how untrustworthy I am as a developer, but you're in control of your destiny, not your fancy calculator.</p> <p>Anyway, now this has been created, I can go back to enjoying <a rel="noopener" target="_blank" href="https://amzn.to/3VJoekN">Bob's book</a>.</p> Sun, 04 Aug 2024 15:50:43 +0100 Ben Burbage https://iambenzo.com/posts/sustainability-starts-at-the-codebase/ https://iambenzo.com/posts/sustainability-starts-at-the-codebase/ <h2 id="being-green"><a class="zola-anchor" href="#being-green" aria-label="Anchor link for: being-green">Being Green</a></h2> <p>As Solution Architects in the modern world, we are charged with the responsibility of ensuring our solutions are sustainable, green, or environmentally friendly.</p> <p>Doing so means that the data centres we use place less demand on the world's precious resources for manufacturing the hardware that our solutions run on.</p> <p>A typical area of focus is the use of "green" data centres. These green data centres often use energy-efficient cooling solutions, make use of renewable energy sources and may well have a hardware recycling/re-purposing scheme.</p> <p>It's not too difficult to achieve the use of green data centres in the cloud, the big three cloud providers all have "green" credentials <sup class="footnote-reference" id="fr-1-1"><a href="#fn-1">[1]</a></sup> <sup class="footnote-reference" id="fr-2-1"><a href="#fn-2">[2]</a></sup> <sup class="footnote-reference" id="fr-3-1"><a href="#fn-3">[3]</a></sup>.</p> <p>Another area of focus is the efficient utilisation of physical resource through lifecycle management (assessment, maintenance, upgrades), resource optimisation (deploying infrastructure that meets workload requirements, rather than exceeds them) and virtualisation.</p> <p>This second area of focus is where I'd like to spend some time with you today.</p> <p>There's a plethora of tools which promise to help you optimise your infrastructure use. These tools usually work by spending some time monitoring the CPU and Memory usage of your applications and infrastructure utilisation before reporting back where you can reduce the compute.</p> <p>Actioning the reported information results in a more energy and computationally efficient system (i.e. through a reduction of wasted CPU cycles) and, if you're in the cloud, a more cost efficient system (i.e. through paying for less CPUs and Memory). <em>Rock and roll! Job done....right?</em></p> <p>Well, if you've just completed a cloud migration with a load of monolithic applications, this might be the only thing you can do for now. But there's more that <em>can</em> be done.</p> <h2 id="a-scenario"><a class="zola-anchor" href="#a-scenario" aria-label="Anchor link for: a-scenario">A Scenario</a></h2> <p>Let's form a scenario where you have an existing monolithic application, written in Java, running in a Virtual Machine (VM) and are looking to break it up into smaller, containerised applications and deploy them to a Kubernetes cluster.</p> <p>You and your team have already gone through defining your application boundaries and followed the domain driven design principals to the letter...<em>there's a plan.</em></p> <p>Naturally, as Java aficionados, you've selected Spring Boot as your framework of choice for the back-end API work. Hold the phone for just a moment - there's an opportunity here to revisit the programming language used to craft your application's new roots.</p> <blockquote> <p>I'm going to be picking on Java a little here, as I've seen it used extensively in an enterprise application developer environment - and it allows me to draw on my experiences.</p> </blockquote> <p><em>"Why should I reconsider the programming language used?"</em> I'm glad you asked. Let's revisit the [abridged] Software Development Life Cycle (SDLC) a little...</p> <ul> <li>Developer writes code <ul> <li>Maybe pulling in additional third party dependencies, which may well be acquired via an internal artifact repository</li> </ul> </li> <li>Developer pushes code to a remote code repository and creates a Pull Request (PR)</li> <li>Continuous Integration (CI) pipeline kicks in <ul> <li>Builds application <ul> <li>Pulls third-party dependencies in the process</li> </ul> </li> <li>Tests application</li> <li>Pushes application build output to an artifact repository <ul> <li>Vulnerability scans run here</li> </ul> </li> <li>Containerises application</li> <li>Pushes application container to a container registry, ready for deployment <ul> <li>Vulnerability scans also run here</li> </ul> </li> </ul> </li> </ul> <p>Assuming all stages of the CI job passed and completed successfully, the developer's PR can be merged and the updated application is then ready for deployment.</p> <p>Let’s break down the infrastructure used to complete this single cycle:</p> <table><thead><tr><th>Step</th><th>Infrastructure</th><th>Notes</th></tr></thead><tbody> <tr><td>Developer writes code</td><td>Storage, CPU, Memory</td><td>For pulling dependencies from the self-hosted artifact repository</td></tr> <tr><td>Developer pushes code</td><td>Storage, CPU, Memory</td><td>Only if using a self-hosted source code repository</td></tr> <tr><td>CI pipeline kicks in</td><td>CPU, Memory</td><td>Assuming self-hosted pipeline runners - billed in time if not</td></tr> <tr><td>Pushing to artifact repository</td><td>Storage</td><td></td></tr> <tr><td>Pushing to container registry</td><td>Storage</td><td></td></tr> </tbody></table> <p>You can see that I’ve listed storage as key infrastructure here. Whilst it isn’t as expensive to purchase as computational resource, efficient use of storage means that less storage drives need to be provisioned in the data centre.</p> <p>Another layer here is network. The source code, all of the application dependencies, the final application binary and the containerised application get transmitted over the network.</p> <p>The size of data being transmitted at each stage of the CI process has bandwidth demands, which drives how much network infrastructure is required and used within the data centre - and how much bandwidth is available for other processes running simultaneously on the same network.</p> <p>With our scenario in mind, the size of the application containers also impacts how quickly they can be retrieved from a container registry by a Kubernetes cluster at deployment time. This will have repercussions on the first startup time of your application.</p> <p>Hopefully, you can see how optimising the storage space your applications demand reverberates through your infrastructure.</p> <h2 id="in-my-experience"><a class="zola-anchor" href="#in-my-experience" aria-label="Anchor link for: in-my-experience">In My Experience...</a></h2> <p>Back to the question: <em>"Why should I reconsider the programming language used?"</em></p> <p>At a previous job, I was generously granted some time to rewrite one of our Java REST APIs in <a rel="noopener" target="_blank" href="https://go.dev/">GoLang</a>. The driver for this activity was to compare the performance characteristics of the two languages - and to appease my nagging. We'll focus on the impact the experiment had on the infrastructure here though:</p> <table><thead><tr><th>Language</th><th>Number of Dependencies</th><th>Binary Size</th><th>Container Size</th></tr></thead><tbody> <tr><td>Java (Spring Boot)</td><td>142</td><td>43MB</td><td>130MB <sup class="footnote-reference" id="fr-4-1"><a href="#fn-4">[4]</a></sup></td></tr> <tr><td>GoLang</td><td>3</td><td>8.3MB</td><td><a href="/posts/tiny-golang-containers">9.72MB</a></td></tr> </tbody></table> <p>Take a moment to consider how dramatic the difference in infrastructure usage these numbers imply.</p> <p>Firstly, there are fewer third-party dependencies in the GoLang version because the application could be built almost completely using GoLang's standard library - only reaching out to a third party for a database driver. This leads to a reduction in caching requirements for the artifact repository and also reduces the surface area for vulnerabilities in the codebase.</p> <p>The GoLang binary size is over four times smaller than the Java equivalent. Again, this reduces the utilisation of the artifact repository and demands less bandwidth from the network in the transferring of the application binary.</p> <p>Finally, the GoLang application container is dramatically smaller than the Java application container. This reduces the storage capacity requirements for the container registry and reduces deployment time. Again, there's less demand on the network in transferring the containerised application to and from the container registry.</p> <p>Before this post becomes an advertisement for GoLang, I will quickly report that:</p> <ul> <li>Application binary and container had zero vulnerabilities</li> <li>API response times reduced from multiple seconds to microseconds</li> <li>Each instance of the application could handle significantly more customer requests before having to scale out horizontally</li> <li>Scaling out was much faster because application start-up was near-instant (apposed to 20-30 seconds for Spring Boot)</li> <li>The application used a fraction of the CPU and Memory at runtime</li> </ul> <p>The two versions of the application provided the exact same functionality - they were simply written in two different programming languages.</p> <p>Whilst I've largely focussed on the impact of storage and network usage so far, it's important to note that the GoLang application also used less computational resource - CPU and Memory. This means that you can fit more applications into the same size of compute infrastructure, or look to reduce the size of that compute infrastructure.</p> <p>I bet you can guess which version of the application made it through to production...and what language we adopted for new work.</p> <p>Imagine how different your infrastructure landscape would look after taking some time to evaluate the choice of programming language used in your applications. Maybe the components would be the same but they'd certainly be smaller in size. You'd be reducing the stress on your data centre, doing your bit for the planet <em>and</em> saving some money - all at the same time!</p> <h2 id="going-forwards"><a class="zola-anchor" href="#going-forwards" aria-label="Anchor link for: going-forwards">Going Forwards</a></h2> <p>Choosing a programming language is significantly easier to do when in a greenfield environment - where you're creating new things from scratch.</p> <p>For brownfield environments, where you have existing applications, you could do something similar to me - take a small application and rewrite it as an experiment. Alternatively, the next time a new piece of development work presents itself, use that as an opportunity to assess how the application should be crafted.</p> <p>Whilst a new language can involve a learning process for your development teams, it's also a growth opportunity for them. A chance to expand their skillset and look at their work from a fresh perspective.</p> <p>Matthias Endler and Jeremy Soller discuss the introduction of a new programming language in depth on <a rel="noopener" target="_blank" href="https://corrode.dev/podcast/s02e07-system76/">this episode</a> (about 1 hour in) of the wonderful "Rust In Production" Podcast - it's clearly focused on introducing the <a rel="noopener" target="_blank" href="https://www.rust-lang.org/">Rust</a> programming language into an organisation but the concepts apply to any language you may be considering.</p> <p>Have the conversation with your development team. Whilst a different programming language isn't always viable, there may be options with your current programming language. With Java, you can reduce your application container size with <a rel="noopener" target="_blank" href="https://dev.java/learn/jvm/tools/core/jlink/">jlink</a>, or making use of <a rel="noopener" target="_blank" href="https://www.graalvm.org/">GraalVM</a> native images.</p> <h2 id="to-conclude"><a class="zola-anchor" href="#to-conclude" aria-label="Anchor link for: to-conclude">To Conclude</a></h2> <p>Carefully considering the codebase has a rippling effect throughout your solution and can lead to more efficient use of your infrastructure. Efficient use of your infrastructure leads to a reduction in hardware needed in the data centres you use, and places less demand on the world's precious resources which are used to manufacture that hardware.</p> <hr><ol class="footnotes-list"> <li id="fn-1"> <p><a rel="noopener" target="_blank" href="https://azure.microsoft.com/en-gb/explore/global-infrastructure/sustainability/">Azure Sustainability</a> <a href="#fr-1-1">↩</a></p> </li> <li id="fn-2"> <p><a rel="noopener" target="_blank" href="https://aws.amazon.com/sustainability/">Amazon Web Services Sustainability</a> <a href="#fr-2-1">↩</a></p> </li> <li id="fn-3"> <p><a rel="noopener" target="_blank" href="https://cloud.google.com/sustainability/">Google Cloud Sustainability</a> <a href="#fr-3-1">↩</a></p> </li> <li id="fn-4"> <p>The Java container requires the Java Runtime Environment for the application to run. We managed to reduce the size of the container image by over 100Mb through arduous use of <a rel="noopener" target="_blank" href="https://dev.java/learn/jvm/tools/core/jlink/">jlink</a>. <a href="#fr-4-1">↩</a></p> </li> </ol> Mon, 29 Apr 2024 09:36:26 +0100 Ben Burbage https://iambenzo.com/posts/cli-just/ https://iambenzo.com/posts/cli-just/ <p>I'm getting older. My memory and my wrists aren't as nimble as they once were.</p> <p>When I do get a sliver of time to work on a personal project, it's nice to have a couple of project specific one-liners that I can run in my terminal to save me from the inevitable chore of re-reading the documentation of whatever build tool or process I want to use.</p> <p><a rel="noopener" target="_blank" href="https://just.systems/">Just</a> is a tool that supplements my aging brain wonderfully. It's essentially a command runner in the guise of a Makefile.</p> <p>All I have to do is craft a small <code>.justfile</code> in the root directory of a project, whilst the commands are all fresh in my mind. When I finally return to my project, I smash <code>just -l</code> into my terminal and I'll be welcomed by a list of actions I can execute, based on the project I'm in.</p> <p>Take this blog, for example. Do I remember the <code>hugo new content…</code> command each year I pressure myself into writing a new blog post? <em>Hell no!</em></p> <p>I have a <code>.justfile</code> for my blog, and after checking my usual actions with <code>just</code> (no <code>-l</code> needed for me, you'll see why), I'm reminded that I can run <code>just post</code> followed by a post title to create a new blog post and open it in a text editor.</p> <p>Here's my blog's <code>.justfile</code> in all its glory:</p> <pre data-lang="make" class="language-make z-code"><code class="language-make" data-lang="make"><span class="z-source z-makefile"><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> `just` override </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">_default</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"></span><span class="z-meta z-function z-body z-makefile"></span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">just</span></span><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>l</span></span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> Serve the site on localhost </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">serve</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"></span><span class="z-meta z-function z-body z-makefile"></span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">hugo</span></span><span class="z-meta z-function-call z-arguments z-shell"> serve<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>D</span></span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> Create a new blog post (titles-like-this) </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">post title</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"></span><span class="z-meta z-function z-body z-makefile"></span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">hugo</span></span><span class="z-meta z-function-call z-arguments z-shell"> new content posts/<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>title<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>.md</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">mkdir</span></span><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>p</span> ./content/posts/<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>title<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>/assets</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">mv</span></span><span class="z-meta z-function-call z-arguments z-shell"> ./content/posts/<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>title<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>.md ./content/posts/<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>title<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>/index.md</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">hx</span></span><span class="z-meta z-function-call z-arguments z-shell"> ./content/posts/<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>title<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>/index.md</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> Start a tmux session </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">tmux</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"></span><span class="z-meta z-function z-body z-makefile"></span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">tmux</span></span><span class="z-meta z-function-call z-arguments z-shell"> new<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>Pd</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>s</span> blog<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>n</span> code</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">tmux</span></span><span class="z-meta z-function-call z-arguments z-shell"> neww<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>Pd</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> blog<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>F</span> 2<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>n</span> serve</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">tmux</span></span><span class="z-meta z-function-call z-arguments z-shell"> send-keys<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> blog:2 <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>just serve<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> Enter</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">firefox</span></span><span class="z-meta z-function-call z-arguments z-shell"> http://localhost:1313/</span> <span class="z-keyword z-operator z-logical z-job z-shell">&amp;</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">tmux</span></span><span class="z-meta z-function-call z-arguments z-shell"> attach<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>d</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> blog</span></span> </span></span></code></pre> <p>Here, there are four recipes, or actions as I sometimes call them. The <code>_default</code> recipe means that I can use the <code>just</code> command without the <code>-l</code> to list my available actions. The <code>serve</code> recipe simply gets my website running locally.</p> <p>The <code>post</code> recipe creates a new blog entry with a title that I provide at invocation (<code>just post super-sexy-catchy-post-title</code>), wrangles that post into the directory structure that I like and then opens that new post in my text editor of the day.</p> <p>Finally, the <code>tmux</code> recipe spawns a new tmux session with two windows - one for activities and one for serving my website locally. It also cracks open a browser window, loading the locally hosted instance of my site. This is useful for when I might decide to work on my theme or site functionality.</p> <p>Using this tool allows me to start being productive quickly, with less typing - saving my poor wrists from extraneous fatigue.</p> <p>Here's an example of a <code>.justfile</code> that I might use for in a directory that contains a number of dockerfiles:</p> <pre data-lang="make" class="language-make z-code"><code class="language-make" data-lang="make"><span class="z-source z-makefile"><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> Variables for defaults </span></span><span class="z-source z-makefile"><span class="z-variable z-other z-makefile">defaultFile</span> <span class="z-keyword z-operator z-assignment z-makefile">:=</span></span> <span class="z-source z-makefile"><span class="z-meta z-string z-makefile"><span class="z-string z-unquoted z-makefile">&#39;bestOne&#39;</span></span> </span><span class="z-source z-makefile"> </span><span class="z-source z-makefile"><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> `just` override </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">_default</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"></span><span class="z-meta z-function z-body z-makefile"></span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">just</span></span><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>l</span></span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> Build a dockerfile </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">build file=defaultFile</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"></span><span class="z-meta z-function z-body z-makefile"></span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> build<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>f</span> ./<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>file<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>.dockerfile<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> <span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>file<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>:taaaag .</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> Build and run a dockerfile </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">run file=defaultFile</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"><span class="z-string z-unquoted z-makefile">(build file)</span></span><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> run<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>it</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>rm</span> <span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>file<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>:taaaag</span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> (Hidden) Build and Tag container image </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">_tag version file=defaultFile</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"><span class="z-string z-unquoted z-makefile">(build file)</span></span><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> tag <span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>file<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>:taaaag container-registry.com/<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>file<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>:<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>version<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span></span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-comment z-line z-number-sign z-makefile"><span class="z-punctuation z-definition z-comment z-makefile">#</span> Build, Tag and Push a container image </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-makefile"><span class="z-entity z-name z-function z-makefile">publish version file=defaultFile</span></span><span class="z-keyword z-operator z-assignment z-makefile">:</span> <span class="z-meta z-function z-arguments z-makefile"><span class="z-string z-unquoted z-makefile">(_tag version file)</span></span><span class="z-meta z-function z-body z-makefile"> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"></span><span class="z-meta z-function z-body z-makefile"> <span class="z-source z-shell"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> push container-registry.com/<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>file<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span>:<span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span>version<span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span></span></span> </span></span><span class="z-source z-makefile"><span class="z-meta z-function z-body z-makefile"> </span></span></code></pre> <p>This one's a little more complex but it shows how you can chain recipes together. For example, the <code>run</code> recipe will first run the <code>build</code> recipe.</p> <p>I won't go into much more detail here - hopefully you see how I find this utility useful. If you're interested in checking <a rel="noopener" target="_blank" href="https://github.com/casey/just">Just</a> out, it's <a rel="noopener" target="_blank" href="https://just.systems/man/en/">documentation</a> is pretty thorough.</p> Mon, 22 Apr 2024 13:47:09 +0100 Ben Burbage https://iambenzo.com/posts/my-favourite-card-game/ https://iambenzo.com/posts/my-favourite-card-game/ <p>I used to travel a lot for work, and got into the habit of carrying around a pack of playing cards - for downtime at a bar, or when stuck waiting out yet another train cancellation. The playing cards have seen plenty of daylight at home and during family gatherings too!</p> <p>The card game I’m about to tell you about has to be my favourite. I won’t disclose what I call the game - it’s a naughty word - but I will share how to play the game because I’ve had so much fun with it over the years.</p> Fri, 14 Jan 2022 16:52:40 +0000 Ben Burbage https://iambenzo.com/posts/seven-minute-rule/ https://iambenzo.com/posts/seven-minute-rule/ <p>I value my time.</p> <p>Whilst working, delivering is the top priority for me and anything that detracts from that could be a hindrance.</p> <p>I work hard to enusure that I'm not in meetings where I won't be in a position to add value, or where the meeting is of no value to me.</p> Thu, 18 Nov 2021 10:54:24 +0000 Ben Burbage https://iambenzo.com/posts/tiny-golang-containers/ https://iambenzo.com/posts/tiny-golang-containers/ <p>Over this year, I've slowly become more Golang oriented as a developer and I'm really enjoying the journey that choice is taking me on.</p> <p>Traditionally, I've built web services with Spring Boot and Java but more recently, I've noticed there are a number of benefits to developing such services with Golang and it's standard library: Lower runtime resource requirements and no runtime environment required to name a few.</p> <p>Another benefit I'd like to discuss here is achieving <em>tiny</em> containerized applications with Golang.</p> <h2 id="why"><a class="zola-anchor" href="#why" aria-label="Anchor link for: why">Why?</a></h2> <p>The main advantages of creating container images which are small in size are that they:</p> <ul> <li>Require less storage space in a container registry</li> <li>Require less bandwidth for upload and download</li> <li>Require less available disk in runtime environment (e.g Kubernetes)</li> </ul> <p>The above points typically keep cloud subscriptions costs down and, in extension, your product team happy 🙂</p> <p>Another key advantage to stripping down container images to their absolute bare necessities is that it reduces that container's surface area for exploitation - meaning safer deployments too.</p> <h2 id="how"><a class="zola-anchor" href="#how" aria-label="Anchor link for: how">How?</a></h2> <h3 id="sample-application"><a class="zola-anchor" href="#sample-application" aria-label="Anchor link for: sample-application">Sample Application</a></h3> <p>To show how to create slender containers with Golang, we need a sample application to containerize, so I created one: <a rel="noopener" target="_blank" href="https://github.com/iambenzo/dirtyhttp-example">https://github.com/iambenzo/dirtyhttp-example</a></p> <p>It's a simple web service, which exposes an in-memory data structure for the storing and retrieval of user information via HTTP request. (Only around 200 lines of code, including spacing and comments if you want to check it out)</p> <p>Usage instructions are included in the README file.</p> <h3 id="full-fat-container"><a class="zola-anchor" href="#full-fat-container" aria-label="Anchor link for: full-fat-container">Full Fat Container</a></h3> <p>In the same way some of us struggle to digest full fat milk, our container registries would eventually become bloated with full fat containers.</p> <p>Let's see how much damage a creamy Golang container with all the bells and whistles does:</p> <pre data-lang="dockerfile" class="language-dockerfile z-code"><code class="language-dockerfile" data-lang="dockerfile"><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Use Alpine for its small footprint </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">FROM</span> golang:<span class="z-entity z-name z-enum z-tag-digest">alpine</span> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">WORKDIR </span>/app </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Download our dependencies </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> go.mod ./ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> go.sum ./ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">RUN </span>go mod download </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Build the application binary </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> *.go ./ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">RUN </span>go build -o /dirtyhttp-example </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">EXPOSE </span>8080 </span><span class="z-source z-dockerfile"><span class="z-keyword z-operator z-dockerfile">CMD </span>[ <span class="z-string z-quoted z-double z-dockerfile"><span class="z-punctuation z-definition z-string z-begin z-dockerfile">&quot;</span>/dirtyhttp-example<span class="z-punctuation z-definition z-string z-end z-dockerfile">&quot;</span></span> ] </span></code></pre> <p>The command to build the container locally is:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> build<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> go-example:fat<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>f</span> Dockerfile.fat .</span> </span></code></pre> <p>Once docker has finished working it's magic, we'll have a container that looks like this:</p> <p><img src="https://iambenzo.com/posts/tiny-golang-containers/img/full-fat.png" alt="Full Fat" /></p> <p>You can see above that our "fat" container image requires a meaty 398MB of storage space.</p> <p>To run the container image, you'd use the following command:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> run<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>it</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>rm</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>p</span> 8080:8080 go-example:fat</span> </span></code></pre> <h3 id="going-semi-skimmed"><a class="zola-anchor" href="#going-semi-skimmed" aria-label="Anchor link for: going-semi-skimmed">Going Semi-Skimmed</a></h3> <p>The wonderful developers at Google have made a "runtime" container available to eliminate some of the bloat in our previous container.</p> <p>To use it properly, we need to make use of a multi-stage build. This is where we use one [intermediary] docker image to build our application and send the output binary to a slimmed down image where the application will actually run.</p> <pre data-lang="dockerfile" class="language-dockerfile z-code"><code class="language-dockerfile" data-lang="dockerfile"><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Build </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">FROM</span> golang:<span class="z-entity z-name z-enum z-tag-digest">1.16-buster</span> <span class="z-keyword z-control z-dockerfile">AS</span> <span class="z-variable z-stage-name">build</span> </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">WORKDIR </span>/app </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> go.mod ./ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> go.sum ./ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">RUN </span>go mod download </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> *.go ./ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">RUN </span>go build -o /dirtyhttp-example </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Runtime </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">FROM</span> gcr.io/distroless/base-debian10 </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">WORKDIR </span>/ </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> --from=<span class="z-variable z-stage-name">build</span> /dirtyhttp-example /dirtyhttp-example </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">EXPOSE </span>8080 </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">USER </span>nonroot:nonroot </span><span class="z-source z-dockerfile"><span class="z-keyword z-operator z-dockerfile">ENTRYPOINT </span>[<span class="z-string z-quoted z-double z-dockerfile"><span class="z-punctuation z-definition z-string z-begin z-dockerfile">&quot;</span>/dirtyhttp-example<span class="z-punctuation z-definition z-string z-end z-dockerfile">&quot;</span></span>] </span></code></pre> <p>The output container image in the above scenario is the second [runtime] one, based on <a rel="noopener" target="_blank" href="http://gcr.io/distroless/base-debian10">gcr.io/distroless/base-debian10</a> , where the only addition to that image is the application binary which was built in a full-fat Golang image. Let's build it:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> build<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> go-example:semi<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>f</span> Dockerfile.semi .</span> </span></code></pre> <p><img src="https://iambenzo.com/posts/tiny-golang-containers/img/semi-skimmed.png" alt="Semi-Skimmed" /></p> <p>The "semi" version of our container image is already significantly smaller - 26.6MB! This is just under 15 times smaller than our previous "fat" version.</p> <p>To run the container image, you'd use the following command:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> run<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>it</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>rm</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>p</span> 8080:8080 go-example:semi</span> </span></code></pre> <h3 id="the-magic-soya-bean"><a class="zola-anchor" href="#the-magic-soya-bean" aria-label="Anchor link for: the-magic-soya-bean">The Magic Soya Bean</a></h3> <p>We're at a good size now that we're utilising Google's stripped-out runtime base image - but we can go even further.</p> <p>Because we know our codebase is pure Golang code and doesn't depend on any C code, we can build a statically linked binary in our stage-one (build) image and use the smallest base image available - scratch - as the base image our stage-two (runtime) image:</p> <pre data-lang="dockerfile" class="language-dockerfile z-code"><code class="language-dockerfile" data-lang="dockerfile"><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Builder image </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">FROM</span> golang:<span class="z-entity z-name z-enum z-tag-digest">alpine</span> <span class="z-keyword z-control z-dockerfile">AS</span> <span class="z-variable z-stage-name">build</span> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">RUN </span>apk update &amp;&amp; apk add --no-cache ca-certificates &amp;&amp; update-ca-certificates </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">WORKDIR </span>/app </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Download our dependencies </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> go.mod ./ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> go.sum ./ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">RUN </span>go mod download </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> *.go ./ </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Build a statically linked binary </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">RUN </span>CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o /dirtyhttp-example </span><span class="z-source z-dockerfile"> </span><span class="z-source z-dockerfile"><span class="z-comment z-dockerfile"><span class="z-punctuation z-definition z-comment z-dockerfile">#</span> Super small runtime base image </span></span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">FROM</span> scratch </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> --from=<span class="z-variable z-stage-name">build</span> /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">COPY</span> --from=<span class="z-variable z-stage-name">build</span> /dirtyhttp-example /dirtyhttp-example </span><span class="z-source z-dockerfile"><span class="z-keyword z-control z-dockerfile">EXPOSE </span>8080 </span><span class="z-source z-dockerfile"><span class="z-keyword z-operator z-dockerfile">ENTRYPOINT </span>[<span class="z-string z-quoted z-double z-dockerfile"><span class="z-punctuation z-definition z-string z-begin z-dockerfile">&quot;</span>/dirtyhttp-example<span class="z-punctuation z-definition z-string z-end z-dockerfile">&quot;</span></span>] </span></code></pre> <p>Let's quickly go over the fancy adjustments to the build step:</p> <p>First, <code>CGO_ENABLED=0</code> disables the C interoperability and is the part of the command that means we build a binary that's been statically linked with any libraries we use.</p> <p>Next, <code>GOOS=linux</code> tells the build command that we're building a binary for the linux platform.</p> <p>The <code>-a</code> flag forces the rebuilding of all packages, even if they're up-to-date.</p> <p>Finally, the <code>-installsuffix</code> flag simply separates the current build output from any default build output in the package installation directory.</p> <p>Now that we've covered that off, let's see how the container image looks:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> build<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> go-example:soya<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>f</span> Dockerfile.soya .</span> </span></code></pre> <p><img src="https://iambenzo.com/posts/tiny-golang-containers/img/soya.png" alt="Soya Bean" /></p> <p>We've achieved an even slimmer container image - 6.83MB! Our "soya" image is just under 4 times smaller than our semi-skimmed image and a massive 58 times smaller than our full fat image!</p> <p>To run the container image, you'd use the following command:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> run<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>it</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>rm</span><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>p</span> 8080:8080 go-example:soya</span> </span></code></pre> <h2 id="summary"><a class="zola-anchor" href="#summary" aria-label="Anchor link for: summary">Summary</a></h2> <p>Typically, container registry SKUs place limits on storage space, image upload bandwidth and image download bandwidth. Increasing, or surpassing those limits costs cash money.</p> <p>With that in mind, it's our duty as penny-pinching product teams to ensure that our container images are as small as possible.</p> <p>Taking advantage of multi-stage container builds has a huge impact on container image size - a step that should be taken regardless of programming language.</p> <p>With Golang, we've been able to take the multi-stage build one step further by building a binary that will run on a <code>FROM scratch</code> base image. I can imagine that this could be applicable to any compiled language that doesn't need a runtime (C++, Rust, etc).</p> <p>I know that Spring Native is in beta right now - once it's out of beta, I'd love to see if comparable image sizes can be built with Java.</p> <p>With containers this small, it'd be a while before you start paying more for your platform!</p> Wed, 07 Oct 2020 00:00:00 +0000 Ben Burbage https://iambenzo.com/posts/cqrs-helps-me-eat-without-distraction/ https://iambenzo.com/posts/cqrs-helps-me-eat-without-distraction/ <p>Let's ask the most important question anyone worth knowing can be asked: How many chicken nuggets can I eat in 3 minutes?</p> Thu, 30 Apr 2020 00:00:00 +0000 Ben Burbage https://iambenzo.com/posts/microservice-resillience/ https://iambenzo.com/posts/microservice-resillience/ <p>It's pretty normal for you to want to have a stable system.</p> <p>What happens though, when parts of your system depend upon other 3rd party systems? What happens, when your cloud provider lumps you with a network that sometimes decides to disable itself, where the only fix is to give them even more money?</p> <blockquote> <p>Answer: Design and build with resilience in mind.</p> </blockquote> <h2 id="designing-for-known-failures"><a class="zola-anchor" href="#designing-for-known-failures" aria-label="Anchor link for: designing-for-known-failures">Designing For Known Failures</a></h2> <p>We can start by simply digging into our experience pool to unearth some of the obstacles we may have previously encountered - there's nothing new under the sun and a problem you've previously had is likely to be a problem you will experience again. Are any of your historic solutions applicable in your new, or current system?</p> <p>Architect away from single points of failure when designing your system - ensure the ability to provide redundancy for each of our system's components and also the ability to easily scale out.</p> <p>When designing your system, you can consider the Bulkhead Design Pattern:</p> <h3 id="bulkheads"><a class="zola-anchor" href="#bulkheads" aria-label="Anchor link for: bulkheads">Bulkheads</a></h3> <p>Bulkheads are used to seal off parts of boats and ships to prevent sinking.</p> <p><img src="https://upload.wikimedia.org/wikipedia/commons/e/e8/Compartments_and_watertight_subdivision_of_a_ship%27s_hull_%28Seaman%27s_Pocket-Book%2C_1943%29.jpg" alt="Bulkhead" /></p> <p>As a pattern in Microservice Architecture, it's a way of sealing off failure and avoid cascading failure (one failure leading to many more failures in the same part, and other parts of a system).</p> <p>The core concept here, is to separate out your components in a way where one failure doesn't impact the whole system and ensure degraded functionality when part of the system goes down.</p> <p>Degraded functionality is a term to express that a piece of functionality may gracefully disappear or be partially complete.</p> <h4 id="okay-so"><a class="zola-anchor" href="#okay-so" aria-label="Anchor link for: okay-so">Okay, so...</a></h4> <p>You can start by thinking about how you might want to divide your system up.</p> <p>The whole Microservice Architecture ethos lends itself to this pattern quite naturally - small, isolated services, each with their own data stores that can only be accessed via that service.</p> <p>To take that further, you can consider a service's criticality - you don't want a high volume, low critical service bombarding a message queue and hindering a highly critical and perhaps low volume service from functioning correctly.</p> <p>In that scenario, you'd provide your critical service dedicated resources, like it's own message queue to stop it's functionality being impacted by less important stuff. This can be applied to any shared resource, like a database which could be split into several databases as needed.</p> <p>Microservices suit horizontal scaling well - essentially running multiple instances of a service in order to meet demand.</p> <p>The use of Load Balancers will stop any incoming calls being directed to a misbehaving instance of a service, isolating that service from use until it returns to functioning order.</p> <p>We can also reject incoming calls to a service when that service is at maximum capacity - this is called Load Shedding.</p> <p>Finally, we can make sure we use the <a href="https://iambenzo.com/posts/microservice-resillience/#circuit-breakers">Circuit Breaker Pattern</a> on all synchronous 3rd party (upstream) calls - we'll dive into what that means momentarily.</p> <h3 id="embracing-failure"><a class="zola-anchor" href="#embracing-failure" aria-label="Anchor link for: embracing-failure">Embracing Failure</a></h3> <p>Things go wrong. That upstream service might not be accepting calls right now. Your DNS service might be trying to take today off as annual leave.</p> <p>Whilst we now have a pattern to feed into our architecture, there are three patterns that work hand-in-hand within our components to help mitigate complete system meltdown: Timeouts, Retries and Circuit Breakers.</p> <p><img src="https://iambenzo.com/posts/microservice-resillience/img/circuit-breaker.png" alt="Circuit Breaker Pattern" /></p> <h4 id="timeouts"><a class="zola-anchor" href="#timeouts" aria-label="Anchor link for: timeouts">Timeouts</a></h4> <p>If you spent any time interacting with the world-wide web, you've come across a timeout error before. It means that your browser has become fed up with waiting for data from the website you're trying to access.</p> <p>You can define how long your services wait for outgoing calls to complete and then what to do in the event that a call does take too long. This means you can avoid cascading failures by handling timeouts with default, or degraded functionality. Timeouts alone only <em>reduces</em> the risk of a service running out of threads to work on though - it'll still be accepting and trying to handle incoming requests.</p> <h4 id="retries"><a class="zola-anchor" href="#retries" aria-label="Anchor link for: retries">Retries</a></h4> <p>Timeouts are typically used in conjunction with a Retry strategy - which are ideal for combating those transient faults, where there might be a momentary loss of connectivity. Usually, you'll attempt to retry a call either straight away, or after a pre-determined amount of time. After a small number of retries, you should hook back into your timeout handling logic for default or degraded functionality.</p> <p>Please make sure that you log <strong>all</strong> retry attempts, it'll make your life much easier when you perform any tracing.</p> <h4 id="circuit-breakers"><a class="zola-anchor" href="#circuit-breakers" aria-label="Anchor link for: circuit-breakers">Circuit Breakers</a></h4> <p>Finally, you can tie both Timeouts and Retries with the Circuit Breaker pattern to not only eliminate cascading failure but also avoid having your services run out of threads.</p> <p>A circuit breaker monitors your outgoing calls for any timeouts or errors and trips when a defined failure threshold is reached.</p> <p>When tripped, your circuit breaker is 'open' and will return an error, or a default response on behalf of the upstream service without actually calling that upstream service - wave goodbye to further timeouts and hello to thread availability.</p> <p>After some time, the circuit breaker will go into a 'half-open' state and attempt to reconnect with the upstream service. If successful, the circuit breaker will 'close' up and everything will return to functioning as designed. If unsuccessful, the circuit breaker will remain 'open' and attempt to open up a little later.</p> <p>You can write your own circuit breaker using the many open source examples out there - make sure it's thread safe and at least has a configurable failure threshold and open state duration. There are 3rd party libraries kicking around, <a rel="noopener" target="_blank" href="https://github.com/resilience4j/resilience4j">Resilience4J</a> is one I've come across for Java developers.</p> <p>It's important to log your circuit breaker state changes for tracing too.</p> <h3 id="degrading-and-defaulting-functionality"><a class="zola-anchor" href="#degrading-and-defaulting-functionality" aria-label="Anchor link for: degrading-and-defaulting-functionality">Degrading and Defaulting Functionality</a></h3> <p>If we're going to fail, we should fail as fast as we can, to at least remain responsive to our users. Even though we can't always avoid sending an error message back to our caller, we should attempt providing degraded or default functionality first.</p> <p>We've already explained that degrading functionality is a term to express that a piece of functionality may gracefully disappear or be partially complete. The circuit breaker pattern allows for this partial shift in functionality.</p> <p>An example of degraded functionality is when an advertisement service is unable to return data to a website. After a circuit breaker trips, the degraded functionality could mean that adverts aren't displayed until the advertisement service starts working again.</p> <p>Default functionality is what you provide when all else fails. Normally this will be some data from a cache or, if we must, an error message.</p> <h2 id="fin"><a class="zola-anchor" href="#fin" aria-label="Anchor link for: fin">Fin</a></h2> <p>Thank you for reading this post, I know it's a long one for me.</p> <p>Hopefully, you've found it useful - if you have any thoughts or opinions on the content in this post, please do reach out to me in the comments section below.</p> Wed, 25 Mar 2020 00:00:00 +0000 Ben Burbage https://iambenzo.com/posts/maven-archetypes/ https://iambenzo.com/posts/maven-archetypes/ <p>I don't know where you stand on the raging war between Gradle and Maven for build tool of choice for Java developers but one thing that keeps me using Maven is that it has core functionality for archetypes.</p> <p>An archetype is a templated project that has configurable options available for a user of the archetype. When you use an archetype, you plug in the options made available to you and a few moments later, a project based on that archetype and your parameters has been generated for you.</p> <p>This has great advantages in an entity based API delivery - where you're interacting with a single entity (e.g. products or customers) per service. Typically for these services, you're exposing CRUD operations with small amounts of business logic and they contain a lot of boilerplate code.</p> <p>Imagine that, with a single command, all of your boilerplate code is taken care of and that you just need to focus on adding fields to your entity and business logic - lovely!</p> <h2 id="how-to-archetype"><a class="zola-anchor" href="#how-to-archetype" aria-label="Anchor link for: how-to-archetype">How to Archetype</a></h2> <h3 id="generation"><a class="zola-anchor" href="#generation" aria-label="Anchor link for: generation">Generation</a></h3> <p>Here, I'll show you some of the archetype features I've used and provide a <a rel="noopener" target="_blank" href="https://github.com/iambenzo/maven-archetype-example">sample archetype</a> for you to go through and/or use.</p> <p>The easiest way to get going, is to create a project exactly as you normally would. When you're done, navigate to the project's base directory and enter the following in a terminal:</p> <pre data-lang="sh" class="language-sh z-code"><code class="language-sh" data-lang="sh"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">mvn</span></span><span class="z-meta z-function-call z-arguments z-shell"> archetype:create-from-project</span> </span></code></pre> <p>This will create an archetype from your current project. This archetype will be found in <code>/target/generated-sources/archetype</code>. I would advise taking a copy of this directory to work on in another part of your filesystem.</p> <p>Take a brief moment to look at the <code>pom.xml</code> in the archetype project root directory. Edit as you like.</p> <p>Now that we have the base archetype, it's time to start adding parameters to it. These parameters will be used to replace parts of your code and filenames with user-specified values, when they use your archetype.</p> <h3 id="adding-parameters"><a class="zola-anchor" href="#adding-parameters" aria-label="Anchor link for: adding-parameters">Adding Parameters</a></h3> <p>First we need to define what parameters should be made available. This is done in the <code>src/main/resources/META-INF/maven/archetype-metadata.xml</code> file.</p> <p>There are some properties made available to us without any configuration but any custom properties should be defined in a <code>&lt;requiredProperties&gt;&lt;/requiredProperties&gt;</code> block like so:</p> <pre data-lang="xml" class="language-xml z-code"><code class="language-xml" data-lang="xml"><span class="z-text z-xml"><span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperties</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span> <span class="z-entity z-other z-attribute-name z-localname z-xml">key</span><span class="z-punctuation z-separator z-key-value z-xml">=</span><span class="z-string z-quoted z-double z-xml"><span class="z-punctuation z-definition z-string z-begin z-xml">&quot;</span>entityName<span class="z-punctuation z-definition z-string z-end z-xml">&quot;</span></span><span class="z-punctuation z-definition z-tag z-end z-xml">/&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span> <span class="z-entity z-other z-attribute-name z-localname z-xml">key</span><span class="z-punctuation z-separator z-key-value z-xml">=</span><span class="z-string z-quoted z-double z-xml"><span class="z-punctuation z-definition z-string z-begin z-xml">&quot;</span>entityNameLowerCase<span class="z-punctuation z-definition z-string z-end z-xml">&quot;</span></span><span class="z-punctuation z-definition z-tag z-end z-xml">/&gt;</span></span> </span><span class="z-text z-xml"> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span> <span class="z-entity z-other z-attribute-name z-localname z-xml">key</span><span class="z-punctuation z-separator z-key-value z-xml">=</span><span class="z-string z-quoted z-double z-xml"><span class="z-punctuation z-definition z-string z-begin z-xml">&quot;</span>includeCreate<span class="z-punctuation z-definition z-string z-end z-xml">&quot;</span></span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span>true<span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span> <span class="z-entity z-other z-attribute-name z-localname z-xml">key</span><span class="z-punctuation z-separator z-key-value z-xml">=</span><span class="z-string z-quoted z-double z-xml"><span class="z-punctuation z-definition z-string z-begin z-xml">&quot;</span>includeRead<span class="z-punctuation z-definition z-string z-end z-xml">&quot;</span></span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span>true<span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span> <span class="z-entity z-other z-attribute-name z-localname z-xml">key</span><span class="z-punctuation z-separator z-key-value z-xml">=</span><span class="z-string z-quoted z-double z-xml"><span class="z-punctuation z-definition z-string z-begin z-xml">&quot;</span>includeUpdate<span class="z-punctuation z-definition z-string z-end z-xml">&quot;</span></span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span>true<span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span> <span class="z-entity z-other z-attribute-name z-localname z-xml">key</span><span class="z-punctuation z-separator z-key-value z-xml">=</span><span class="z-string z-quoted z-double z-xml"><span class="z-punctuation z-definition z-string z-begin z-xml">&quot;</span>includeDelete<span class="z-punctuation z-definition z-string z-end z-xml">&quot;</span></span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span>true<span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span> <span class="z-entity z-other z-attribute-name z-localname z-xml">key</span><span class="z-punctuation z-separator z-key-value z-xml">=</span><span class="z-string z-quoted z-double z-xml"><span class="z-punctuation z-definition z-string z-begin z-xml">&quot;</span>includeEvent<span class="z-punctuation z-definition z-string z-end z-xml">&quot;</span></span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span>true<span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">defaultValue</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> <span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperty</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span><span class="z-text z-xml"> </span><span class="z-text z-xml"><span class="z-meta z-tag z-xml"><span class="z-punctuation z-definition z-tag z-begin z-xml">&lt;/</span><span class="z-entity z-name z-tag z-localname z-xml">requiredProperties</span><span class="z-punctuation z-definition z-tag z-end z-xml">&gt;</span></span> </span></code></pre> <p>I'd keep this file open, as you'll likely be dipping into and out of it whilst configuring and testing your parameters.</p> <p>Whilst you have nothing else open though, now might be a good time to ensure any unnecessary directories and/or files are removed from the <code>&lt;fileSets&gt;&lt;/fileSets&gt;</code> section as everything listed here will be part of every application generated from your archetype.</p> <p>Now that we know where to edit properties and file sets, we can navigate through our code, found in <code>src/main/resources/archetype-resources</code> and start replacing some stuff with property placeholders.</p> <h3 id="using-parameters"><a class="zola-anchor" href="#using-parameters" aria-label="Anchor link for: using-parameters">Using Parameters</a></h3> <p>An example of an out-the-box property is the <code>package</code> property, you will see that Maven has been nice enough to replace all of your package statements with something that looks like this: <code>package ${package};</code>, or <code>package ${package}.repository;</code> etc., depending upon which Java file you open.</p> <p>Every time you try to generate a project from an archetype, you will be asked by Maven for the base package name. Maven will replace every instance of <code>${package}</code> across your codebase with your input value.</p> <p>The user of our archetype will also be asked for values to assign to the required properties we recently defined too. We can use the same (<code>${value}</code>) syntax to indicate where these values should be used across our codebase:</p> <pre data-lang="java" class="language-java z-code"><code class="language-java" data-lang="java"><span class="z-source z-java"><span class="z-storage z-modifier z-java">public</span> <span class="z-support z-class z-java">Mono</span><span class="z-meta z-generic z-java"><span class="z-punctuation z-definition z-generic z-begin z-java">&lt;</span><span class="z-support z-class z-java">ServerResponse</span><span class="z-punctuation z-definition z-generic z-end z-java">&gt;</span></span> getAll$<span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityName<span class="z-punctuation z-section z-block z-end z-java">}</span></span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">s</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span><span class="z-support z-class z-java">ServerRequest</span> request<span class="z-punctuation z-section z-parens z-end z-java">)</span></span> <span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-support z-class z-java">Flux</span><span class="z-meta z-generic z-java"><span class="z-punctuation z-definition z-generic z-begin z-java">&lt;</span>$</span><span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityName<span class="z-punctuation z-section z-block z-end z-java">}</span></span><span class="z-keyword z-operator z-comparison z-java">&gt;</span> $<span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityNameLowerCase<span class="z-punctuation z-section z-block z-end z-java">}</span></span>s <span class="z-meta z-assignment z-rhs z-java"><span class="z-keyword z-operator z-assignment z-java">=</span> repository<span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">findAll</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span><span class="z-punctuation z-section z-parens z-end z-java">)</span></span></span><span class="z-punctuation z-terminator z-java">;</span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-keyword z-control z-flow z-return z-java">return</span> <span class="z-support z-class z-java">ServerResponse</span><span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">ok</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span><span class="z-punctuation z-section z-parens z-end z-java">)</span></span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">contentType</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span><span class="z-constant z-other z-java">APPLICATION_JSON</span><span class="z-punctuation z-section z-parens z-end z-java">)</span></span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">body</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span>$</span><span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityNameLowerCase<span class="z-punctuation z-section z-block z-end z-java">}</span></span>s<span class="z-punctuation z-separator z-comma z-java">,</span> $<span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityName<span class="z-punctuation z-section z-block z-end z-java">}</span></span><span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-variable z-language z-java">class</span><span class="z-invalid z-illegal z-stray-parens-end">)</span><span class="z-punctuation z-terminator z-java">;</span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-end z-java">}</span></span> </span></code></pre> <p>You may have noticed that in the earlier <code>requiredProperties</code> example, we defined a few properties that had a default value of <code>true</code>, hinting that they might be used in a boolean expression - good catch:</p> <pre data-lang="java" class="language-java z-code"><code class="language-java" data-lang="java"><span class="z-source z-java">#<span class="z-keyword z-control z-conditional z-if z-java">if</span> <span class="z-meta z-parens z-java"><span class="z-punctuation z-section z-parens z-begin z-java">(</span>$</span><span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>includeRead<span class="z-punctuation z-section z-block z-end z-java">}</span></span> <span class="z-keyword z-operator z-comparison z-java">==</span> <span class="z-string z-quoted z-double z-java"><span class="z-punctuation z-definition z-string z-begin z-java">&quot;</span>true<span class="z-punctuation z-definition z-string z-end z-java">&quot;</span></span>) </span><span class="z-source z-java"> <span class="z-storage z-modifier z-java">public</span> <span class="z-support z-class z-java">Mono</span><span class="z-meta z-generic z-java"><span class="z-punctuation z-definition z-generic z-begin z-java">&lt;</span><span class="z-support z-class z-java">ServerResponse</span><span class="z-punctuation z-definition z-generic z-end z-java">&gt;</span></span> getAll$<span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityName<span class="z-punctuation z-section z-block z-end z-java">}</span></span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">s</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span><span class="z-support z-class z-java">ServerRequest</span> request<span class="z-punctuation z-section z-parens z-end z-java">)</span></span> <span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-support z-class z-java">Flux</span><span class="z-meta z-generic z-java"><span class="z-punctuation z-definition z-generic z-begin z-java">&lt;</span>$</span><span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityName<span class="z-punctuation z-section z-block z-end z-java">}</span></span><span class="z-keyword z-operator z-comparison z-java">&gt;</span> $<span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityNameLowerCase<span class="z-punctuation z-section z-block z-end z-java">}</span></span>s <span class="z-meta z-assignment z-rhs z-java"><span class="z-keyword z-operator z-assignment z-java">=</span> repository<span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">findAll</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span><span class="z-punctuation z-section z-parens z-end z-java">)</span></span></span><span class="z-punctuation z-terminator z-java">;</span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-keyword z-control z-flow z-return z-java">return</span> <span class="z-support z-class z-java">ServerResponse</span><span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">ok</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span><span class="z-punctuation z-section z-parens z-end z-java">)</span></span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">contentType</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span><span class="z-constant z-other z-java">APPLICATION_JSON</span><span class="z-punctuation z-section z-parens z-end z-java">)</span></span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-meta z-function-call z-java"><span class="z-variable z-function z-java">body</span><span class="z-punctuation z-section z-parens z-begin z-java">(</span>$</span><span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityNameLowerCase<span class="z-punctuation z-section z-block z-end z-java">}</span></span>s<span class="z-punctuation z-separator z-comma z-java">,</span> $<span class="z-meta z-block z-java"><span class="z-punctuation z-section z-block z-begin z-java">{</span>entityName<span class="z-punctuation z-section z-block z-end z-java">}</span></span><span class="z-punctuation z-accessor z-dot z-java">.</span><span class="z-variable z-language z-java">class</span><span class="z-invalid z-illegal z-stray-parens-end">)</span><span class="z-punctuation z-terminator z-java">;</span> </span></span><span class="z-source z-java"><span class="z-meta z-block z-java"> <span class="z-punctuation z-section z-block z-end z-java">}</span></span> </span><span class="z-source z-java">#end </span></code></pre> <p>This says that if the <code>includeRead</code> property is set to <code>true</code>, include this section of code. Otherwise, don't include this code block.</p> <p>You can use Maven properties in file names <code>__likeThis__</code>:</p> <p><img src="https://iambenzo.com/posts/maven-archetypes/img/file-properties.png" alt="PropertyFileName" /></p> <h3 id="post-generation-script"><a class="zola-anchor" href="#post-generation-script" aria-label="Anchor link for: post-generation-script">Post-Generation Script</a></h3> <p>Finally, you can also provide Maven a groovy script (<code>src/main/resources/META-INF/archetype-post-generate.groovy</code> - it has to have this filename and be in this exact location) to run any custom commands <em>after</em> a project has been created from your archetype.</p> <p>The benefit of this groovy script is that it has access to all of the properties used to generate a project, so you can do some file clean-up or some extra work based upon certain property values:</p> <pre data-lang="groovy" class="language-groovy z-code"><code class="language-groovy" data-lang="groovy"><span class="z-source z-groovy"><span class="z-meta z-import z-groovy"><span class="z-keyword z-other z-import z-groovy">import</span> <span class="z-storage z-type z-import z-groovy">java.nio.file.Files </span></span></span><span class="z-source z-groovy"><span class="z-meta z-import z-groovy"><span class="z-keyword z-other z-import z-groovy">import</span> <span class="z-storage z-type z-import z-groovy">java.nio.file.Path </span></span></span><span class="z-source z-groovy"><span class="z-meta z-import z-groovy"><span class="z-keyword z-other z-import z-groovy">import</span> <span class="z-storage z-type z-import z-groovy">java.nio.file.Paths </span></span></span><span class="z-source z-groovy"> </span><span class="z-source z-groovy"><span class="z-storage z-type z-class z-groovy">Path</span> projectPath <span class="z-keyword z-operator z-assignment z-groovy">=</span> <span class="z-storage z-type z-class z-groovy">Paths</span><span class="z-punctuation z-accessor z-dot z-groovy">.</span><span class="z-meta z-method-call z-groovy"><span class="z-meta z-method z-groovy">get</span><span class="z-punctuation z-definition z-method-parameters z-begin z-groovy">(</span>request<span class="z-punctuation z-accessor z-dot z-groovy">.</span>outputDirectory<span class="z-punctuation z-definition z-separator z-parameter z-groovy">,</span> request<span class="z-punctuation z-accessor z-dot z-groovy">.</span>artifactId<span class="z-punctuation z-definition z-method-parameters z-end z-groovy">)</span></span> </span><span class="z-source z-groovy"> </span><span class="z-source z-groovy"><span class="z-storage z-type z-class z-groovy">String</span> includeEvent <span class="z-keyword z-operator z-assignment z-groovy">=</span> properties<span class="z-punctuation z-accessor z-dot z-groovy">.</span><span class="z-meta z-method-call z-groovy"><span class="z-meta z-method z-groovy">get</span><span class="z-punctuation z-definition z-method-parameters z-begin z-groovy">(</span><span class="z-string z-quoted z-double z-groovy"><span class="z-punctuation z-definition z-string z-begin z-groovy">&quot;</span>includeEvent<span class="z-punctuation z-definition z-string z-end z-groovy">&quot;</span></span><span class="z-punctuation z-definition z-method-parameters z-end z-groovy">)</span></span> </span><span class="z-source z-groovy"><span class="z-storage z-type z-class z-groovy">String</span> entityName <span class="z-keyword z-operator z-assignment z-groovy">=</span> properties<span class="z-punctuation z-accessor z-dot z-groovy">.</span><span class="z-meta z-method-call z-groovy"><span class="z-meta z-method z-groovy">get</span><span class="z-punctuation z-definition z-method-parameters z-begin z-groovy">(</span><span class="z-string z-quoted z-double z-groovy"><span class="z-punctuation z-definition z-string z-begin z-groovy">&quot;</span>entityName<span class="z-punctuation z-definition z-string z-end z-groovy">&quot;</span></span><span class="z-punctuation z-definition z-method-parameters z-end z-groovy">)</span></span> </span><span class="z-source z-groovy"> </span><span class="z-source z-groovy"><span class="z-keyword z-control z-groovy">if</span> (includeEvent <span class="z-keyword z-operator z-comparison z-groovy">!=</span> <span class="z-string z-quoted z-double z-groovy"><span class="z-punctuation z-definition z-string z-begin z-groovy">&quot;</span>true<span class="z-punctuation z-definition z-string z-end z-groovy">&quot;</span></span>) <span class="z-meta z-block z-groovy"><span class="z-punctuation z-section z-block z-begin z-groovy">{</span> </span></span><span class="z-source z-groovy"><span class="z-meta z-block z-groovy"> <span class="z-storage z-type z-class z-groovy">Files</span><span class="z-punctuation z-accessor z-dot z-groovy">.</span>deleteIfExists projectPath<span class="z-punctuation z-accessor z-dot z-groovy">.</span><span class="z-meta z-method-call z-groovy"><span class="z-meta z-method z-groovy">resolve</span><span class="z-punctuation z-definition z-method-parameters z-begin z-groovy">(</span><span class="z-string z-quoted z-double z-groovy"><span class="z-punctuation z-definition z-string z-begin z-groovy">&quot;</span>src/main/java/<span class="z-punctuation z-definition z-string z-end z-groovy">&quot;</span></span> <span class="z-keyword z-operator z-arithmetic z-groovy">+</span> packagePath <span class="z-keyword z-operator z-arithmetic z-groovy">+</span> <span class="z-string z-quoted z-double z-groovy"><span class="z-punctuation z-definition z-string z-begin z-groovy">&quot;</span>/model/<span class="z-punctuation z-definition z-string z-end z-groovy">&quot;</span></span> <span class="z-keyword z-operator z-arithmetic z-groovy">+</span> entityName <span class="z-keyword z-operator z-arithmetic z-groovy">+</span> <span class="z-string z-quoted z-double z-groovy"><span class="z-punctuation z-definition z-string z-begin z-groovy">&quot;</span>Event.java<span class="z-punctuation z-definition z-string z-end z-groovy">&quot;</span></span><span class="z-punctuation z-definition z-method-parameters z-end z-groovy">)</span></span> </span></span><span class="z-source z-groovy"><span class="z-meta z-block z-groovy"> <span class="z-storage z-type z-class z-groovy">Files</span><span class="z-punctuation z-accessor z-dot z-groovy">.</span>deleteIfExists projectPath<span class="z-punctuation z-accessor z-dot z-groovy">.</span><span class="z-meta z-method-call z-groovy"><span class="z-meta z-method z-groovy">resolve</span><span class="z-punctuation z-definition z-method-parameters z-begin z-groovy">(</span><span class="z-string z-quoted z-double z-groovy"><span class="z-punctuation z-definition z-string z-begin z-groovy">&quot;</span>src/main/java/resources/public/index.html<span class="z-punctuation z-definition z-string z-end z-groovy">&quot;</span></span><span class="z-punctuation z-definition z-method-parameters z-end z-groovy">)</span></span> </span></span><span class="z-source z-groovy"><span class="z-meta z-block z-groovy"><span class="z-punctuation z-section z-block z-end z-groovy">}</span></span> </span></code></pre> <h3 id="local-installation"><a class="zola-anchor" href="#local-installation" aria-label="Anchor link for: local-installation">Local Installation</a></h3> <p>Now that you've developed your archetype, I'm sure that you'll want to actually use it. From the root directory of your archetype project, run:</p> <pre data-lang="sh" class="language-sh z-code"><code class="language-sh" data-lang="sh"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">mvn</span></span><span class="z-meta z-function-call z-arguments z-shell"> clean install</span> </span></code></pre> <p>We can start using it now!</p> <p>To use your archetype we navigate to where we'd like to generate a new project and then enter the following into a terminal:</p> <pre data-lang="sh" class="language-sh z-code"><code class="language-sh" data-lang="sh"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">mvn</span></span><span class="z-meta z-function-call z-arguments z-shell"> archetype:generate <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DarchetypeGroupId=com.iambenzo<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DarchetypeArtifactId=webflux-crud-archetype<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> </span></code></pre> <p>Where the values for the parameters match those defined in your archetype's <code>pom.xml</code> - This will allow you to generate a new project in "Interactive Mode", meaning that you'll be asked for values for each required property one-by-one.</p> <p>You can use the same syntax <code>"-DentityName=Dave"</code> to define your properties ahead of this step. For example:</p> <pre data-lang="sh" class="language-sh z-code"><code class="language-sh" data-lang="sh"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">mvn</span></span><span class="z-meta z-function-call z-arguments z-shell"> archetype:generate <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DarchetypeGroupId=com.iambenzo<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DarchetypeArtifactId=webflux-crud-archetype<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DgroupId=com.dave<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DartifactId=dave-api-example<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-Dversion=1.0.0<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DentityName=Dave<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DentityNameLowerCase=dave<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DincludeCreate=true<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DincludeRead=true<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DincludeUpdate=true<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DincludeDelete=true<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>-DincludeEvent=true<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> </span></code></pre> <p>Maven will give you the opportunity to change your mind on any or the properties before generating your new project.</p> <h2 id="voila"><a class="zola-anchor" href="#voila" aria-label="Anchor link for: voila">Voila!</a></h2> <p>There you have it. Hopefully you can see the power and potential use-cases for Maven archetypes.</p> <p>If this blog post is a little difficult to follow, do let me know. You can also <a rel="noopener" target="_blank" href="https://github.com/iambenzo/maven-archetype-example">check out the archetype I created</a> alongside writing this post for extra context.</p> Tue, 04 Feb 2020 00:00:00 +0000 Ben Burbage https://iambenzo.com/posts/flyway/ https://iambenzo.com/posts/flyway/ <p>Anyone who knows me well, knows that I like to make things easier for myself where I can. I'd like to briefly show you a tool that helps me do that when creating DB interacting web services in Java...Flyway.</p>