Photo by Kelly Sikkema on Unsplash

So, I recently had the opportunity to extensively dig into Gitlab’s CI/CD. We have a monorepo that I wanted to deploy automatically, and it took quite a bit of work to optimize it and work as I expected.

To set the stage, here’s our core .gitlab-ci.yml file (now):

image: node:18-alpine3.19

variables:
  AWS_REGION: us-east-1
  AWS_DEFAULT_REGION: us-east-1
  FF_USE_FASTZIP: "true"
  ARTIFACT_COMPRESSION_LEVEL: "fast"
  CACHE_COMPRESSION_LEVEL: "fast"

stages:
  - prepare
  - module-caches
  - layers
  - build
  - deploy

#******************************************************************

include:
  - local: ".gitlab/cache-ci.yml"
  - local: ".gitlab/rules-ci.yml"
  - local: ".gitlab/build-ci.yml"
  - local: ".gitlab/deploy-ci.yml"
  - local: "apps/landing/.gitlab-cache-ci.yml"
  - local: "apps/landing/.gitlab-ci.yml"
  - local: "apps/quickbooks-connector/.gitlab-ci.yml"
  - local: "apps/reviews/reviews-api/.gitlab-cache-ci.yml"
  - local: "apps/reviews/reviews-api/.gitlab-ci.yml"
  - local: "apps/reviews/reviews-ui/.gitlab-cache-ci.yml"
  - local: "apps/reviews/reviews-ui/.gitlab-ci.yml"
  - local: "apps/stocks/stocks-api/.gitlab-cache-ci.yml"
  - local: "apps/stocks/stocks-api/.gitlab-ci.yml"
  - local: "apps/stocks/stocks-ui/.gitlab-cache-ci.yml"
  - local: "apps/stocks/stocks-ui/.gitlab-ci.yml"
  - local: "apps/tnb/tnb-api/.gitlab-cache-ci.yml"
  - local: "apps/tnb/tnb-api/.gitlab-ci.yml"
  - local: "apps/tnb/tnb-ui/.gitlab-cache-ci.yml"
  - local: "apps/tnb/tnb-ui/.gitlab-ci.yml"
  - local: "packages/pinnacle-layer/.gitlab-cache-ci.yml"
  - local: "packages/pinnacle-layer/.gitlab-ci.yml"

# variables:
#   CI_DEBUG_TRACE: true

#******************************************************************

always:
  stage: prepare
  script:
    - env | sort

Here are some of the things that drove me crazy:

In starting this blog, I wanted to use AWS S3/CloudFront. I initially investigated SST (ion specifically). However, I kept running into issues with SST and NixOS, so I bailed on SST and moved directly to Pulumi. Pulumi is a Terraform competitor, and I have been curious to try it out.

They have a static site starter. This is actually a decent start, but I ran into a few problems deploying my new Eleventy Blog. The home page worked fine, but as you drive into sub-folders (like /blog/), I started getting 403 Forbidden errors.

Have you ever wished the Databricks widgets were a little more intelligent? When you change one, the others adjust?

That’s not too hard, but it’s a bit tricky. Here’s the secret: The widgets must be replaced as they change! For example:

Let’s say you have a date input, and whenever it changes, you want a second widget to change. That second widget needs to be suffixed with an increasing number as the contents change. Consider:

Recently, I ran into a case where I needed to check if 11 different fields were null. Yes, I could have used Copilot (or my preferred Codeium) to generate it for me, but I knew it had to be easier. There had to be an easier way…

Here’s the scenario:

• You have a list of conditions in a python list (in my case, I pasted it as separate lines and did a conditions = """condition 1\ncondition2\n...".split("\n")). Doesn’t matter how you specify your conditions, just that it’s a list somehow
• You have a way of associating this list with a set of Databricks columns

Here’s what I found: