Skip to main content

Run x86 Images With Kubernetes on Apple Silicon

ยท 4 min read
Sven Strittmatter
Core Developer

A close up of a computer processor chip

Cover photo by Bill Fairs on Unsplash.

Maybe you've heard from the shiny new CPUs from Apple: Silicon. Besides the good things (low power consumption, less fan noise) they have not so shiny drawbacks. One ran into is the problem of running containers built with/for x86 architecture. Yes, the problem itself is completely solved: Multi arch images. But, not every project builds them. No, I'm not looking at you DefectDojo ๐Ÿ˜‰ BTW secureCodeBox provides multi arch images ๐Ÿค— So, I tinkered around with my Mac to get our secureCodeBox setup with DefectDojo up and running on Silicon Macs. Since there was not much help out there in the Internet I use this post to summarize the steps to get it run, for later reference.

Colima FTWโ€‹

I use Colima since roundabout a year now as drop in replacement for Docker Desktop. Works great. It was never necessary to read docs. It runs x86 images emulated via Qemu. But running single containers is not sufficient for secureCodeBox. Kubernetes is mandatory. Until now, I used Minikube, but it can't run x86 images on Silicon Macs. KIND also does not support them, as my colleagues told me. Some days ago, I told a friend about Colima, and he said: "Oh, nice. It can start a Kubernetes cluster."

Remember, I've never read the docs ๐Ÿ˜ฌ To install Colima and start a Kubernetes just execute (I assume you have Homebrew installed.):

brew install colima
colima start -f --kubernetes --arch x86_64
caution

This will emulate an x86 vm under the hood. It is not virtualized as usual. This brings a performance penalty.

Should I Use Brew Services to Launch Colima at Login?โ€‹

TL;DR: No, don't!

Brew offers very simple solution to start such services on login it. Just simply run brew services start colima and Colima will always start on login.

caution

Never use brew services with sudo! This will break your Homebrew installation: You can't update anymore without hassle. The reason for that: Homebrew assumes that it is always executed in the context of an unprivileged user. If you run brew services with sudo files wil be written with "root" as owner. Since Homebrew always runs with your unprivileged user it can't modify such files anymore. Been there, done that. Its no good!

The "problem" with brew services ia, that it always uses the LaunchAgents plist-File from the brew. For Colima this means that brew services start colima always copies the file from the Homebrew's Formula to ~/Library/LaunchAgents/homebrew.mxcl.colima.plist. But since this LaunchAgents definition invokes colima without the arguments --kubernetes and --arch x86_64 you need to modify it:

...
<key>ProgramArguments</key>
<array>
<string>/opt/homebrew/opt/colima/bin/colima</string>
<string>start</string>
<string>-f</string>
</array>
...

If you modify this file and restart the daemon via brew services your changes will be lost! And this is by design.

You have two options:

  1. Either start it by hand: colima start --kubernetes --arch x86_64 or
  2. handroll your own LaunchDaemon:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>/opt/homebrew/bin:/opt/homebrew/sbin:/usr/bin:/bin:/usr/sbin:/sbin</string>
</dict>
<key>KeepAlive</key>
<dict>
<key>SuccessfulExit</key>
<true/>
</dict>
<key>Label</key>
<string>de.weltraumschaf.colima</string>
<key>LimitLoadToSessionType</key>
<array>
<string>Aqua</string>
<string>Background</string>
<string>LoginWindow</string>
<string>StandardIO</string>
</array>
<key>ProgramArguments</key>
<array>
<string>/opt/homebrew/opt/colima/bin/colima</string>
<string>start</string>
<string>-f</string>
<string>--kubernetes</string>
<string>--arch</string>
<string>x86_64</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>StandardErrorPath</key>
<string>/opt/homebrew/var/log/colima.log</string>
<key>StandardOutPath</key>
<string>/opt/homebrew/var/log/colima.log</string>
<key>WorkingDirectory</key>
<string>/Users/sst</string>
</dict>
</plist>

And store it in the file ~/Library/LaunchAgents/de.weltraumschaf.colima.plist. Obviously, change "de.weltraumschaf" to whatever you like. Instead of Homebrew, now you need to use launchctl to interact with the LaunchAgent.

Install secureCodeBox with DefectDojoโ€‹

The rest is straight forward. To install secureCodeBox simply execute (as documented here):

helm --namespace securecodebox-system \
upgrade \
--install \
--create-namespace \
securecodebox-operator \
oci://ghcr.io/securecodebox/helm/operator

Then install the scanners you want, e.g. [Nmap](https://nmap.org/:

helm install nmap oci://ghcr.io/securecodebox/helm/nmap
kubectl get scantypes

To install DefectDojo the easiest way is to clone their repo and install from it (as documented here):

git clone https://github.com/DefectDojo/django-DefectDojo
cd django-DefectDojo

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm dependency update ./helm/defectdojo

helm upgrade --install \
defectdojo \
./helm/defectdojo \
--set django.ingress.enabled=true \
--set django.ingress.activateTLS=false \
--set createSecret=true \
--set createRabbitMqSecret=true \
--set createRedisSecret=true \
--set createMysqlSecret=true \
--set createPostgresqlSecret=true \
--set host="defectdojo.default.svc.cluster.local" \
--set "alternativeHosts={localhost}"

Get DefectDojo admin user password:

echo "DefectDojo admin password: $(kubectl \
get secret defectdojo \
--namespace=default \
--output jsonpath='{.data.DD_ADMIN_PASSWORD}' \
| base64 --decode)"

Finally forward port to service:

kubectl port-forward svc/defectdojo-django 8080:80 -n default

Now you can visit the DefectDojo web UI at http://localhost:8080.