Deploy on OpenShift
Overview
Run the as a Deployment in its own namespace. The image is built inside the cluster from the signed britive-broker RPM onto ubi-micro, so it carries no package manager or build tools. The broker reads its tenant and pool token from environment variables, needs no broker-config.yml, and runs as OpenShift’s arbitrary non-root UID. Checkout scripts reach clusters, including this one, through kubeconfigs mounted from a Secret, the same way they would reach a remote cluster.
What you’ll accomplish:
- Create the broker namespace, identities and kubeconfig Secret
- Store the tenant subdomain and token in a Secret
- Build the broker image in-cluster and deploy it
- Confirm the broker connects to its pool
Before You Begin
- OpenShift 4.x with cluster-admin access (
oc) - Tenant subdomain:
mycompanyforhttps://mycompany.britive-app.com - Broker pool token: System Administration → Brokers and Broker Pools → open the pool → Broker Pool Token → Copy Token
- Outbound HTTPS (443) from the cluster to
<tenant>.britive-app.com, the MQTT endpoint the broker receives at bootstrap (a*.britive-app.comhost), S3 (for downloaded scripts), anddownload.britive.comanddl.k8s.ioduring the image build
Private API clusters (for example private ARO) work the same way: run oc from a host inside the cluster’s network, such as a jump host or self-hosted runner.
Deploy
Create the namespace and identities
britive-broker is the identity checkout scripts use through the kubeconfig. britive-broker-pod runs the pod and mounts no API token.
apiVersion: v1
kind: Namespace
metadata:
name: britive-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: britive-broker
namespace: britive-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: britive-broker-pod
namespace: britive-systemoc apply -f broker-identity.yamlGrant britive-broker the roles your resource types need, for example the namespace and scan roles in OpenShift namespace resource type.
Store the broker kubeconfig
Mint a token for britive-broker and write a kubeconfig named after the cluster key your resources use (cluster_name). Set the token lifetime to your rotation schedule.
KEY=<cluster>
API=https://api.<cluster-domain>:6443
TOKEN=$(oc -n britive-system create token britive-broker --duration=168h)
oc config --kubeconfig "$KEY.kubeconfig" set-cluster "$KEY" --server "$API"
oc config --kubeconfig "$KEY.kubeconfig" set-credentials britive-broker --token "$TOKEN"
oc config --kubeconfig "$KEY.kubeconfig" set-context "$KEY" --cluster "$KEY" --user britive-broker
oc config --kubeconfig "$KEY.kubeconfig" use-context "$KEY"
oc -n britive-system create secret generic britive-broker-kubeconfigs --from-file="$KEY.kubeconfig"
rm "$KEY.kubeconfig"If the API certificate is not publicly trusted, add --certificate-authority <ca-file> --embed-certs to set-cluster.
Store the tenant and pool token
oc -n britive-system create secret generic britive-broker-config \
--from-literal=BRITIVE_BROKER_TENANT_SUBDOMAIN=<tenant> \
--from-literal=BRITIVE_BROKER_AUTH_TOKEN='<pool-token>'Write the Dockerfile
Multi-stage: full UBI installs the RPM from the signed Britive repository and the runtime packages into a ubi-micro root. ca-certificates is required (ubi-micro has no CA bundle). hostname is required: the broker names itself from it. kubectl is linked as oc for scripts that use plain Kubernetes commands; install the oc client instead if your scripts need OpenShift-specific commands.
ARG UBI_TAG=9.8
FROM registry.access.redhat.com/ubi9/ubi-micro:${UBI_TAG} AS micro
FROM registry.access.redhat.com/ubi9/ubi:${UBI_TAG} AS build
ARG BROKER_VERSION=""
ARG KUBECTL_VERSION=v1.34.13
RUN rpm --import https://download.britive.com/keys/britive.gpg \
&& printf '%s\n' '[britive]' 'name=Britive Central Downloads' \
'baseurl=https://download.britive.com/rpm/stable' 'enabled=1' 'gpgcheck=1' \
'repo_gpgcheck=1' 'gpgkey=https://download.britive.com/keys/britive.gpg' \
> /etc/yum.repos.d/britive.repo \
&& dnf -y install "britive-broker${BROKER_VERSION:+-$BROKER_VERSION}" \
&& /opt/britive-broker/britive-broker help >/dev/null
COPY --from=micro / /rootfs/
RUN dnf -y install --installroot /rootfs --releasever 9 \
--setopt=reposdir=/etc/yum.repos.d --setopt=install_weak_deps=False --nodocs \
ca-certificates jq sed grep hostname \
&& dnf --installroot /rootfs clean all
RUN curl -fsSLo /tmp/kubectl "https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl" \
&& echo "$(curl -fsSL "https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl.sha256") /tmp/kubectl" | sha256sum -c - \
&& install -m 0755 /tmp/kubectl /rootfs/usr/local/bin/kubectl \
&& ln -s kubectl /rootfs/usr/local/bin/oc
COPY entrypoint.sh /rootfs/opt/broker/entrypoint.sh
# The RPM ships the binary 0750; OpenShift runs an arbitrary UID in group 0.
RUN mkdir -p /rootfs/opt/britive-broker /rootfs/var/lib/britive \
&& cp /opt/britive-broker/britive-broker /rootfs/opt/britive-broker/ \
&& chmod 0755 /rootfs/opt/broker/entrypoint.sh \
&& chgrp -R 0 /rootfs/opt/britive-broker /rootfs/var/lib/britive \
&& chmod -R g=u /rootfs/opt/britive-broker /rootfs/var/lib/britive
FROM micro
COPY --from=build /rootfs/ /
USER 1001
WORKDIR /var/lib/britive
ENTRYPOINT ["/opt/broker/entrypoint.sh"]#!/usr/bin/env bash
set -euo pipefail
: "${BRITIVE_BROKER_TENANT_SUBDOMAIN:?}" "${BRITIVE_BROKER_AUTH_TOKEN:?}"
mkdir -p "${BRITIVE_BROKER_CACHE_PATH:-$PWD/cache}" "$(dirname "${BRITIVE_BROKER_LOG_FILE:-$PWD/logs/x}")"
exec /opt/britive-broker/britive-broker runPut your local scripts (if any) in the same folder and copy them into /rootfs in the build stage. Scripts uploaded to Britive are downloaded at checkout and need no image change.
Build the image in the cluster
oc -n britive-system new-build --name britive-broker --strategy docker --binary
oc -n britive-system start-build britive-broker --from-dir broker --follow --waitBinary builds ignore --build-arg. To pin a broker version, set the BROKER_VERSION default in the Dockerfile.
Deploy the broker
apiVersion: apps/v1
kind: Deployment
metadata:
name: britive-broker
namespace: britive-system
spec:
replicas: 1
selector:
matchLabels: { app: britive-broker }
template:
metadata:
labels: { app: britive-broker }
spec:
serviceAccountName: britive-broker-pod
automountServiceAccountToken: false
containers:
- name: broker
image: image-registry.openshift-image-registry.svc:5000/britive-system/britive-broker:latest
workingDir: /var/lib/britive
envFrom:
- secretRef: { name: britive-broker-config }
env:
- { name: BRITIVE_BROKER_CONFIG_VERSION, value: "2" }
- { name: BRITIVE_BROKER_CACHE_PATH, value: /var/lib/britive/cache }
- { name: BRITIVE_BROKER_LOG_FILE, value: /var/lib/britive/logs/britive-broker.log }
- { name: KUBECONFIG_DIR, value: /etc/britive/kubeconfigs }
resources:
requests: { cpu: 100m, memory: 256Mi }
limits: { cpu: 500m, memory: 768Mi }
securityContext:
allowPrivilegeEscalation: false
capabilities: { drop: ["ALL"] }
volumeMounts:
- { name: kubeconfigs, mountPath: /etc/britive/kubeconfigs, readOnly: true }
- { name: work, mountPath: /var/lib/britive }
volumes:
- name: kubeconfigs
secret: { secretName: britive-broker-kubeconfigs }
- name: work
emptyDir: {}oc apply -f broker-deployment.yaml
oc -n britive-system rollout status deploy/britive-brokerBRITIVE_BROKER_CONFIG_VERSION=2 enables scans and rotation in addition to checkout and check-in. Without a broker-config.yml (and its resource_types section) the broker serves every resource type routed to its pool; dedicate the pool to the resource types this broker should run.
Verify
oc -n britive-system logs deploy/britive-broker | grep -E "starting|Connection success"msg="No config file found. Using environment variables only." path=config/broker-config.yml
msg="Britive broker starting" version=3.1.1 configVersion=2
msg="Connection success" clientID=agent:...The broker appears as connected in its pool, named after the pod.
The broker logs its MQTT connection details, including credential fields, at INFO level. Treat broker logs as sensitive and redact them before sharing.
Operations
| Task | How |
|---|---|
| Rotate the broker’s cluster token | Re-run the kubeconfig step, then oc -n britive-system rollout restart deploy/britive-broker |
| Upgrade the broker | Change BROKER_VERSION (or leave it empty for the latest), oc start-build ... --follow, then oc rollout restart |
| Read broker activity | oc -n britive-system logs on the newest pod; during a rollout deploy/ may pick the terminating one |
Troubleshoot
| Symptom | Cause | Fix |
|---|---|---|
tls: failed to verify certificate: x509: certificate signed by unknown authority | Image without a CA bundle | Install ca-certificates into the ubi-micro root |
permission denied running /opt/britive-broker/britive-broker | Binary not executable for the arbitrary UID | chgrp -R 0 and chmod -R g=u on /opt/britive-broker |
Broker bootstrap failed ... HTTP status code: 403 | Wrong or revoked pool token, or wrong subdomain | Recreate britive-broker-config with a current token |
Checkout scripts fail with kubeconfig not found | Secret key differs from the resource’s cluster_name | Name the kubeconfig <cluster_name>.kubeconfig |