Skip to content

Deploy on OpenShift

Overview

Run the as a Deployment in its own namespace. The image is built inside the cluster from the signed britive-broker RPM onto ubi-micro, so it carries no package manager or build tools. The broker reads its tenant and pool token from environment variables, needs no broker-config.yml, and runs as OpenShift’s arbitrary non-root UID. Checkout scripts reach clusters, including this one, through kubeconfigs mounted from a Secret, the same way they would reach a remote cluster.

What you’ll accomplish:

  • Create the broker namespace, identities and kubeconfig Secret
  • Store the tenant subdomain and token in a Secret
  • Build the broker image in-cluster and deploy it
  • Confirm the broker connects to its pool

Before You Begin

  • OpenShift 4.x with cluster-admin access (oc)
  • Tenant subdomain: mycompany for https://mycompany.britive-app.com
  • Broker pool token: System Administration → Brokers and Broker Pools → open the pool → Broker Pool Token → Copy Token
  • Outbound HTTPS (443) from the cluster to <tenant>.britive-app.com, the MQTT endpoint the broker receives at bootstrap (a *.britive-app.com host), S3 (for downloaded scripts), and download.britive.com and dl.k8s.io during the image build

Private API clusters (for example private ARO) work the same way: run oc from a host inside the cluster’s network, such as a jump host or self-hosted runner.

Deploy

Create the namespace and identities

britive-broker is the identity checkout scripts use through the kubeconfig. britive-broker-pod runs the pod and mounts no API token.

broker-identity.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: britive-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: britive-broker
  namespace: britive-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: britive-broker-pod
  namespace: britive-system
oc apply -f broker-identity.yaml

Grant britive-broker the roles your resource types need, for example the namespace and scan roles in OpenShift namespace resource type.

Store the broker kubeconfig

Mint a token for britive-broker and write a kubeconfig named after the cluster key your resources use (cluster_name). Set the token lifetime to your rotation schedule.

KEY=<cluster>
API=https://api.<cluster-domain>:6443
TOKEN=$(oc -n britive-system create token britive-broker --duration=168h)
oc config --kubeconfig "$KEY.kubeconfig" set-cluster "$KEY" --server "$API"
oc config --kubeconfig "$KEY.kubeconfig" set-credentials britive-broker --token "$TOKEN"
oc config --kubeconfig "$KEY.kubeconfig" set-context "$KEY" --cluster "$KEY" --user britive-broker
oc config --kubeconfig "$KEY.kubeconfig" use-context "$KEY"
oc -n britive-system create secret generic britive-broker-kubeconfigs --from-file="$KEY.kubeconfig"
rm "$KEY.kubeconfig"

If the API certificate is not publicly trusted, add --certificate-authority <ca-file> --embed-certs to set-cluster.

Store the tenant and pool token

oc -n britive-system create secret generic britive-broker-config \
  --from-literal=BRITIVE_BROKER_TENANT_SUBDOMAIN=<tenant> \
  --from-literal=BRITIVE_BROKER_AUTH_TOKEN='<pool-token>'

Write the Dockerfile

Multi-stage: full UBI installs the RPM from the signed Britive repository and the runtime packages into a ubi-micro root. ca-certificates is required (ubi-micro has no CA bundle). hostname is required: the broker names itself from it. kubectl is linked as oc for scripts that use plain Kubernetes commands; install the oc client instead if your scripts need OpenShift-specific commands.

broker/Dockerfile
ARG UBI_TAG=9.8
FROM registry.access.redhat.com/ubi9/ubi-micro:${UBI_TAG} AS micro

FROM registry.access.redhat.com/ubi9/ubi:${UBI_TAG} AS build
ARG BROKER_VERSION=""
ARG KUBECTL_VERSION=v1.34.13
RUN rpm --import https://download.britive.com/keys/britive.gpg \
 && printf '%s\n' '[britive]' 'name=Britive Central Downloads' \
      'baseurl=https://download.britive.com/rpm/stable' 'enabled=1' 'gpgcheck=1' \
      'repo_gpgcheck=1' 'gpgkey=https://download.britive.com/keys/britive.gpg' \
      > /etc/yum.repos.d/britive.repo \
 && dnf -y install "britive-broker${BROKER_VERSION:+-$BROKER_VERSION}" \
 && /opt/britive-broker/britive-broker help >/dev/null
COPY --from=micro / /rootfs/
RUN dnf -y install --installroot /rootfs --releasever 9 \
      --setopt=reposdir=/etc/yum.repos.d --setopt=install_weak_deps=False --nodocs \
      ca-certificates jq sed grep hostname \
 && dnf --installroot /rootfs clean all
RUN curl -fsSLo /tmp/kubectl "https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl" \
 && echo "$(curl -fsSL "https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl.sha256")  /tmp/kubectl" | sha256sum -c - \
 && install -m 0755 /tmp/kubectl /rootfs/usr/local/bin/kubectl \
 && ln -s kubectl /rootfs/usr/local/bin/oc
COPY entrypoint.sh /rootfs/opt/broker/entrypoint.sh
# The RPM ships the binary 0750; OpenShift runs an arbitrary UID in group 0.
RUN mkdir -p /rootfs/opt/britive-broker /rootfs/var/lib/britive \
 && cp /opt/britive-broker/britive-broker /rootfs/opt/britive-broker/ \
 && chmod 0755 /rootfs/opt/broker/entrypoint.sh \
 && chgrp -R 0 /rootfs/opt/britive-broker /rootfs/var/lib/britive \
 && chmod -R g=u /rootfs/opt/britive-broker /rootfs/var/lib/britive

FROM micro
COPY --from=build /rootfs/ /
USER 1001
WORKDIR /var/lib/britive
ENTRYPOINT ["/opt/broker/entrypoint.sh"]
broker/entrypoint.sh
#!/usr/bin/env bash
set -euo pipefail
: "${BRITIVE_BROKER_TENANT_SUBDOMAIN:?}" "${BRITIVE_BROKER_AUTH_TOKEN:?}"
mkdir -p "${BRITIVE_BROKER_CACHE_PATH:-$PWD/cache}" "$(dirname "${BRITIVE_BROKER_LOG_FILE:-$PWD/logs/x}")"
exec /opt/britive-broker/britive-broker run

Put your local scripts (if any) in the same folder and copy them into /rootfs in the build stage. Scripts uploaded to Britive are downloaded at checkout and need no image change.

Build the image in the cluster

oc -n britive-system new-build --name britive-broker --strategy docker --binary
oc -n britive-system start-build britive-broker --from-dir broker --follow --wait

Binary builds ignore --build-arg. To pin a broker version, set the BROKER_VERSION default in the Dockerfile.

Deploy the broker

broker-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: britive-broker
  namespace: britive-system
spec:
  replicas: 1
  selector:
    matchLabels: { app: britive-broker }
  template:
    metadata:
      labels: { app: britive-broker }
    spec:
      serviceAccountName: britive-broker-pod
      automountServiceAccountToken: false
      containers:
        - name: broker
          image: image-registry.openshift-image-registry.svc:5000/britive-system/britive-broker:latest
          workingDir: /var/lib/britive
          envFrom:
            - secretRef: { name: britive-broker-config }
          env:
            - { name: BRITIVE_BROKER_CONFIG_VERSION, value: "2" }
            - { name: BRITIVE_BROKER_CACHE_PATH, value: /var/lib/britive/cache }
            - { name: BRITIVE_BROKER_LOG_FILE, value: /var/lib/britive/logs/britive-broker.log }
            - { name: KUBECONFIG_DIR, value: /etc/britive/kubeconfigs }
          resources:
            requests: { cpu: 100m, memory: 256Mi }
            limits: { cpu: 500m, memory: 768Mi }
          securityContext:
            allowPrivilegeEscalation: false
            capabilities: { drop: ["ALL"] }
          volumeMounts:
            - { name: kubeconfigs, mountPath: /etc/britive/kubeconfigs, readOnly: true }
            - { name: work, mountPath: /var/lib/britive }
      volumes:
        - name: kubeconfigs
          secret: { secretName: britive-broker-kubeconfigs }
        - name: work
          emptyDir: {}
oc apply -f broker-deployment.yaml
oc -n britive-system rollout status deploy/britive-broker

BRITIVE_BROKER_CONFIG_VERSION=2 enables scans and rotation in addition to checkout and check-in. Without a broker-config.yml (and its resource_types section) the broker serves every resource type routed to its pool; dedicate the pool to the resource types this broker should run.

Verify

oc -n britive-system logs deploy/britive-broker | grep -E "starting|Connection success"
msg="No config file found. Using environment variables only." path=config/broker-config.yml
msg="Britive broker starting" version=3.1.1 configVersion=2
msg="Connection success" clientID=agent:...

The broker appears as connected in its pool, named after the pod.

The broker logs its MQTT connection details, including credential fields, at INFO level. Treat broker logs as sensitive and redact them before sharing.

Operations

TaskHow
Rotate the broker’s cluster tokenRe-run the kubeconfig step, then oc -n britive-system rollout restart deploy/britive-broker
Upgrade the brokerChange BROKER_VERSION (or leave it empty for the latest), oc start-build ... --follow, then oc rollout restart
Read broker activityoc -n britive-system logs on the newest pod; during a rollout deploy/ may pick the terminating one

Troubleshoot

SymptomCauseFix
tls: failed to verify certificate: x509: certificate signed by unknown authorityImage without a CA bundleInstall ca-certificates into the ubi-micro root
permission denied running /opt/britive-broker/britive-brokerBinary not executable for the arbitrary UIDchgrp -R 0 and chmod -R g=u on /opt/britive-broker
Broker bootstrap failed ... HTTP status code: 403Wrong or revoked pool token, or wrong subdomainRecreate britive-broker-config with a current token
Checkout scripts fail with kubeconfig not foundSecret key differs from the resource’s cluster_nameName the kubeconfig <cluster_name>.kubeconfig

Next Steps

Last updated on