Kubernetes
A popular option for Orchestration is Kubernetes. Kubernetes is an open-source system for automating the deployment, scaling and management of containerized applications. We provide examples that you can use to quickly get started using Kubernetes to orchestrate your Synapse deployment. These examples include an Aha cell, an Axon, a Cortex, the Maxmind connector, and the Optic UI.
Since all Telepath services connect via Aha, this allows for easy lookup of services via Aha. This allows for users to
ignore most application awareness of port numbers. For example, the Maxmind connector can easily be added to the
Cortex via service.add maxmind aha://root:demo@maxmind.aha.demo.net
.
The Optic deployment uses an initContainers
container to copy the TLS certificates into the service directory for
Optic. The Traefik IngressRouteTCP
directs all TLS traffic to the service to the Optic service. Since the TLS
certificates have been put into the Cell directory for Optic, and the IngressRouteTCP
acts a TLS passthrough,
users are using TLS end to end to connect to Optic.
Passwords used for doing inter-service communications are stored in Kubernetes Secrets and are interpolated from environment variables to form Telepath URLs when needed. To keep these examples from being too large, passwords are shared between services.
The following examples make the following assumptions:
A PersistentVolumeClaim provider is available. These examples use Digital Ocean block storage.
Traefik is available to provide
IngressRouteTCP
providers. The examples here are treated as TLS passthrough examples with a default websecureentryPoint
, which means the service must provide its own TLS endpoint. Further Traefik configuration for providing TLS termination and connecting to backend services over TLS is beyond the scope of this documentation.There is a
cert-manager
Certificate provider available to generate a Let’s Encrypt TLS certificate.There is a secret
regcred
available which can be used to pull a Docker pull secret that can access the private images.
Single Pod
This single pod example can be readily used, provided that the assumptions noted earlier are accounted for. The DNS name for the Certificate, IngressRouteTCP, and SYN_OPTIC_NETLOC value would need to be updated to account for your own DNS settings.
apiVersion: v1
kind: Secret
metadata:
name: ex00-passwds
labels:
app: demostack
data:
# secretpass
corepass: "c2VjcmV0cGFzcw=="
# ssapterces
svcpass: "c3NhcHRlcmNlcw=="
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: cert-ex00
labels:
app: examplestack
instance: ex00
spec:
secretName: ex00-staging-tls
duration: 2160h # 90d
renewBefore: 360h # 15d
subject:
organizations:
- your-org
commonName: ex00.k8s.yourdomain.tld
isCA: false
privateKey:
algorithm: RSA
encoding: PKCS1
size: 2048
usages:
- server auth
- client auth
dnsNames:
- ex00.k8s.yourdomain.tld
issuerRef:
name: le-http-staging-issuer
kind: ClusterIssuer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: examplestack-ex00
labels:
app: examplestack
instance: ex00
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 30Gi
storageClassName: do-block-storage
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: examplestack-ex00
labels:
app: examplestack
instance: ex00
spec:
selector:
matchLabels:
app: examplestack
template:
metadata:
labels:
app: examplestack
instance: ex00
spec:
volumes:
- name: ex-data
persistentVolumeClaim:
claimName: examplestack-ex00
- name: optic-tls
secret:
secretName: ex00-staging-tls
items:
- key: tls.key
path: sslkey.pem
- key: tls.crt
path: sslcert.pem
initContainers:
- name: init-optic00
image: vertexproject/synapse:v2.x.x
imagePullPolicy: Always
command: ['/bin/bash']
args:
- -c
- >-
cp -v --target-directory /vertex/storage /vertex/tls/sslkey.pem /vertex/tls/sslcert.pem
volumeMounts:
- name: optic-tls
mountPath: /vertex/tls
readOnly: true
- name: ex-data
mountPath: /vertex/storage
subPath: optic
containers:
- name: core00
image: vertexproject/synapse-cortex:v2.x.x
command: ["python"]
args: ["-O", "-m", "synapse.servers.cortex", "/vertex/storage"]
env:
- name: SYN_CORTEX_DMON_LISTEN
value: "tcp://0.0.0.0:27492/"
- name: SYN_CORTEX_STORM_LOG
value: "true"
- name: SYN_CORTEX_STORM_LOG_LEVEL
value: INFO
- name: SYN_CORTEX_HTTPS_PORT
value: null
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_CORTEX_AHA_NETWORK
value: aha.demo.net
- name: SYN_CORTEX_AHA_NAME
value: cortex
- name: SYN_CORTEX_AUTH_PASSWD
valueFrom:
secretKeyRef:
name: ex00-passwds
key: corepass
- name: SYN_SVC_PASSWD
valueFrom:
secretKeyRef:
name: ex00-passwds
key: svcpass
- name: SYN_CORTEX_AHA_REGISTRY
value: tcp://root:$(SYN_SVC_PASSWD)@127.0.0.1:28000/
- name: SYN_CORTEX_AXON
value: aha://root:$(SYN_SVC_PASSWD)@axon.aha.demo.net/
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: core
imagePullPolicy: Always
- name: aha00
image: vertexproject/synapse-aha:master
env:
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_AHACELL_DMON_LISTEN
value: "tcp://0.0.0.0:28000"
- name: SYN_AHACELL_AUTH_PASSWD
valueFrom:
secretKeyRef:
name: ex00-passwds
key: svcpass
- name: SYN_AHACELL_HTTPS_PORT
value: null
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: aha
imagePullPolicy: Always
- name: axon00
image: vertexproject/synapse-axon:v2.x.x
env:
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_AXON_DMON_LISTEN
value: "tcp://0.0.0.0:0"
- name: SYN_AXON_HTTPS_PORT
value: null
- name: SYN_SVC_PASSWD
valueFrom:
secretKeyRef:
name: ex00-passwds
key: svcpass
- name: SYN_AXON_AHA_REGISTRY
value: tcp://root:$(SYN_SVC_PASSWD)@127.0.0.1:28000/
- name: SYN_AXON_AUTH_PASSWD
value: $(SYN_SVC_PASSWD)
- name: SYN_AXON_AHA_NETWORK
value: aha.demo.net
- name: SYN_AXON_AHA_NAME
value: axon
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: axon
imagePullPolicy: Always
- name: maxmind00
image: vertexproject/synapse-maxmind:v2.x.x
env:
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_MAXMIND_DMON_LISTEN
value: "tcp://0.0.0.0:0"
- name: SYN_MAXMIND_HTTPS_PORT
value: null
- name: SYN_SVC_PASSWD
valueFrom:
secretKeyRef:
name: ex00-passwds
key: svcpass
- name: SYN_MAXMIND_AHA_REGISTRY
value: tcp://root:$(SYN_SVC_PASSWD)@127.0.0.1:28000/
- name: SYN_MAXMIND_AUTH_PASSWD
value: $(SYN_SVC_PASSWD)
- name: SYN_MAXMIND_AHA_NETWORK
value: aha.demo.net
- name: SYN_MAXMIND_AHA_NAME
value: maxmind
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: maxmind
imagePullPolicy: Always
- name: optic00
image: vertexproject/optic:v2.x.x
env:
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_OPTIC_EMAIL_HOST
value: exim
- name: SYN_OPTIC_DMON_LISTEN
value: "null"
- name: SYN_OPTIC_NETLOC
value: "ex00.k8s.yourdomain.tld"
- name: SYN_SVC_PASSWD
valueFrom:
secretKeyRef:
name: ex00-passwds
key: svcpass
- name: SYN_OPTIC_AHA_REGISTRY
value: tcp://root:$(SYN_SVC_PASSWD)@127.0.0.1:28000/
- name: SYN_OPTIC_AXON
value: aha://root:$(SYN_SVC_PASSWD)@axon.aha.demo.net/
- name: SYN_CORTEX_PASSWD
valueFrom:
secretKeyRef:
name: ex00-passwds
key: corepass
- name: SYN_OPTIC_CORTEX
value: "aha://root:$(SYN_CORTEX_PASSWD)@cortex.aha.demo.net/"
ports:
- containerPort: 4443
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: optic
imagePullPolicy: Always
restartPolicy: Always
imagePullSecrets:
- name: regcred
---
apiVersion: v1
kind: Service
metadata:
name: examplestack-ex00
labels:
instance: ex00
app: examplestack
namespace: default
spec:
selector:
instance: ex00
ports:
- port: 4443
protocol: TCP
name: optic
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: examplestack-ex00-ingress
labels:
instance: ex00
app: examplestack
spec:
entryPoints:
- websecure
routes:
- match: HostSNI(`ex00.k8s.yourdomain.tld`)
services:
- name: examplestack-ex00
port: 4443
tls:
passthrough: true
Multiple Pods
Each service can also be broken into separate pods. This example is broken down across three sections, a Cortex, an Axon, and other services. This lines up with three distinct Persistent Volume Claims being made to host the data for the services. This isolates the storage between the Cortex, Axon and other services. Each service is deployed into its own pods; and each Telepath-capable service reports itself into an Aha server.
First, the shared Secret.
apiVersion: v1
kind: Secret
metadata:
name: ex01-passwds
labels:
app: demostack
data:
# secretpass
corepass: "c2VjcmV0cGFzcw=="
# ssapterces
svcpass: "c3NhcHRlcmNlcw=="
The Cortex is straightforward. It uses a PVC, it is configured via environment variables, and has its Telepath
port exposed as a service that other Pods can connect to. This example also adds a startupProbe
and
livenessProbe
added to check the Cortex (and other services). This allows us to know when the service is available;
since the Cortex may take some time to load all of the memory maps associated with layer data.
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: examplestack-ex01-cortex
labels:
app: examplestack
instance: ex01
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: do-block-storage
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: examplestack-ex01-cortex
labels:
app: cortex
type: examplestack
instance: ex01
spec:
selector:
matchLabels:
app: cortex
template:
metadata:
labels:
app: cortex
instance: ex01
spec:
volumes:
- name: ex-data
persistentVolumeClaim:
claimName: examplestack-ex01-cortex
containers:
- name: core00
image: vertexproject/synapse-cortex:v2.x.x
command: ["python"]
args: ["-O", "-m", "synapse.servers.cortex", "/vertex/storage"]
env:
- name: SYN_CORTEX_DMON_LISTEN
value: "tcp://0.0.0.0:27492/"
- name: SYN_CORTEX_STORM_LOG
value: "true"
- name: SYN_CORTEX_STORM_LOG_LEVEL
value: INFO
- name: SYN_CORTEX_HTTPS_PORT
value: null
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_CORTEX_AHA_NETWORK
value: aha.demo.net
- name: SYN_CORTEX_AHA_NAME
value: cortex
- name: SYN_CORTEX_AUTH_PASSWD
valueFrom:
secretKeyRef:
name: ex01-passwds
key: corepass
- name: SYN_SVC_PASSWD
valueFrom:
secretKeyRef:
name: ex01-passwds
key: svcpass
- name: SYN_CORTEX_AHA_REGISTRY
value: tcp://root:$(SYN_SVC_PASSWD)@examplestack-ex01-aha:27492/
- name: SYN_CORTEX_AXON
value: aha://root:$(SYN_SVC_PASSWD)@axon.aha.demo.net/
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: core
ports:
- containerPort: 27492
imagePullPolicy: Always
startupProbe:
failureThreshold: 6
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
livenessProbe:
initialDelaySeconds: 20
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
restartPolicy: Always
imagePullSecrets:
- name: regcred
---
apiVersion: v1
kind: Service
metadata:
name: examplestack-ex01-cortex
labels:
instance: ex01
app: cortex
namespace: default
spec:
selector:
app: cortex
instance: ex01
ports:
- port: 27492
protocol: TCP
name: telepath
The Axon is very similar to the Cortex.
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: examplestack-ex01-axon
labels:
app: examplestack
instance: ex01
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: do-block-storage
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: examplestack-ex01-axon
labels:
app: axon
type: examplestack
instance: ex01
spec:
selector:
matchLabels:
app: axon
template:
metadata:
labels:
app: axon
instance: ex01
spec:
volumes:
- name: ex-data
persistentVolumeClaim:
claimName: examplestack-ex01-axon
containers:
- name: axon00
image: vertexproject/synapse-axon:v2.x.x
env:
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_AXON_DMON_LISTEN
value: "tcp://0.0.0.0:27492"
- name: SYN_AXON_HTTPS_PORT
value: null
- name: SYN_SVC_PASSWD
valueFrom:
secretKeyRef:
name: ex01-passwds
key: svcpass
- name: SYN_AXON_AHA_REGISTRY
value: tcp://root:$(SYN_SVC_PASSWD)@examplestack-ex01-aha:27492/
- name: SYN_AXON_AUTH_PASSWD
value: $(SYN_SVC_PASSWD)
- name: SYN_AXON_AHA_NETWORK
value: aha.demo.net
- name: SYN_AXON_AHA_NAME
value: axon
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: axon
imagePullPolicy: Always
startupProbe:
failureThreshold: 6
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
livenessProbe:
initialDelaySeconds: 20
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
restartPolicy: Always
imagePullSecrets:
- name: regcred
---
apiVersion: v1
kind: Service
metadata:
name: examplestack-ex01-axon
labels:
instance: ex01
app: axon
namespace: default
spec:
selector:
app: axon
instance: ex01
ports:
- port: 27492
protocol: TCP
name: telepath
The last set of components shown here is the most complex. It includes the Aha server, the Maxmind connector, and the Optic UI.
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: cert-ex01
labels:
app: examplestack
instance: ex01
spec:
secretName: ex01-staging-tls
duration: 2160h # 90d
renewBefore: 360h # 15d
subject:
organizations:
- vertex project
commonName: ex01.k8s.yourdomain.tld
isCA: false
privateKey:
algorithm: RSA
encoding: PKCS1
size: 2048
usages:
- server auth
- client auth
dnsNames:
- ex01.k8s.yourdomain.tld
issuerRef:
name: le-http-staging-issuer
kind: ClusterIssuer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: examplestack-ex01-svcs
labels:
app: examplestack
instance: ex01
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: do-block-storage
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: examplestack-ex01-aha
labels:
app: aha
type: examplestack
instance: ex01
spec:
selector:
matchLabels:
app: aha
template:
metadata:
labels:
app: aha
instance: ex01
spec:
volumes:
- name: ex-data
persistentVolumeClaim:
claimName: examplestack-ex01-svcs
containers:
- name: aha00
image: vertexproject/synapse-aha:master
env:
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_AHACELL_DMON_LISTEN
value: "tcp://0.0.0.0:27492"
- name: SYN_AHACELL_AUTH_PASSWD
valueFrom:
secretKeyRef:
name: ex01-passwds
key: svcpass
- name: SYN_AHACELL_HTTPS_PORT
value: null
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: aha
imagePullPolicy: Always
startupProbe:
failureThreshold: 6
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
livenessProbe:
initialDelaySeconds: 20
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
restartPolicy: Always
imagePullSecrets:
- name: regcred
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: examplestack-ex01-maxmind
labels:
app: maxmind
type: examplestack
instance: ex01
spec:
selector:
matchLabels:
app: maxmind
template:
metadata:
labels:
app: maxmind
instance: ex01
spec:
volumes:
- name: ex-data
persistentVolumeClaim:
claimName: examplestack-ex01-svcs
containers:
- name: maxmind00
image: vertexproject/synapse-maxmind:v2.x.x
env:
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_MAXMIND_DMON_LISTEN
value: "tcp://0.0.0.0:27492"
- name: SYN_MAXMIND_HTTPS_PORT
value: null
- name: SYN_SVC_PASSWD
valueFrom:
secretKeyRef:
name: ex01-passwds
key: svcpass
- name: SYN_MAXMIND_AHA_REGISTRY
value: tcp://root:$(SYN_SVC_PASSWD)@examplestack-ex01-aha:27492/
- name: SYN_MAXMIND_AUTH_PASSWD
value: $(SYN_SVC_PASSWD)
- name: SYN_MAXMIND_AHA_NETWORK
value: aha.demo.net
- name: SYN_MAXMIND_AHA_NAME
value: maxmind
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: maxmind
imagePullPolicy: Always
startupProbe:
failureThreshold: 6
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
livenessProbe:
initialDelaySeconds: 20
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
restartPolicy: Always
imagePullSecrets:
- name: regcred
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: examplestack-ex01-optic
labels:
app: optic
type: examplestack
instance: ex01
spec:
selector:
matchLabels:
app: optic
template:
metadata:
labels:
app: optic
instance: ex01
spec:
volumes:
- name: ex-data
persistentVolumeClaim:
claimName: examplestack-ex01-svcs
- name: optic-tls
secret:
secretName: ex01-staging-tls
items:
- key: tls.key
path: sslkey.pem
- key: tls.crt
path: sslcert.pem
initContainers:
- name: init-optic00
image: vertexproject/synapse:v2.x.x
imagePullPolicy: Always
command: ['/bin/bash']
args:
- -c
- >-
cp -v --target-directory /vertex/storage /vertex/tls/sslkey.pem /vertex/tls/sslcert.pem
volumeMounts:
- name: optic-tls
mountPath: /vertex/tls
readOnly: true
- name: ex-data
mountPath: /vertex/storage
subPath: optic
containers:
- name: optic00
image: vertexproject/optic:v2.x.x
env:
- name: SYN_LOG_LEVEL
value: DEBUG
- name: SYN_OPTIC_EMAIL_HOST
value: exim
- name: SYN_OPTIC_DMON_LISTEN
value: "null"
- name: SYN_OPTIC_NETLOC
value: "ex01.k8s.yourdomain.tld"
- name: SYN_SVC_PASSWD
valueFrom:
secretKeyRef:
name: ex01-passwds
key: svcpass
- name: SYN_OPTIC_AHA_REGISTRY
value: tcp://root:$(SYN_SVC_PASSWD)@examplestack-ex01-aha:27492/
- name: SYN_OPTIC_AXON
value: aha://root:$(SYN_SVC_PASSWD)@axon.aha.demo.net/
- name: SYN_CORTEX_PASSWD
valueFrom:
secretKeyRef:
name: ex01-passwds
key: corepass
- name: SYN_OPTIC_CORTEX
value: "aha://root:$(SYN_CORTEX_PASSWD)@cortex.aha.demo.net/"
ports:
- containerPort: 4443
volumeMounts:
- mountPath: /vertex/storage
name: ex-data
subPath: optic
imagePullPolicy: Always
startupProbe:
failureThreshold: 6
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
livenessProbe:
initialDelaySeconds: 20
timeoutSeconds: 20
periodSeconds: 20
exec:
command: ['python', '-m', 'synapse.tools.healthcheck', '-c', 'cell:///vertex/storage']
restartPolicy: Always
imagePullSecrets:
- name: regcred
---
apiVersion: v1
kind: Service
metadata:
name: examplestack-ex01-aha
labels:
instance: ex01
app: aha
namespace: default
spec:
selector:
app: aha
instance: ex01
ports:
- port: 27492
protocol: TCP
name: telepath
---
apiVersion: v1
kind: Service
metadata:
name: examplestack-ex01-optic
labels:
instance: ex01
app: cortex
namespace: default
spec:
selector:
app: optic
instance: ex01
ports:
- port: 4443
protocol: TCP
name: telepath
---
apiVersion: v1
kind: Service
metadata:
name: examplestack-ex01-maxmind
labels:
instance: ex01
app: aha
namespace: default
spec:
selector:
app: maxmind
instance: ex01
ports:
- port: 27492
protocol: TCP
name: telepath
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: examplestack-ex01-ingress
labels:
instance: ex01
app: examplestack
spec:
entryPoints:
- websecure
routes:
- match: HostSNI(`ex01.k8s.yourdomain.tld`)
services:
- name: examplestack-ex01-optic
port: 4443
tls:
passthrough: true