How To Deploy a Local Corda Cluster Using Kubernetes

Corda Mar 30 2023 By: Sneha Damle
Comments

0 Comments

Views

2,028 Views

How To Deploy a Local Corda Cluster Using Kubernetes
Sneha Damle
Sneha Damle Senior Developer Evangelist
Share this post:
Copied

In the first blog post, we discussed Next-Gen Corda’s architecture. In the second blog post, we saw how to start a local all-in-one combined cluster with a static network. This article will show how to deploy a local Corda worker cluster using Kubernetes with a static network.

Diagram of a Static Application Deployment with individual workers deployed on Kubernetes
Figure 1: Static Application Deployment with individual workers deployed on Kubernetes

Pre-requisites

  1. Minikube
  2. Kubectl
  3. Helm
  4. Corda-CLI

In this blog post, I will show you how to use Helm. Helm is a Kubernetes deployment tool that lets you automate the creation, packaging, configuration, and deployment of applications and services to Kubernetes clusters.

We will first start a local Kubernetes cluster using Minikube. After that will install Kafka and Postgres using Helm. We will then install Corda workers to the Kubernetes cluster using Helm.

Please note that Kafka and Database are prerequisites for a Corda cluster deployment. This blog will show how these prerequisites can be installed using Helm charts.

Note: There could be many ways to do the deployment mentioned below, depending on your use case, your needs, and your organisation’s policies. For example, for setting up Kubernetes locally, you could use either Docker DesktopMinikubeKindMicroK8s or Rancher Desktop. For remote setup, there are many options like EKS, AKS and GKE. This blog shows you one way to deploy Kubernetes locally, and it should not be treated as a setting stone standard. Similarly, we currently don’t have any specific guidance on the configuration of PostgreSQL and Kafka. Helm charts to set up Kafka and Postgres provided by R3 are only for a development setup.

Step 1: Start a Kubernetes cluster using Minikube

For this blog, I will be executing the below commands on a Mac M1 computer. There are multiple ways to create a Kubernetes cluster. For this blog, I will use Minikube to start a local kubernetes cluster. Run the below command and this will start a Kubernetes cluster running on your system.


minikube start --cpus=6 --memory=8G

Kubernetes local cluster has started.

Step 2: Install Postgres, Kafka from a packaged Helm chart

Corda Helm charts are pushed as an OCI bundle to a docker registry. To install Kafka and Postgres is to install a packaged Helm chart as an OCI bundle pushed onto a docker registry, as shown below.


helm install prereqs -n corda --create-namespace oci://registry-1.docker.io/corda/corda-dev-prereqs --timeout 10m --wait

Pulled: registry-1.docker.io/corda/corda-dev-prereqs:0.2.0
Digest: sha256:577cbc404efa224e9cc8b5c4b6ca09574b515639d36a08a39f30290f5f888ba6
NAME: prereqs
LAST DEPLOYED: Thu Feb 16 14:19:05 2023
NAMESPACE: corda
STATUS: deployed
REVISION: 1
NOTES:
Kafka and PostgreSQL have been installed.

**WARNING** These are not intended for production use.

The following overrides should be used when installing a Corda Helm Chart into the same namespace:

bootstrap:
  kafka:
    replicas: 1
db:
  cluster:
    host: "prereqs-postgres"
    username:
      value: "corda"
    password:
      valueFrom:
        secretKeyRef:
          name: "prereqs-postgres"
          key: "corda-password"
kafka:
  bootstrapServers: "prereqs-kafka:9092"
  sasl:
    enabled: true
    mechanism: "PLAIN"
    username:
      value: "admin"
    password:
      valueFrom:
        secretKeyRef:
          name: "prereqs-kafka"
          key: "admin-password"
  tls:
    enabled: true
    truststore:
      valueFrom:
        secretKeyRef:
          name: "prereqs-kafka"
          key: "ca.crt"

To access PostgreSQL from outside the Kubernetes cluster:

kubectl port-forward -n corda svc/prereqs-postgres 5432 &
export PGUSER=corda
export PGPASSWORD=$(kubectl get secret -n corda prereqs-postgres -o go-template='{{ index .data "corda-password" | base64decode }}')
psql -h localhost -p 5432 -d cordacluster

To access Kafka from outside the Kubernetes cluster:

export KAFKA_PASSWORD=$(kubectl get secret -n corda prereqs-kafka -o go-template='{{ index .data "admin-password" | base64decode }}')
echo "security.protocol=SASL_PLAINTEXT" > client.properties
echo "sasl.mechanism=PLAIN" >> client.properties
echo "sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"$KAFKA_PASSWORD\";" >> client.properties
kubectl port-forward -n corda $(kubectl get pods -n corda --selector=app.kubernetes.io/component=kafka,app.kubernetes.io/instance=prereqs -o=name) 9094 &
kafka-topics --list --bootstrap-server localhost:9094 --command-config client.properties

This will install Kafka and Postgres into the Corda Kubernetes namespace. The --wait option ensures all the pods are ready before returning.The timeout is set to 10 minutes to allow time to pull the images.

Note: Here Kafka is a single broker and should be used only for development purposes.

To know more about the details, Helm charts are pushed at the corda-helm-prereq repo.

Step 3: Port Forward for Postgres 

Use the output from step 2 and expose the cluster’s Postgres to the outside world by executing the below command.


kubectl port-forward -n corda svc/prereqs-postgres 5432 &

You should also set the default namespace to Corda by executing below command.


kubectl config set-context --current --namespace=corda

Step 4: Install Corda using packaged Helm chart and pre-built docker images

It is possible to deploy a pre-built version of the Helm chart and the associated Docker images from the Artifactory.


helm install corda oci://registry-1.docker.io/corda/corda --version 5.0.0-Gecko1.0 --namespace corda --values https://gist.githubusercontent.com/snedamle/3fbadeb35f11ad13fecb204b099655cd/raw/94fc09db43a9048497157c3778866fd488090a3f/corda-dev-prereqs.yaml --wait

Pulled: registry-1.docker.io/corda/corda:5.0.0-Gecko1.0
Digest: sha256:c8e5aa99c47fa3aab94e93bb46e77d8c4d72b67bd9b15432cc423673c61b13f2
NAME: corda
LAST DEPLOYED: Fri Mar 24 13:31:53 2023
NAMESPACE: corda
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Extract the credentials for the initial admin user:
kubectl get secret corda-initial-admin-user -o go-template='{{ .data.username | base64decode }}'
kubectl get secret corda-initial-admin-user -o go-template='{{ .data.password | base64decode }}'

2. Expose the API endpoint on localhost by running this command:
kubectl port-forward --namespace corda deployment/corda-rest-worker 8888 &

3. The API endpoint definition can then be accessed via: https://localhost:8888/api/v1/swagger

The corda-dev-prereqs-yaml file lets you override the default Corda cluster configuration, e.g. the number of replicas. The above command will install Corda worker jars from the docker images and tells the chart where to find the Kafka and PostgreSQL instances created in the previous step. 

Note: I have set up the database using the Helm chart. You could manually set up the database as well. You can manually generate the SQL scripts using the Corda-Cli tool.

The instructions on the GitHub wiki talk about how to build the Corda Docker images and then install Corda using the Helm chart from the source.

Step 5: Kubernetes Cluster Port forward and verify if the cluster is up

Follow the instructions, which are the output of the previous step, to verify if the cluster is running. Forward the port by executing the below command and hit https://localhost:8888/api/v1/swagger#/ to confirm if the cluster is up.


kubectl port-forward --namespace corda deployment/corda-rest-worker 8888 &

Once the cluster is up, we can use CSDE’s quickDeploy task to deploy our CorDapp. If you want to read more about what the quickDeploy task does, read the previous blog. Here, we will not run the startCorda task, as we already have a cluster up and running.

Step 6: Retrieve the Corda cluster RPC username and password

Get the Corda Rpc username and password from the output of the previous step. Update the cordaRpcUser and cordaRpcPasswd in CSDE’s gradle.properties to the values retrieved by running the below commands.


kubectl get secret corda-initial-admin-user -o go-template='{{ .data.username | base64decode }}' --namespace corda
kubectl get secret corda-initial-admin-user -o go-template='{{ .data.password | base64decode }}' --namespace corda

Update the RPC username and password in gradle.properties

Step 7: Run CSDE’s quickDeploy gradle task

quickDeploy task will internally create a .CPI file, and upload the .CPI file to the cluster, create and register virtual nodes.

Step 8: View worker logs

You can retrieve the list of worker pods by running the below command.


kubectl get pods -n corda

To retrieve logs for specific worker pods, retrieve the name of the worker from output of the above list and hit the below command.


kubectl logs -f -n corda corda-rest-worker-68f4db8b4-lbfdv

Step 9: Use the HTTP RPC API to hit a flow

You can look at these steps, which discuss triggering a flow.

Step 10: Cleanup

You can use the below commands to clean up the Corda cluster.


kubectl delete namespace corda

In the next blog post, we will talk about dynamic network deployments.

Sneha Damle
Sneha Damle Sneha Damle is a Developer Evangelist at R3, an enterprise blockchain software firm working with a global ecosystem of more than 350 participants across multiple industries from both the private and public sectors to develop on Corda, its open-source blockchain platform.

Leave a Reply

Subscribe to our newsletter to stay up to date on the latest developer news, tools, and articles.