Deploy and run Canal cluster in K8s

相关背景

This article is based on the latest source code of Canal, optimized for business compatibility, then compiled, packaged, and deployed to K8s. Since the creation of Canal’s binlog account, MySQL database deployment, and related package changes have already been discussed in the previous article, such content will not be covered in the following text.

Due to the service mode using RocketMQ, invalid compilation and packaging are avoided. Comment out the sections related to the client-adapter and example modules in the pom.xml file under the directory.

In the Canal source code (under the root directory), the script files (*.sh) in the docker folder are used to compile and package the source code into a container, as well as to run the container. The purposes of the related files are described as follows:

• build.sh: Packages the source code into two container images, canal-admin and canal-deployer. Dockerfile: Used to define and build the canal-deployer Docker image Dockerfile_admin: Used to define and build the canal-admin Docker image
• image/app.sh: Used for starting and stopping the canal-deployer application within the container.
• image/app_admin.sh: Used for starting and stopping the canal-admin application within the container.

Other documents will not be explained for the time being. Now, we will discuss the two parts, Canal Admin and Canal Deployer, on how to deploy and run K8s.

Canal Admin

Change the running script

Due to the need to adapt to the existing CI/CD pipeline, the build.sh script needs to be transformed into pipeline steps, as well as adjust the corresponding script file: image/app_admin.sh.

转化 build.sh 为流水线步骤

The content related to Admin in build.sh is as follows:

rm -rf $BASE/canal.*.tar.gz ;
cd $BASE/../ && mvn clean package -Dmaven.test.skip -Denv=release && cd $current_path ;
cp $BASE/../target/canal.admin-*.tar.gz $BASE/
docker build --no-cache -t canal-admin:xxx $BASE/ -f $BASE/Dockerfile_admin

The variable $BASE is the directory path of build.sh, which is /path/canal/docker. The steps to implement the functionality of this file are as follows:

1. Delete all files with the name format canal.*.tar.gz under this directory.
2. Go back to the /path/canal directory and use Maven to clean and package the Canal source code. The final compressed file is generated in the /path/canal/target directory. 3.Copy the files with the name format canal.admin-*.tar.gz from the /path/canal/target directory to the /path/canal/docker directory. According to the definition in the /path/canal/docker/Dockerfile_admin file, package the relevant files under the /path/canal/docker directory into a container image. According to the definition in the /path/canal/docker/Dockerfile_admin file, package the relevant files under the /path/canal/docker directory into a container image.

Change image/app_admin.sh

Due to the use of an independent MySQL database, MySQL in the container is no longer in use, therefore, it is necessary to remove the content related to MySQL from this file. At the same time, when using an independent database, it is necessary to replace the application.yml in the K8s ConfigMap configuration under the original conf directory, that is, add the following content in the start_admin() function:

/bin/rm -f /home/admin/canal-admin/conf/application.yml
cp -f /appconfig/application.yml /home/admin/canal-admin/conf/application.yml

The above code first deletes the original default configuration, then replaces it with the application.yml from the K8s ConfigMap configuration.

In which, appconfig is the mount directory for the K8s ConfigMap. Beforehand, it is necessary to create a new configuration with the key application.yml on K8s. When the container starts, it will generate the application.yml file in the appconfig directory.

If not handled this way, K8s deployment canal-admin data is not saved to canal_manager。

The start_admin() function of image/app_admin.sh finally looks like this:

function start_admin() {
    /bin/rm -f /home/admin/canal-admin/conf/application.yml
    cp -f /appconfig/application.yml /home/admin/canal-admin/conf/application.yml
    echo "start admin ..."
    serverPort=`perl -le 'print $ENV{"server.port"}'`
    if [ -z "$serverPort" ] ; then
        serverPort=8080
    fi
    su admin -c 'cd /home/admin/canal-admin/bin/ && sh restart.sh 1>>/tmp/start.log 2>&1'
    sleep 5
    #check start
    checkStart "canal" "nc 127.0.0.1 $serverPort -w 1 -zv 2>/tmp/nc.out && cat /tmp/nc.out | grep -c Connected" 30
}

Source Code Compilation and Packaging

Before using the pipeline for packaging, we can perform a pre-validation in the relevant environment using the following command:

mvn clean package -Dmaven.test.skip -Denv=release

The files packaged in the Linux environment are located in the /path/canal/target directory, with file names formatted as canal.admin-*.tar.gz and canal.deployer-*.tar.gz。

Build container images

Adjust Docker proxy address

Due to domestic policy reasons in China, access to hub.docker.com to obtain images is not possible. Before building container images, it is necessary to update the available domestic mirror proxies. The following are the proxy addresses available as of now:

{
  "registry-mirrors": [
    "https://docker.chenby.cn",
    "https://hub-mirror.c.163.com",
    "https://registry.docker-cn.com",
    "https://mirror.ccs.tencentyun.com",
    "https://hub-mirror.c.163.com"
  ]
}

Adjust Dockerfile_admin

Since the canal-admin container does not require MySQL, there is no need to package it into the container image. Only the base image in Dockerfile_admin needs to be changed to the base image used in the (canal-deployer)’s Dockerfile, that is:

Due to domestic policy reasons in China, access to hub.docker.com to obtain images is not possible. Before building container images, it is necessary to update the available domestic mirror proxies. The following are the proxy addresses available as of now:

##删除原 FROM canal/osadmin:v3-amd64，替换为
FROM canal/osbase:v3-amd64
…

Building image

Execute build command:

docker build --no-cache -t canal-admin:xxx /path/canal/docker/ -f /path/canal/docker/Dockerfile_admin

After the build is complete, upload the package to Harbor, then delete the local image to free up space.

Deploy to K8s

Due to the canal-admin-service.yaml and canal-admin-deployment.yaml, which vary depending on the environment, they will not be elaborated here. Only the partial configuration of canal-admin-configmap.yaml is provided:

apiVersion: v1
data:
  application.yml: |-
    server:
      port: 8080
    spring:
      jackson:
        date-format: yyyy-MM-dd HH:mm:ss
        time-zone: GMT+8

    spring.datasource:
      address: ipaddress:3306
      database: canal_manager
      username: canal_user
      password: canal_password
      driver-class-name: com.mysql.jdbc.Driver
      url: jdbc:mysql://${spring.datasource.address}/${spring.datasource.database}?useUnicode=true&characterEncoding=UTF-8&useSSL=false
      hikari:
        maximum-pool-size: 30
        minimum-idle: 1

    canal:
      adminUser: admin
      adminPasswd: 123456
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","data":{"application.properties":"\nserver.port=8080"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"canal-admin-configmap","namespace":"canal"}}
  creationTimestamp: '2024-08-15T08:38:31Z'
  name: canal-admin-configmap
  namespace: canal
  resourceVersion: '2291952292'
  uid: 37ec96ff-5187-4585-a7ad-1dfce51cf7d2

In the above configuration, server.port=8080 is because K8s sets the port for canal-admin-service to 8080, consistency is necessary, otherwise access will not be possible.

Canal Deployer

(To be continued)