vlambda博客
学习文章列表

树莓派k8s集群安装mysql及监控

安装准备

  • 树莓派k8s集群

root@pi4-master01:~# kubectl get nodes -o wideNAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIMEpi4-master01 Ready master 4d18h v1.15.10 192.168.5.18 <none> Ubuntu 20.04 LTS 5.4.0-1011-raspi docker://18.9.9pi4-node01 Ready node 4d17h v1.15.10 192.168.5.19 <none> Ubuntu 20.04 LTS 5.4.0-1011-raspi docker://18.9.9pi4-node02 Ready node 4d17h v1.15.10 192.168.5.20 <none> Ubuntu 20.04 LTS 5.4.0-1011-raspi docker://18.9.9
  • 树莓派k8s集群已安装helm和nginx-ingress

root@pi4-master01:~/k8s/cluster-monitoring-0.37.0# helm versionClient: &version.Version{SemVer:"v2.15.0", GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750", GitTreeState:"clean"}Server: &version.Version{SemVer:"v2.15.0+unreleased", GitCommit:"9668ad4d90c5e95bd520e58e7387607be6b63bb6", GitTreeState:"dirty"}root@pi4-master01:~/k8s/cluster-monitoring-0.37.0# helm listNAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACEnginx-ingress 1 Fri Jul 3 17:11:20 2020 DEPLOYED nginx-ingress-0.9.5 0.10.2 default root@pi4-master01:~/k8s/cluster-monitoring-0.37.0# kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d21hnginx-ingress-controller NodePort 10.110.89.242 192.168.5.18 80:12001/TCP,443:12002/TCP 4d16hnginx-ingress-default-backend ClusterIP 10.104.65.1 <none> 80/TCP 4d16h
  • 树莓派k8s集群已安装存储类,并设置为默认存储

root@pi4-master01:~/k8s/cluster-monitoring-0.37.0# kubectl get storageclassNAME PROVISIONER AGElocal-path (default) rancher.io/local-path 4d15h

对了,如果你还没有树莓派k8s集群,这里有一篇树莓派 k8s 集群入坑指南(点击阅读原文直达链接)可以参考,欢迎入坑。

helm安装mysql

root@pi4-master01:~# helm repo remove stable"stable" has been removed from your repositoriesroot@pi4-master01:~# helm repo add stable http://mirror.azure.cn/kubernetes/charts/"stable" has been added to your repositoriesroot@pi4-master01:~# helm repo listNAME URLlocal http://127.0.0.1:8879/chartsstable http://mirror.azure.cn/kubernetes/charts/root@pi4-master01:~# helm search mysqlNAME CHART VERSION APP VERSION DESCRIPTIONstable/mysql 1.6.6 5.7.30 Fast, reliable, scalable, and easy to use open-source rel...stable/mysqldump 2.6.0 2.4.1 A Helm chart to help backup MySQL databases using mysqldump stable/prometheus-mysql-exporter 0.7.0 v0.11.0 A Helm chart for prometheus mysql exporter with cloudsqlp...
  • 使用helm安装mysql命令如下

helm install -n mysql \stable/mysql \--set image=biarms/mysql \--set imageTag=5.7.30-linux-arm64v8 \--set busybox.images=arm64v8/busybox \--set mysqlRootPassword=abc123

该chart中涉及的镜像均是x86的,只要把相关的镜像换成arm64v8架构的镜像即可。另外,该chart默认开启了持久化存储,不指定storageclass的话,将使用k8s的默认存储。因为我们的树莓派k8s集群已经设置了local-path为默认storageclass,所以不需要关闭持久化存储既可正常安装。

  • 查看创建的pod

root@pi4-master01:~# kubectl get pod | grep mysqlmysql-56b767d54b-tfrng 1/1 Running 0 89m
  • 查看创建的svc

root@pi4-master01:~# kubectl get svc | grep mysqlmysql ClusterIP 10.108.134.203 <none> 3306/TCP 90m
  • 查看创建的pv,pvc

root@pi4-master01:~# kubectl get pvc,pvNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEpersistentvolumeclaim/mysql Bound pvc-7e596f23-af80-480f-8713-fe3dc9e42157 8Gi RWO local-path 91m
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGEpersistentvolume/pvc-7e596f23-af80-480f-8713-fe3dc9e42157 8Gi RWO Delete Bound default/mysql local-path

验证

  • 进入mysql容器

root@pi4-master01:~# kubectl exec -it mysql-56b767d54b-tfrng bashroot@mysql-56b767d54b-tfrng:/#
  • 连接mysql服务

root@mysql-56b767d54b-tfrng:/# mysql -uroot -pabc123mysql: [Warning] Using a password on the command line interface can be insecure.Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 1128Server version: 5.7.30-0ubuntu0.18.04.1 (Ubuntu)
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> status--------------mysql Ver 14.14 Distrib 5.7.30, for Linux (aarch64) using EditLine wrapper
Connection id: 1128Current database: Current user: root@localhostSSL: Not in useCurrent pager: stdoutUsing outfile: ''Using delimiter: ;Server version: 5.7.30-0ubuntu0.18.04.1 (Ubuntu)Protocol version: 10Connection: Localhost via UNIX socketServer characterset: latin1Db characterset: latin1Client characterset: latin1Conn. characterset: latin1UNIX socket: /var/run/mysqld/mysqld.sockUptime: 1 hour 33 min 54 sec
Threads: 1 Questions: 2259 Slow queries: 0 Opens: 113 Flush tables: 1 Open tables: 0 Queries per second avg: 0.400--------------
mysql>

基于helm在树莓派k8s集群安装mysql还是很方便的,只需要使用arm64v8架构的镜像即可。但是如果想更部署的mysql更具有个性,还是很有必要研究一下该chart的其它配置项,这些配置项我们可以通过helm inspect命令来查看

root@pi4-master01:~# helm inspect values stable/mysql## mysql image version## ref: https://hub.docker.com/r/library/mysql/tags/##image: "mysql"imageTag: "5.7.30"
strategy: type: Recreate
busybox: image: "busybox" tag: "1.32"
testFramework: enabled: true image: "bats/bats" tag: "1.2.1" imagePullPolicy: IfNotPresent securityContext: {}
## Specify password for root user#### Default: random 10 character string# mysqlRootPassword: testing
## Create a database user### mysqlUser:## Default: random 10 character string# mysqlPassword:
## Allow unauthenticated access, uncomment to enable### mysqlAllowEmptyPassword: true
## Create a database### mysqlDatabase:
## Specify an imagePullPolicy (Required)## It's recommended to change this to 'Always' if the image tag is 'latest'## ref: http://kubernetes.io/docs/user-guide/images/#updating-images##imagePullPolicy: IfNotPresent
## Additionnal arguments that are passed to the MySQL container.## For example use --default-authentication-plugin=mysql_native_password if older clients need to## connect to a MySQL 8 instance.args: []
extraVolumes: | # - name: extras # emptyDir: {}
extraVolumeMounts: | # - name: extras # mountPath: /usr/share/extras # readOnly: true
extraInitContainers: | # - name: do-something # image: busybox # command: ['do', 'something']
## A string to add extra environment variables# extraEnvVars: |# - name: EXTRA_VAR# value: "extra"
# Optionally specify an array of imagePullSecrets.# Secrets must be manually created in the namespace.# ref: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod# imagePullSecrets: # - name: myRegistryKeySecretName
## Node selector## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselectornodeSelector: {}
## Affinity## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinityaffinity: {}
## Tolerations for pod assignment## Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/##tolerations: []
livenessProbe: initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 5 successThreshold: 1 failureThreshold: 3
readinessProbe: initialDelaySeconds: 5 periodSeconds: 10 timeoutSeconds: 1 successThreshold: 1 failureThreshold: 3
## Persist data to a persistent volumepersistence: enabled: true ## database data Persistent Volume Storage Class ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, standard on ## GKE, AWS & OpenStack) ## # storageClass: "-" accessMode: ReadWriteOnce size: 8Gi annotations: {}
## Use an alternate scheduler, e.g. "stork".## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/### schedulerName:
## Security contextsecurityContext: enabled: false runAsUser: 999 fsGroup: 999
## Configure resource requests and limits## ref: http://kubernetes.io/docs/user-guide/compute-resources/##resources: requests: memory: 256Mi cpu: 100m
# Custom mysql configuration files pathconfigurationFilesPath: /etc/mysql/conf.d/
# Custom mysql configuration files used to override default mysql settingsconfigurationFiles: {}# mysql.cnf: |-# [mysqld]# skip-name-resolve# ssl-ca=/ssl/ca.pem# ssl-cert=/ssl/server-cert.pem# ssl-key=/ssl/server-key.pem
# Custom mysql init SQL files used to initialize the databaseinitializationFiles: {}# first-db.sql: |-# CREATE DATABASE IF NOT EXISTS first DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;# second-db.sql: |-# CREATE DATABASE IF NOT EXISTS second DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
metrics: enabled: false image: prom/mysqld-exporter imageTag: v0.10.0 imagePullPolicy: IfNotPresent resources: {} annotations: {} # prometheus.io/scrape: "true" # prometheus.io/port: "9104" livenessProbe: initialDelaySeconds: 15 timeoutSeconds: 5 readinessProbe: initialDelaySeconds: 5 timeoutSeconds: 1 flags: [] serviceMonitor: enabled: false additionalLabels: {}
## Configure the service## ref: http://kubernetes.io/docs/user-guide/services/service: annotations: {} ## Specify a service type ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types type: ClusterIP port: 3306 # nodePort: 32000 # loadBalancerIP:
## Pods Service Account## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/serviceAccount: ## Specifies whether a ServiceAccount should be created ## create: false ## The name of the ServiceAccount to use. ## If not set and create is true, a name is generated using the mariadb.fullname template # name:
ssl: enabled: false secret: mysql-ssl-certs certificates:# - name: mysql-ssl-certs# ca: |-# -----BEGIN CERTIFICATE-----# ...# -----END CERTIFICATE-----# cert: |-# -----BEGIN CERTIFICATE-----# ...# -----END CERTIFICATE-----# key: |-# -----BEGIN RSA PRIVATE KEY-----# ...# -----END RSA PRIVATE KEY-----
## Populates the 'TZ' system timezone environment variable## ref: https://dev.mysql.com/doc/refman/5.7/en/time-zone-support.html#### Default: nil (mysql will use image's default timezone, normally UTC)## Example: 'Australia/Sydney'# timezone:
# Deployment AnnotationsdeploymentAnnotations: {}
# To be added to the database server pod(s)podAnnotations: {}podLabels: {}
## Set pod priorityClassName# priorityClassName: {}

## Init container resources defaultsinitContainer: resources: requests: memory: 10Mi cpu: 10m

显然,配置项很丰富,从mysql的配置文件、mysql启动pod的时区、是否开启监控都能支持。

开启监控

开启监控前,需要在集群部署prometheus,否则不会有效果。可以参考进行部署。

开启监控,需要设置metrics.enabled和metrics.serviceMonitor.enabled为true,且metrics.image需要设置为支持arm64v8架构的镜像监控才能使用,这里选择的镜像是pi4k8s/prom-mysqld-exporter:v0.12.1,具体的Dockerfile参考如下

FROM arm64v8/ubuntu:18.04
RUN sed -i "s/ports.ubuntu.com/mirrors.aliyun.com/g" /etc/apt/sources.list \ && cd opt && apt update && apt upgrade -y \ && apt -y install wget \ && wget -q "https://github.com/prometheus/mysqld_exporter/releases/download/v0.12.1/mysqld_exporter-0.12.1.linux-arm64.tar.gz" \ && tar -zxf mysqld_exporter-0.12.1.linux-arm64.tar.gz \ && rm -rf mysqld_exporter-0.12.1.linux-arm64.tar.gz \ && apt remove -y --purge wget && apt clean \ && rm -rf /tmp/* /var/cache/* /usr/share/doc/* /usr/share/man/* /var/lib/apt/lists/* \ && mv /opt/mysqld_exporter-0.12.1.linux-arm64/mysqld_exporter /bin/mysqld_exporter
USER nobodyEXPOSE 9104ENTRYPOINT [ "/bin/mysqld_exporter" ]
# arm64v8# docker build . -t pi4k8s/prom-mysqld-exporter:v0.12.1

使用helm安装开启监控的mysql命令如下

helm install -n mysql \stable/mysql \--set image=biarms/mysql \--set imageTag=5.7.30-linux-arm64v8 \--set busybox.image=arm64v8/busybox \--set mysqlRootPassword=abc123 \--set metrics.enabled=true \--set metrics.image=pi4k8s/prom-mysqld-exporter \--set metrics.imageTag=v0.12.1 \--set metrics.serviceMonitor.enabled=true

确认安装状态

root@pi4-master01:~# kubectl get po|grep mysqlmysql-7b74677bf4-shchz 2/2 Running 0 11hroot@pi4-master01:~# kubectl get svc|grep mysqlmysql ClusterIP 10.107.31.127 <none> 3306/TCP,9104/TCP 11hroot@pi4-master01:~# kubectl get servicemonitor|grep mysqlmysql 11h在prometheus查看exporter是否生效


生效后在grafana系统配置mysql监控面板

  • mysql监控面板 :https://grafana.com/grafana/dashboards/7362

部署步骤如下:
树莓派k8s集群安装mysql及监控
树莓派k8s集群安装mysql及监控
点"Upload .json file"按钮,选择已经下载好的mysql监控面板文件 ,进入面试设置界面如下

设置好相应参数后,点"Import"按钮,继续导入,引导入mysql监控页面如下

总结

本实践简单介绍了如何使用chart在树莓派k8s集群用helm部署mysql并配置对于mysql的监控,读者可以基于自己的需要自行调整mysql参数以便适用更多场景。

参考:

  • 树莓派 k8s 集群入坑指南(点击阅读原文直达链接)