使用 Flomesh 进行 Dubbo 服务治理
写在最前
和上一篇一样，这次同样是在无代码侵入的情况下对 Dubbo 服务治理的提升。
更多治理场景陆续添加中，有兴趣的可关注 https://github.com/flomesh-io/service-mesh-dubbo-demo。
开源的 Pipy 作为 Flomesh 的核心，得益于其轻量及灵活性可以通过编程的方式轻松快速的支持多中平台的服务发现机制，比如 Eureka、Consul、Nacos 等。
概览
细节
环境搭建
搭建 Kubernetes 环境，可以选择 kubeadm 进行集群搭建。也可以选择 minikube、k3s、Kind 等，本文使用 k3s。
使用 k3d[1] 安装 k3s[2]。k3d 将在 Docker 容器中运行 k3s，因此需要保证已经安装了 Docker。
$ k3d cluster create dubbo-demo -p "80:80@loadbalancer" --k3s-server-arg '--no-deploy=traefik'安装 Flomesh
从仓库 https://github.com/flomesh-io/service-mesh-dubbo-demo 克隆代码。进入到 release目录。
所有 Flomesh 组件以及用于 demo 的 yamls 文件都位于这个目录中。
$ kubectl apply -f artifacts/cert-manager-v1.3.1.yamlcustomresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io createdcustomresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io createdcustomresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io createdcustomresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io createdcustomresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io createdcustomresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io creatednamespace/cert-manager createdserviceaccount/cert-manager-cainjector createdserviceaccount/cert-manager createdserviceaccount/cert-manager-webhook createdclusterrole.rbac.authorization.k8s.io/cert-manager-cainjector createdclusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers createdclusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers createdclusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates createdclusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders createdclusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges createdclusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim createdclusterrole.rbac.authorization.k8s.io/cert-manager-view createdclusterrole.rbac.authorization.k8s.io/cert-manager-edit createdclusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io createdclusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io createdclusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews createdrole.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection createdrole.rbac.authorization.k8s.io/cert-manager:leaderelection createdrole.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving createdrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection createdrolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection createdrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving createdservice/cert-manager createdservice/cert-manager-webhook createddeployment.apps/cert-manager-cainjector createddeployment.apps/cert-manager createddeployment.apps/cert-manager-webhook createdmutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook createdvalidatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
注意: 要保证 cert-manager 命名空间中所有的 pod 都正常运行：
$ kubectl get pod -n cert-managerNAME READY STATUS RESTARTS AGEcert-manager-cainjector-59f76f7fff-ggmdm 1/1 Running 0 32scert-manager-59f6c76f4b-r2h5r 1/1 Running 0 32scert-manager-webhook-56fdcbb848-sdnxb 1/1 Running 0 32s
安装 Pipy Operator
$ kubectl apply -f artifacts/pipy-operator.yaml执行完命令后会看到类似的结果：
namespace/flomesh createdcustomresourcedefinition.apiextensions.k8s.io/proxies.flomesh.io createdcustomresourcedefinition.apiextensions.k8s.io/proxyprofiles.flomesh.io createdserviceaccount/operator-manager createdrole.rbac.authorization.k8s.io/leader-election-role createdclusterrole.rbac.authorization.k8s.io/manager-role createdclusterrole.rbac.authorization.k8s.io/metrics-reader createdclusterrole.rbac.authorization.k8s.io/proxy-role createdrolebinding.rbac.authorization.k8s.io/leader-election-rolebinding createdclusterrolebinding.rbac.authorization.k8s.io/manager-rolebinding createdclusterrolebinding.rbac.authorization.k8s.io/proxy-rolebinding createdconfigmap/manager-config createdservice/operator-manager-metrics-service createdservice/proxy-injector-svc createdservice/webhook-service createddeployment.apps/operator-manager createddeployment.apps/proxy-injector createdcertificate.cert-manager.io/serving-cert createdissuer.cert-manager.io/selfsigned-issuer createdmutatingwebhookconfiguration.admissionregistration.k8s.io/mutating-webhook-configuration createdmutatingwebhookconfiguration.admissionregistration.k8s.io/proxy-injector-webhook-cfg createdvalidatingwebhookconfiguration.admissionregistration.k8s.io/validating-webhook-configuration created
注意：要保证 flomesh 命名空间中所有的 pod 都正常运行：
$ kubectl get pod -n flomeshNAME READY STATUS RESTARTS AGEproxy-injector-6d5c774bc-rspmc 1/1 Running 0 21soperator-manager-c95cd449-xxc77 0/1 Running 0 38s
安装 Ingress 控制器：ingress-pipy
$ kubectl apply -f artifacts/ingress-pipy.yamlnamespace/ingress-pipy createdcustomresourcedefinition.apiextensions.k8s.io/ingressglobalhooks.flomesh.io createdcustomresourcedefinition.apiextensions.k8s.io/ingressrules.flomesh.io createdserviceaccount/ingress-pipy createdrole.rbac.authorization.k8s.io/ingress-pipy-leader-election-role createdclusterrole.rbac.authorization.k8s.io/ingress-pipy-role createdrolebinding.rbac.authorization.k8s.io/ingress-pipy-leader-election-rolebinding createdclusterrolebinding.rbac.authorization.k8s.io/ingress-pipy-rolebinding createdconfigmap/ingress-config createdservice/ingress-pipy-cfg createdservice/ingress-pipy-controller createdservice/ingress-pipy-defaultbackend createdservice/webhook-service createddeployment.apps/ingress-pipy-cfg createddeployment.apps/ingress-pipy-controller createddeployment.apps/ingress-pipy-manager createdcertificate.cert-manager.io/serving-cert createdissuer.cert-manager.io/selfsigned-issuer createdmutatingwebhookconfiguration.admissionregistration.k8s.io/mutating-webhook-configuration configuredvalidatingwebhookconfiguration.admissionregistration.k8s.io/validating-webhook-configuration configured
检查 ingress-pipy 命名空间下 pod 的状态：
$ kubectl get pod -n ingress-pipyNAME READY STATUS RESTARTS AGEsvclb-ingress-pipy-controller-qwbk9 1/1 Running 0 90singress-pipy-cfg-6c54d5b9b6-6s7lz 1/1 Running 0 90singress-pipy-manager-7988dfbf4f-lxr4b 1/1 Running 0 90singress-pipy-controller-9d4698887-zrpfd 1/1 Running 0 90s
至此，你已经成功安装 Flomesh 的所有组件，包括 operator 和 ingress 控制器。
运行 Demo
创建命名空间
Demo 运行在另一个独立的命名空间 flomesh-dubbo 中，执行命令 kubectl apply -f dubbo-mesh/templates/namespace.yaml 来创建该命名空间。如果你 describe 该命名空间你会发现其使用了 flomesh.io/inject=true 标签。
这个标签告知 operator 的 admission webHook 拦截标注的命名空间下 pod 的创建。
$ kubectl describe ns flomesh-dubboName: flomesh-dubboLabels: app.kubernetes.io/managed-by=Helmapp.kubernetes.io/name=dubbo-meshapp.kubernetes.io/version=1.19.0flomesh.io/inject=truehelm.sh/chart=dubbo-mesh-0.1.0kubernetes.io/metadata.name=flomesh-dubboAnnotations: <none>Status: ActiveNo resource quota.No LimitRange resource.
创建 ProxyProfile 资源
$ kubectl apply -f artifacts/proxy-profile.yamlproxyprofile.flomesh.io/poc-pf-dubbo createdproxyprofile.flomesh.io/poc-pf-http created
创建 mock 服务
$ kubectl apply -f dubbo-mesh/templates/configmap-mock.yaml$ kubectl apply -f dubbo-mesh/templates/configmap-proxychains.yaml$ kubectl apply -f dubbo-mesh/templates/deployment-mock.yaml$ kubectl apply -f dubbo-mesh/templates/service-mock.yaml
部署服务
$ kubectl apply -f artifacts/deployment.yaml测试
准备
//Obtain the controller IP//Here, we append port.ingressAddr=`kubectl get svc ingress-pipy-controller -n ingress-pipy -o jsonpath='{.spec.clusterIP}'`:80
这里我们使用了是 k3d 创建的 k3s，命令中加入了 -p 80:80@loadbalancer 选项。我们可以使用 127.0.0.1:80 来访问 ingress 控制器。这里执行命令 ingressAddr=127.0.0.1:80。
Ingress 规则中，我们为每个规则指定了 host，因此每个请求中需要通过 HTTP 请求头 Host 提供对应的 host。
或者在 /etc/hosts 添加记录：
$ kubectl get ing ingress-canary-router -n flomesh-dubbo -o jsonpath="{range .spec.rules[*]}{.host}{'\n'}"dubbo.demo.flomesh.cn//添加记录到 /etc/hosts127.0.0.1 dubbo.demo.flomesh.cn
灰度
v1、v2 服务只能访问对应版本的服务。
$ curl --location --request POST 'http://127.0.0.1:80/hello' \--header 'Host: dubbo.demo.flomesh.cn' \--header 'x-canary-version: v1' \--header 'Content-Type: application/json' \--data-raw '{"name":"Flomesh"}'V1-[hello-service] : Hello, Flomesh, Today is (2021-08-17), Time is (04:06:56.823)$ curl --location --request POST 'http://127.0.0.1:80/hello' \--header 'Host: dubbo.demo.flomesh.cn' \--header 'x-canary-version: v2' \--header 'Content-Type: application/json' \--data-raw '{"name":"Flomesh"}'V2-[hello-service] : Hello, Flomesh, Today is (Tue, 2021-Aug-17), Time is (04:06:37 +0000)
引用链接
[1] k3d: https://k3d.io/[2] k3s: https://github.com/k3s-io/k3s