7.8 实现服务A/B测试

上一章

7.10 灰度发布与A/B测试结合

下一章

更多图书

7.9 实现服务灰度发布

灰度发布又叫金丝雀发布,通过Istio提供的服务路由功能可以轻松实现灰度发布。当我们的服务需要上线新版本时,可以先只把部分流量引入到新版本,等到确认新版本没有问题时,再把全部流量切换到新版本上。通过调节服务版本路由的权重,就可以达到上述目的。当然,为了不浪费资源,对服务的新版本我们只需要部署很少的实例,然后确认服务没有问题后,再增加新版本的服务实例,减少旧版本的服务实例。

对service-go服务实行灰度发布,使用的路由规则配置文件如下:


apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: service-go
spec:
  host: service-go
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: service-go
spec:
  hosts:
  - service-go
  http:
  - route:
    - destination:
        host: service-go
        subset: v1
      weight: 70
    - destination:
        host: service-go
        subset: v2
      weight: 30

【实验】

1)删除service-go服务的v2版本的Pod:


$ kubectl delete deploy service-go-v2
$ kubectl get pod -l app=service-go
NAME                                READY      STATUS       RESTARTS    AGE
service-go-v1-7cc5c6f574-2k6wv      2/2        Running      0           4m38s

2)为了模拟真实环境,我们提前把service-go服务扩容到5个实例,也就是5个Pod,使用如下方式增加实例个数:


$ kubectl scale --replicas=5 deployment/service-go-v1
$ kubectl get pod -l app=service-go
service-go-v1-7cc5c6f574-9g52l     2/2       Running     0          29s
service-go-v1-7cc5c6f574-kkmmh     2/2       Running     0          29s
service-go-v1-7cc5c6f574-kqwg5     2/2       Running     0          7m24s
service-go-v1-7cc5c6f574-pl4dt     2/2       Running     0          29s
service-go-v1-7cc5c6f574-snd7b     2/2       Running     0          29s

3)创建用于测试的Pod:


$ kubectl apply -f kubernetes/dns-test.yaml

4)模拟生产环境,流量全部转发到服务service-go的v1版本:


$ kubectl apply -f istio/route/virtual-service-go-v1.yaml

5)访问service-go服务,此时所有的请求只会落到v1版本上:


$ kubectl exec dns-test -c dns-test -- curl -s http://service-go/env
{"message":"go v1"}

6)部署新版本服务,此时只创建两个v2版本的service-go服务实例用来准备接收请求:


$ kubectl apply -f service/go/service-go-v2.yaml
$ kubectl scale --replicas=2 deployment/service-go-v2
$ kubectl get pod -l app=service-go
NAME                               READY     STATUS      RESTARTS     AGE
service-go-v1-7cc5c6f574-9g52l     2/2       Running     0            6m40s
service-go-v1-7cc5c6f574-kkmmh     2/2       Running     0            6m40s
service-go-v1-7cc5c6f574-kqwg5     2/2       Running     0            13m
service-go-v1-7cc5c6f574-pl4dt     2/2       Running     0            6m40s
service-go-v1-7cc5c6f574-snd7b     2/2       Running     0            6m40s
service-go-v2-7656dcc478-7xn4h     2/2       Running     0            19s
service-go-v2-7656dcc478-qmkr8     2/2       Running     0            8s

7)给service-go服务v2版本分配30%流量:


$ kubectl apply -f istio/route/virtual-service-go-canary.yaml

8)访问service-go服务,此时请求会分别落到v1、v2两个版本上:


$ kubectl exec dns-test -c dns-test -- curl -s http://service-go/env
{"message":"go v1"}
$ kubectl exec dns-test -c dns-test -- curl -s http://service-go/env
{"message":"go v2"}

9)观察服务的性能、响应时间、错误率等数据,当服务达到要求时,增加service-go服务的v2版本实例数:


$ kubectl scale --replicas=3 deployment/service-go-v2
$ kubectl get pod -l app=service-go
NAME                               READY     STATUS      RESTARTS     AGE
service-go-v1-7cc5c6f574-9g52l     2/2       Running     0            15m
service-go-v1-7cc5c6f574-kkmmh     2/2       Running     0            15m
service-go-v1-7cc5c6f574-kqwg5     2/2       Running     0            22m
service-go-v1-7cc5c6f574-pl4dt     2/2       Running     0            15m
service-go-v1-7cc5c6f574-snd7b     2/2       Running     0            15m
service-go-v2-7656dcc478-78jt7     2/2       Running     0            116s
service-go-v2-7656dcc478-7xn4h     2/2       Running     0            9m
service-go-v2-7656dcc478-qmkr8     2/2       Running     0            8m49s

10)给service-go服务v2版本分配50%流量:


$ kubectl apply -f istio/route/virtual-service-go-canary-50.yaml

11)减少v1版本的实例数,增加service-go服务的v2版本实例数:


$ kubectl scale --replicas=2 deployment/service-go-v1
$ kubectl scale --replicas=4 deployment/service-go-v2
$ kubectl get pod -l app=service-go
NAME                               READY     STATUS      RESTARTS     AGE
service-go-v1-7cc5c6f574-kkmmh     2/2       Running     0            17m
service-go-v1-7cc5c6f574-kqwg5     2/2       Running     0            24m
service-go-v2-7656dcc478-78jt7     2/2       Running     0            4m26s
service-go-v2-7656dcc478-7xn4h     2/2       Running     0            11m
service-go-v2-7656dcc478-qmkr8     2/2       Running     0            11m
service-go-v2-7656dcc478-t2t57     2/2       Running     0            60s

12)给service-go服务v2版本分配70%流量:


$ kubectl apply -f istio/route/virtual-service-go-canary-70.yaml

13)增加service-go服务的v2版本实例数:


$ kubectl scale --replicas=5 deployment/service-go-v2
$ kubectl get pod -l app=service-go
NAME                               READY     STATUS      RESTARTS     AGE
service-go-v1-7cc5c6f574-kkmmh     2/2       Running     0            19m
service-go-v1-7cc5c6f574-kqwg5     2/2       Running     0            26m
service-go-v2-7656dcc478-4nn2q     2/2       Running     0            19s
service-go-v2-7656dcc478-78jt7     2/2       Running     0            5m53s
service-go-v2-7656dcc478-7xn4h     2/2       Running     0            12m
service-go-v2-7656dcc478-qmkr8     2/2       Running     0            12m
service-go-v2-7656dcc478-t2t57     2/2       Running     0            2m27s

14)把流量全部引入service-go服务的v2版本:


$ kubectl apply -f istio/route/virtual-service-go-v2.yaml

15)访问service-go服务,此时所有的请求只会落到v2版本上:


$ kubectl exec dns-test -c dns-test -- curl -s http://service-go/env
{"message":"go v2"}

16)减少或者删除service-go服务v1版本的实例,完成服务的灰度发布:


$ kubectl scale --replicas=0 deployment/service-go-v1
$ kubectl get pod -l app=service-go
NAME                               READY     STATUS      RESTARTS     AGE
service-go-v2-7656dcc478-4nn2q     2/2       Running     0            2m35s
service-go-v2-7656dcc478-78jt7     2/2       Running     0            8m9s
service-go-v2-7656dcc478-7xn4h     2/2       Running     0            15m
service-go-v2-7656dcc478-qmkr8     2/2       Running     0            15m
service-go-v2-7656dcc478-t2t57     2/2       Running     0            4m43s
$ kubectl delete deployment service-go-v1

17)清理。

恢复环境到实验的初始状态:


$ kubectl delete -f kubernetes/dns-test.yaml
$ kubectl delete -f istio/route/virtual-service-go-v2.yaml
$ kubectl scale --replicas=1 deployment/service-go-v2
$ kubectl apply -f service/go/service-go.yaml
$ kubectl get pod -l app=service-go
NAME                               READY     STATUS      RESTARTS     AGE
service-go-v1-7cc5c6f574-2282d     2/2       Running     0            16s
service-go-v2-7656dcc478-7xn4h     2/2       Running     0            22m