Istio通过设置VirtualService中的retries字段来指定服务的重试机制。
使用示例如下:
1 apiVersion: networking.istio.io/v1alpha3 2 kind: VirtualService 3 metadata: 4 name: service-node 5 spec: 6 hosts: 7 - service-node 8 http: 9 - route: 10 - destination: 11 host: service-node 12 retries: 13 attempts: 3 14 perTryTimeout: 2s
第12~14行定义了重试规则,当调用service-node服务时,如果服务出错,就需要进行重试,最多可以重试3次,每次调用超时为2秒,每次重试的时间间隔由Istio决定,重试时间间隔一般会大于25毫秒。
【实验】
1)创建并发测试的Pod:
$ kubectl apply -f kubernetes/fortio.yaml
2)部署httpbin服务:
$ kubectl apply -f kubernetes/httpbin.yaml $ kubectl get pod -l app=httpbin NAME READY STATUS RESTARTS AGE httpbin-b67975b8f-vmbtv 2/2 Running 0 49s
3)创建httpbin服务路由规则:
$ kubectl apply -f istio/route/virtual-service-httpbin.yaml
4)访问httpbin服务:
$ kubectl exec fortio -c fortio /usr/local/bin/fortio -- load -curl http://httpbin:8000/status/200 HTTP/1.1 200 OK server: envoy date: Wed, 16 Jan 2019 14:03:00 GMT content-type: text/html; charset=utf-8 access-control-allow-origin: * access-control-allow-credentials: true content-length: 0 x-envoy-upstream-service-time: 33 $ kubectl exec fortio -c fortio /usr/local/bin/fortio -- load -c 10 -qps 0 -n 100 -loglevel Error http://httpbin:8000/status/200%2C200%2C200%2C200%2C500 14:18:37 I logger.go:97> Log level is now 4 Error (was 2 Info) Fortio 1.0.1 running at 0 queries per second, 2->2 procs, for 100 calls: http://httpbin:8000/status/200%2C200%2C200%2C200%2C500 Aggregated Function Time : count 100 avg 0.24802899 +/- 0.06426 min 0.016759858 max 0.390472066 sum 24.8028985 # target 50% 0.252941 # target 75% 0.289706 # target 90% 0.326667 # target 99% 0.376981 # target 99.9% 0.389123 Sockets used: 30 (for perfect keepalive, would be 10) Code 200 : 78 (78.0 %) Code 500 : 22 (22.0 %) All done 100 calls (plus 0 warmup) 248.029 ms avg, 38.5 qps
5)创建httpbin服务重试路由规则:
$ kubectl apply -f istio/resilience/virtual-service-httpbin-retry.yaml
6)访问httpbin服务:
$ kubectl exec fortio -c fortio /usr/local/bin/fortio -- load -curl http://httpbin:8000/status/200 HTTP/1.1 200 OK server: envoy date: Wed, 16 Jan 2019 14:19:14 GMT content-type: text/html; charset=utf-8 access-control-allow-origin: * access-control-allow-credentials: true content-length: 0 x-envoy-upstream-service-time: 5 $ kubectl exec fortio -c fortio /usr/local/bin/fortio -- load -c 10 -qps 0 -n 100 -loglevel Error http://httpbin:8000/status/200%2C200%2C200%2C200%2C500 14:19:32 I logger.go:97> Log level is now 4 Error (was 2 Info) Fortio 1.0.1 running at 0 queries per second, 2->2 procs, for 100 calls: http://httpbin:8000/status/200%2C200%2C200%2C200%2C500 Aggregated Function Time : count 100 avg 0.23708609 +/- 0.1323 min 0.017537636 max 0.793965189 sum 23.7086086 # target 50% 0.226471 # target 75% 0.275 # target 90% 0.383333 # target 99% 0.7 # target 99.9% 0.784569 Sockets used: 13 (for perfect keepalive, would be 10) Code 200 : 97 (97.0 %) Code 500 : 3 (3.0 %) All done 100 calls (plus 0 warmup) 237.086 ms avg, 35.5 qps
从上面的测试结果可以看出,当没有开启服务重试时,服务有大概1/4的请求失败,当开启服务重试之后,服务只有极少数的请求失败。
7)清理:
$ kubectl delete -f kubernetes/fortio.yaml $ kubectl delete -f kubernetes/httpbin.yaml $ kubectl delete -f istio/resilience/virtual-service-httpbin-retry.yaml