在kubernetes上运行Spring Cloud Gateway(二)

https://qingmu.io/2019/08/06/Run-Spring-Cloud-Gateway-on-kubernetes/文章中,我们搭建好了基本的网关的架子。这篇文章中
我们继续剩下的部分,将网关和部署到k8s中,并结合Ingress完成域名访问我们的网关,通过Secret完成网关HTTPS的安全加密。

流程图

  1. 我们可以使用阿里云的SLB作为IP入口,可以将域名直接用*.xxx.com解析到该IP上。
  2. IngressController采用DS+HostNetwork方式运行,管理所有的二级域名,以及HTTPS。
  3. 最后层就是我们的Gateway了,负责自动发现我们的业务服务,并通过Ribbon进行LB。
  4. SLB访问Ingress走HostNetwork,Ingress访问Gateway走IPVS,Gateway访问业务服务走pod网络。

流程图

Docker image

这里我们直接使用maven的docker插件完成一下流程.

1
buildFatjar --> buildDockerImage --> pushDockerImage

一个完整的maven插件配置如下。这个插件在我们前面的文章中已经配置好了,关于这个插件相关的详细你可以参考我的这篇文章 https://qingmu.io/2018/08/07/How-to-run-springcloud-in-docker/

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<configuration>
<imageName>
freemanliu/demo-gatway:v1.0.0
</imageName>
<registryUrl></registryUrl>
<workdir>/work</workdir>
<rm>true</rm>
<env>
<TZ>Asia/Shanghai</TZ>
<JAVA_OPTS>
-XX:+UnlockExperimentalVMOptions \
-XX:+UseCGroupMemoryLimitForHeap \
-XX:MaxRAMFraction=2 \
-XX:CICompilerCount=8 \
-XX:ActiveProcessorCount=8 \
-XX:+UseG1GC \
-XX:+AggressiveOpts \
-XX:+UseFastAccessorMethods \
-XX:+UseStringDeduplication \
-XX:+UseCompressedOops \
-XX:+OptimizeStringConcat
</JAVA_OPTS>
</env>
<baseImage>freemanliu/openjre:8.212</baseImage>
<cmd>
/sbin/tini java ${JAVA_OPTS} -jar ${project.build.finalName}.jar
</cmd>
<!--是否推送image-->
<pushImage>true</pushImage>
<resources>
<resource>
<directory>${project.build.directory}</directory>
<include>${project.build.finalName}.jar</include>
</resource>
</resources>
<serverId>docker-hub</serverId>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
</execution>
</executions>
</plugin>

配置好以上插件之后我们执行以下命令.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
$ mvn clean package -DskipTests
[INFO]
[INFO] -----------------------< io.qingmu:demo-gateway >-----------------------
[INFO] Building demo-gateway 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ demo-gateway ---
[INFO] Deleting /Users/freeman/GithubProjects/demo-gateway/target
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ demo-gateway ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ demo-gateway ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 6 source files to /Users/freeman/GithubProjects/demo-gateway/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ demo-gateway ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/freeman/GithubProjects/demo-gateway/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ demo-gateway ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /Users/freeman/GithubProjects/demo-gateway/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ demo-gateway ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:3.1.2:jar (default-jar) @ demo-gateway ---
[INFO] Building jar: /Users/freeman/GithubProjects/demo-gateway/target/demo-gateway-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] --- spring-boot-maven-plugin:2.1.7.RELEASE:repackage (repackage) @ demo-gateway ---
[INFO] Replacing main artifact with repackaged archive
[INFO]
[INFO] --- docker-maven-plugin:1.2.1:build (default) @ demo-gateway ---
[INFO] Using authentication suppliers: [ConfigFileRegistryAuthSupplier, FixedRegistryAuthSupplier]
[INFO] Copying /Users/freeman/GithubProjects/demo-gateway/target/demo-gateway-0.0.1-SNAPSHOT.jar -> /Users/freeman/GithubProjects/demo-gateway/target/docker/demo-gateway-0.0.1-SNAPSHOT.jar
[INFO] Building image freemanliu/demo-gateway:v1.0.0
Step 1/6 : FROM freemanliu/openjre:8.212

---> 8a33da3f701e
Step 2/6 : ENV JAVA_OPTS -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=2 -XX:CICompilerCount=8 -XX:ActiveProcessorCount=8 -XX:+UseG1GC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:+UseStringDeduplication -XX:+UseCompressedOops -XX:+OptimizeStringConcat

---> Using cache
---> 0f65d8120eb4
Step 3/6 : ENV TZ Asia/Shanghai

---> Using cache
---> 698b12f89dd2
Step 4/6 : WORKDIR /work

---> Using cache
---> 45e2bbd5eb93
Step 5/6 : ADD demo-gateway-0.0.1-SNAPSHOT.jar .

---> 0a76e61162bb
Step 6/6 : CMD /sbin/tini java ${JAVA_OPTS} -jar demo-gateway-0.0.1-SNAPSHOT.jar

---> Running in 034aee2b1d56
Removing intermediate container 034aee2b1d56
---> 19591ef9767b
ProgressMessage{id=null, status=null, stream=null, error=null, progress=null, progressDetail=null}
Successfully built 19591ef9767b
Successfully tagged freemanliu/demo-gateway:v1.0.0
[INFO] Built freemanliu/demo-gateway:v1.0.0
[INFO] Pushing freemanliu/demo-gateway:v1.0.0
The push refers to repository [docker.io/freemanliu/demo-gateway]
854550ddc678: Pushed
607d6bc26ce8: Pushed
e02c5cc56276: Pushed
f1b5933fe4b5: Layer already exists
v1.0.0: digest: sha256:bcc45e9b9edde415523c7e742df55486eb9d109c6864729fdf68eeefd13b3b9e size: 1158
null: null
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 59.288 s
[INFO] Finished at: 2019-09-15T20:26:42+08:00
[INFO] ------------------------------------------------------------------------

查看

完整的执行完打包命令之后,我们的image就被推送到了docker的中央仓库
gateway1

部署到K8S

完成了我们的前置工作,我们就可以编写我们的yaml描述文件

gateway.deployment.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
apiVersion: v1
kind: Service
metadata:
name: demo-gateway
namespace: default
labels:
app: demo-gateway
spec:
ports:
- port: 8084
name: tcp
selector:
app: demo-gateway
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-gateway
namespace: default
spec:
revisionHistoryLimit: 10
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 25%
maxSurge: 25%
replicas: 1
selector:
matchLabels:
app: demo-gateway
template:
metadata:
labels:
app: demo-gateway
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
topologyKey: kubernetes.io/hostname
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- app-gateway # 反亲和,尽量让gateway分散到不同的节点上
weight: 1
containers:
- name: demo-gateway
image: freemanliu/demo-gateway:v1.0.0
imagePullPolicy: Always
lifecycle:
preStop:
httpGet:
port: 8084
path: /spring/shutdown
livenessProbe:
httpGet:
path: /actuator/health
port: 8084
periodSeconds: 5
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 5
readinessProbe:
httpGet:
path: /actuator/health
port: 8084
periodSeconds: 5
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 5
resources:
requests:
memory: 1Gi
limits:
memory: 1Gi
ports:
- containerPort: 8084
env:
- name: EUREKA_SERVER
value: "http://eureka-0.eureka:8761/eureka/,http://eureka-1.eureka:8761/eureka/,http://eureka-2.eureka:8761/eureka/"
- name: SAMPLER_PROBABILITY
value: "1.0"
- name: ZIPKIN
value: "http://10.96.0.13:9411/"

apply

1
$ kubectl apply -f demo.gateway.deployment.yaml

查看

1
2
$ kubectl get po -owide | grep demo-gateway
demo-gateway-c94bc5679-8ld2v 1/1 Running 0 3m35s 172.224.3.23 node1 <none> <none>

通过podip访问

1
2
3
4
5
$ curl -i 172.224.3.23:8084/demo2-service/world
HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8
Content-Length: 5
Date: Tue, 17 Sep 2019 06:53:46 GMT

查看svc

1
2
$ kubectl get svc | grep demo-gateway
demo-gateway ClusterIP 10.111.213.174 <none> 8084/TCP 5m10s

通过svcip访问

1
2
3
4
5
$ curl -i 10.111.213.174:8084/demo2-service/world
HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8
Content-Length: 5
Date: Tue, 17 Sep 2019 06:55:25 GMT

到此我们就将完整的demo部署到了kubernetes中。接下来我们处理ingress,以支持通过域名和https访问我们的服务。

Ingress和Https

这里我们使用域名gateway.qingmu.io访问我们的服务。首先需要申请一个HTTPS证书,自己测试使用可以在阿里云的证书服务中申请一个为期一年的,生产使用建议购买一个通配证书。

上传证书信息到K8S集群

在kubernetes中专门抽象了一个资源类型叫Secret,专门用户存储我们的上传的信息,存在这里的信息,将会被我们的ingress-controller使用,并自动配置https。
我们申请完的证书,选择下载nginx,下载之后的zip包中解压开会有两个文件2232293__gateway.qingmu.io.key,2232293__gateway.qingmu.io.pem

1
2
gateway.qingmu.io_nginx freeman$ ls
2822276_gateway.qingmu.io.key 2822276_gateway.qingmu.io.pem

创建Secret

这里-n代表存储在哪个namespace中,这取决于你的ingress存在哪个namespace中,这里我们都是在default中。

1
2
3
4
$ kubectl -n default create secret tls qingmu-secret \
--key 2822276_gateway.qingmu.io.key \
--cert 2822276_gateway.qingmu.io.pem
secret/qingmu-secret created

查看Secret

1
2
3
$ kubectl get secret -ndefault qingmu-secret
NAME TYPE DATA AGE
qingmu-secret kubernetes.io/tls 2 84s

Ingress

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: gateway-ingress
namespace: default
annotations:
# 是否强制重定向到https
ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
tls:
- hosts:
- gateway.qingmu.io
# 这里需要和我们上面的一致
secretName: qingmu-secret
rules:
- host: gateway.qingmu.io
http:
paths:
- backend:
serviceName: demo-gateway
servicePort: 8084

部署Ingress

1
2
$ kubectl apply -f demo.gateway.ingress.yaml 
ingress.networking.k8s.io/gateway-ingress created

查看

1
2
3
$ kubectl get ingress gateway-ingress
NAME HOSTS ADDRESS PORTS AGE
gateway-ingress gateway.qingmu.io 80, 443 53s

通过域名访问

HTTP访问

访问http端口的时候,服务器会返回重定向308(永久重定向)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ curl -i gateway.qingmu.io/demo2-service/world
HTTP/1.1 308 Permanent Redirect
Server: openresty/1.15.8.1
Date: Tue, 17 Sep 2019 07:41:08 GMT
Content-Type: text/html
Content-Length: 177
Connection: keep-alive
Location: https://gateway.qingmu.io/demo2-service/world

<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>openresty/1.15.8.1</center>
</body>
</html>

Https访问

1
2
3
4
5
6
7
$ curl -i https://gateway.qingmu.io/demo2-service/world
HTTP/2 200
server: openresty/1.15.8.1
date: Tue, 17 Sep 2019 07:40:06 GMT
content-type: text/plain;charset=UTF-8
content-length: 5
strict-transport-security: max-age=15724800; includeSubDomains

到此我们就完整的spring cloud gateway部署到了kubernetes中。

Github

https://github.com/qingmuio/demo-gateway

推荐文章