biglittleant

不被嘲笑的梦想,是不值得去实现的

0%

ingress nginx 服务部署

ingress 简介

Ingress 公开了从集群外部到集群内服务的 HTTP 和 HTTPS 路由。 流量路由由 Ingress 资源上定义的规则控制。

1
2
3
4
5
internet
|
[ Ingress ]
--|-----|--
[ Services ]

可以将 Ingress 配置为服务提供外部可访问的 URL、负载均衡流量、终止 SSL/TLS,以及提供基于名称的虚拟主机等能力。 Ingress 控制器 通常负责通过负载均衡器来实现 Ingress,尽管它也可以配置边缘路由器或其他前端来帮助处理流量。

安装

使用 NodePort的方式部署 ingress-nginx-controller

1
2
3
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/baremetal/deploy.yaml
mv deploy.yaml ingress-deploy-hostnetwork.yaml
kubectl apply -f ingress-deploy-hostnetwork.yaml

检查服务启动是否正常(拉镜像需要等待一段时间)

1
kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx

检查服务是否正常

1
2
3
4
5
6
7
8
9
10
11
12
POD_NAMESPACE=ingress-nginx
POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app.kubernetes.io/name=ingress-nginx --field-selector=status.phase=Running -o jsonpath='{.items[0].metadata.name}')

kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version
-------------------------------------------------------------------------------
NGINX Ingress controller
Release: v0.35.0
Build: 54ad65e32bcab32791ab18531a838d1c0f0811ef
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.19.2

-------------------------------------------------------------------------------

配置本地网络启动

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
.....
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
selector:
matchLabels:
.....
revisionHistoryLimit: 10
minReadySeconds: 0
template:
metadata:
labels:
.....
spec:
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet

修改内容:

  1. kind: DaemonSet 配置服务启动方式为DaemonSet(原来是deployment)。
  2. 配置 spec.hostNetwork: true,本地网络启动。
  3. 配置 dns的策略为: ClusterFirstWithHostNet, 与hostNetwork 配合使用,参考官方文档pod-s-dns-policy

重新同步一下pod文件

1
kubectl apply -f ingress-deploy-hostnetwork.yaml

配置标签,让ingress 只调度到Master节点上

为了让网关的性能更好,我们需要给ingress打标签,让ingress调度到master机器上。同时给master 打上污点,禁止其他pod调度到master节点上。

1
2
3
4
5
6
7
8
9
kubectl get pods -n ingress-nginx  -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx-admission-create-z7sts 0/1 Completed 0 2d16h 172.30.199.211 node7 <none> <none>
ingress-nginx-admission-patch-4qgtz 0/1 Completed 0 2d16h 172.30.139.14 node6 <none> <none>
ingress-nginx-controller-h5fdd 1/1 Running 1 2d16h 192.168.66.16 node6 <none> <none>
ingress-nginx-controller-j6n9z 1/1 Running 0 2d16h 192.168.66.11 node1 <none> <none>
ingress-nginx-controller-kfnxt 1/1 Running 1 2d16h 192.168.66.17 node7 <none> <none>
ingress-nginx-controller-n5pl4 1/1 Running 0 2d16h 192.168.66.12 node2 <none> <none>
ingress-nginx-controller-qzbb5 1/1 Running 0 2d16h 192.168.66.13 node3 <none> <none>

对 node1 node2 打标签

1
2
kubectl label nodes node1 api/ingress-controller=true
kubectl label nodes node2 api/ingress-controller=true

查看标签

1
2
3
4
5
6
7
8
# 为了好看,去掉相同的标签
kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
node1 Ready <none> 3h53m v1.16.6 api/ingress-controller=true,kubernetes.io/hostname=node1
node2 Ready <none> 3h53m v1.16.6 api/ingress-controller=true,kubernetes.io/hostname=node2
node3 Ready <none> 3h53m v1.16.6 kubernetes.io/hostname=node3
node6 Ready <none> 3d4h v1.16.6 kubernetes.io/hostname=node6
node7 Ready <none> 3d4h v1.16.6 kubernetes.io/hostname=node7

增加标签选择,及容忍污点

1
2
3
4
5
6
7
8
9
10
11
# vim ingress-deploy-hostnetwork.yaml
spec:
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
nodeSelector:
api/ingress-controller: "true"
tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Equal"
value: ""
effect: "NoSchedule"

在刚才修改网络的后面继续增加以上命令

从新同步一下配置

1
kubectl apply -f ingress-deploy-hostnetwork.yaml
1
2
3
4
5
6
[root@node1 ~]# kubectl get pods -n ingress-nginx  -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx-admission-create-z7sts 0/1 Completed 0 2d16h 172.30.199.211 node7 <none> <none>
ingress-nginx-admission-patch-4qgtz 0/1 Completed 0 2d16h 172.30.139.14 node6 <none> <none>
ingress-nginx-controller-6s2lp 1/1 Running 0 57m 192.168.66.12 node2 <none> <none>
ingress-nginx-controller-j6n9z 1/1 Running 0 57m 192.168.66.11 node1 <none> <none>

为了避免master节点会被调度新的pod,需要给master节点打上Taints。
如果一个节点标记为 Taints ,除非 pod 也被标识为可以容忍污点节点,否则该 Taints 节点不会被调度 pod。
影响策略是 NoSchedule,只会影响新的 pod 调度

1
2
kubectl taint nodes node1 node-role.kubernetes.io/master=:NoSchedule
kubectl taint nodes node2 node-role.kubernetes.io/master=:NoSchedule
1
2
3
4
5
[root@node1 ~]# kubectl describe node node1 |grep Taints
Taints: node-role.kubernetes.io/master:NoSchedule
[root@node1 ~]# kubectl describe node node2 |grep Taints
Taints: node-role.kubernetes.io/master:NoSchedule
[root@node1 ~]#
1
2
3
4
5
6
7
[root@node1 ~]# kubectl get pods -n ingress-nginx  -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx-admission-create-z7sts 0/1 Completed 0 2d19h 172.30.199.211 node7 <none> <none>
ingress-nginx-admission-patch-4qgtz 0/1 Completed 0 2d19h 172.30.139.14 node6 <none> <none>
ingress-nginx-controller-6s2lp 1/1 Running 0 3h6m 192.168.66.12 node2 <none> <none>
ingress-nginx-controller-kv4xs 1/1 Running 0 3h5m 192.168.66.11 node1 <none> <none>
[root@node1 ~]#

汇总一下配置文件的修改内容

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
.....
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
selector:
matchLabels:
.....
revisionHistoryLimit: 10
minReadySeconds: 0
template:
metadata:
labels:
.....
spec:
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
nodeSelector:
api/ingress-controller: "true"
tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Equal"
value: ""
effect: "NoSchedule"

修改参数如下:

  • kind: Deployment #修改为DaemonSet
  • hostNetwork: true #添加该字段让docker使用物理机网络,在物理机暴露服务端口(80),注意物理机80端口提前不能被占用
  • dnsPolicy: ClusterFirstWithHostNet #使用hostNetwork后容器会使用物理机网络包括DNS,会无法解析内部service,使用此参数让容器使用K8S的DNS
  • nodeSelector:api/ingress-controller: “true” #添加节点标签
  • tolerations: 添加对指定节点容忍度

参考文章

官方仓库
官方文档
k8s配置ingress