k3s部署高可用雷池社区版
作者:约苗-运维师傅-候天鸿(社区32群)
部署前言
长亭科技的雷池开源WAF(Web Application FireWalld)是一款足够简单、足够好用、足够强的免费 WAF。基于业界领先的语义引擎检测技术,作为反向代理接入,保护你的网站不受黑客攻击。
至文档编写时官方开源版本与专业版并不支持HA高可用与集群模式部署。因需在生产环境部署故考虑架构必须至少是HA高可用,而官方不支持,仅在社区中发现有人编写过K8S集群中通过HelmChart的部署方案。
耗时长达一个月对开源版本的测试部署,最终部署方案采用在k3s集群之上采用HelmChart部署,Chart中镜像完全采用官方原镜像不做任何改动。
尽管如此,运行在k3s集群中的雷池WAF服务有且只能有一个POD副本,如有多个副本会导致某些功能异常,如:控制面板首页无访问计数变动、业务能正常转发但入侵行为无法正常检测与拦截。
目前此文档仅实现了WAF服务运行在K3S集群多个节点上,如有节点宕机,节点上的服务可自动切换至其它节点继续运行,并未实现服务多POD副本负载均衡。
雷池WAF官方网站: https://waf-ce.chaitin.cn/
雷池WAF官方GITHUB: https://github.com/chaitin/SafeLine
雷池WAF第三方HelmChart仓库: https://github.com/jangrui/charts
雷池WAF第三方HelmChart源码仓库: https://github.com/jangrui/SafeLine
个人腾讯Coding的雷池HelmChart仓库: https://g-otkk6267-helm.pkg.coding.net/Charts/safeline
部署前准备
准备3台服务器,配置在4C8G,生产环境建议采用8C16G及以上配置,操作系统为Ubuntu22.04,作为K3S集群的服务器。在K3S集群的基础上部署雷池WAF。当前文档配置信息如下:
主机名 | 主机IP | 用途 |
---|---|---|
waf-lan-k3s-master | 192.168.1.9 | K3S集群的master节点 |
waf-lan-k3s-node1 | 192.168.1.7 | K3S集群的node1节点 |
waf-lan-k3s-node2 | 192.168.1.8 | K3S集群的node2节点 |
部署K3S集群
部署k3s-Master节点服务
脚本部署master节点服务
安装k3s版本对标k8s-1.28.8版本,在部署同时禁用自带的traefik网关 与local-storage的存储类型
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn INSTALL_K3S_VERSION=v1.28.8+k3s1 sh -s - --disable traefik --disable local-storage
kubectl get nodes -owide #查看集群信息
kubectl get pod -A #查看集群pod启动情况
参照官方部署文档如下: https://docs.k3s.io/zh/quick-start https://docs.k3s.io/zh/installation/configuration 可部署升级的k3s版本,请查询官方如下地址: https://update.k3s.io/v1-release/channels
安装完毕后会有如下实用工具: kubectl、crictl、ctr、k3s-killall.sh 和 k3s-uninstall.sh
调整集群kubeconfig文件
- k3s默认的kubeconfig文件存放路径:
ls -l /etc/rancher/k3s/k3s.yaml
- 解决k3s下默认无/root/.kube/config文件问题
echo "export KUBECONFIG=/etc/rancher/k3s/k3s.yaml" >> /etc/profile
source /etc/profile
mkdir -p /root/.kube/
ln -sf /etc/rancher/k3s/k3s.yaml /root/.kube/config
cat /root/.kube/config #查看kubeconfig文件内容
- 将Master节点配置上一个污点,使其不可调度
kubectl taint nodes waf-lan-k3s-master node-role.kubernetes.io/control-plane:NoSchedule
# 查看是否设置成功
kubectl describe nodes waf-lan-k3s-master |grep Taints
# 命令显示内容如下:
Taints: node-role.kubernetes.io/control-plane:NoSchedule
配置k3s集群中的私有仓库信息
默认k3s集群中containerd中配置docker镜像的私有仓库地址为空,加速地址为默认的dockerhub仓库域名,必须手动进行指定才能正常拉取私有仓库中的容器镜像与使用国内镜像加速源。
k3s集群中自定义私有仓库地址,需要在master节点服务器上的/etc/rancher/k3s
目录中新建registries.yaml文件。
在此文档中registries.yaml文件内容如下(注意:此文件中的密码值不是真实存在的):
mirrors:
"docker.io":
endpoint:
- "http://192.168.1.138:5000"
- "https://tshf4sa5.mirror.aliyuncs.com"
"harbor.scmttec.com":
endpoint:
- "https://harbor.scmttec.com"
configs:
"harbor.scmttec.com":
auth:
username: "robot$pull_image"
password: "b9RCmoH5vN5ZA0"
tls:
insecure_skip_verify: false
文件存放好以后,重启k3s服务使配置生效
systemctl restart k3s
安装helm工具
- helm官方下载工具压缩包并解压安装
下载页面:https://github.com/helm/helm/releases
下载地址:https://get.helm.sh/helm-v3.12.0-linux-amd64.tar.gz
wget https://get.helm.sh/helm-v3.12.0-linux-amd64.tar.gz
tar zxf helm-v3.12.0-linux-amd64.tar.gz
cp ./linux-amd64/helm /usr/local/bin/
rm -rf ./linux-amd64/ #删除解压出来的目录
- 配置helm命令和kubectl命令的自动补全功能
echo "source <(kubectl completion bash)" >> /etc/profile #添加kubectl命令自动补全
echo "source <(helm completion bash)" >> /etc/profile #添加helm命令自动补全
source /etc/profile #让功能立即生效
- 查看helm命令是否能正常使用
helm list -n kube-system
#显示如下信息:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
traefik kube-system 1 2023-05-29 06:39:26.751316258 +0000 UTC deployed traefik-21.2.1+up21.2.0 v2.9.10
traefik-crd kube-system 1 2023-05-29 06:39:22.816811082 +0000 UTC deployed traefik-crd-21.2.1+up21.2.0 v2.9.10
部署k3s-Node节点服务
获 取k3s-Master节点的token文件
在部署k3s-node节点服务即k3s-agent服务之前,需要获取k3s-server服务的token文件,文件位置为master节点的/var/lib/rancher/k3s/server/token文件,在master节点执行如下命令获取token值:
cat /var/lib/rancher/k3s/server/token
#显示结果如下:
K10f890abb83ce8cdf8b5dbeff8edb628d10cc53c23fa9f56152db0cf22454546bf::server:fec37548170a2affec1e3ebd1fdf708d
通过官方脚本直接安装k3s-agent服务
k3s官方有k3s-agent服务的安装脚本,通过curl直接进行安装,在安装的同时指定server即master节点的IP地址与token值。 在2台Node节点服务器上直接执行命令如下:
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn INSTALL_K3S_VERSION=v1.28.8+k3s1 K3S_URL=https://192.168.1.9:6443 K3S_TOKEN=K10f890abb83ce8cdf8b5dbeff8edb628d10cc53c23fa9f56152db0cf22454546bf::server:fec37548170a2affec1e3ebd1fdf708d sh -
执行后在master节点通过kubectl命令查看新加节点状态:
kubectl get nodes -owide
# 显示结果如下:
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
waf-lan-k3s-master Ready control-plane,master 17d v1.28.8+k3s1 192.168.1.9 <none> Ubuntu 22.04.1 LTS 5.15.0-69-generic containerd://1.7.11-k3s2
waf-lan-k3s-node2 Ready <none> 17d v1.28.8+k3s1 192.168.1.8 <none> Ubuntu 22.04.1 LTS 5.15.0-69-generic containerd://1.7.11-k3s2
waf-lan-k3s-node1 Ready <none> 17d v1.28.8+k3s1 192.168.1.7 <none> Ubuntu 22.04.1 LTS 5.15.0-69-generic containerd://1.7.11-k3s2
至此K3S集群基础服务已成功部署完毕!
安装k3s集群存储与网关组件
安装Nginx-Ingress组件
通过nginx-ingress官方github站点获取最新安装版本及文件,并对官方部署文件进行修改后得出最终文件nginx-ingress-v1.10.0.yaml。
官方github仓库地址: https://github.com/kubernetes/ingress-nginx
官方对裸机部署注意事项: https://github.com/kubernetes/ingress-nginx/blob/controller-v1.10.0/docs/deploy/baremetal.md
官方裸机部署文件地址: https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/baremetal/deploy.yaml
修改后的nginx-ingress-v1.10.0.yaml部署文件内容如下:
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- coordination.k8s.io
resourceNames:
- ingress-nginx-leader
resources:
- leases
verbs:
- get
- update
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- list
- watch
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- list
- watch
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: v1
data:
allow-snippet-annotations: "false"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-controller
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
#type: NodePort
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
apiVersion: apps/v1
#kind: Deployment
kind: DaemonSet
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
#strategy:
# rollingUpdate:
# maxUnavailable: 1
# type: RollingUpdate
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
spec:
hostNetwork: true
dnsPolicy: ClusterFirst
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-nginx-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
#image: registry.k8s.io/ingress-nginx/controller:v1.10.0@sha256:42b3f0e5d0846876b1791cd3afeb5f1cbbe4259d6f35651dcc1b5c980925379c
image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.10.0
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
- containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
readOnlyRootFilesystem: false
runAsNonRoot: true
runAsUser: 101
seccompProfile:
type: RuntimeDefault
volumeMounts:
- mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
- name: webhook-cert
secret:
secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission-create
spec:
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
#image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0@sha256:44d1d0e9f19c63f58b380c5fddaca7cf22c7cee564adeff365225a5df5ef3334
image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65532
seccompProfile:
type: RuntimeDefault
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
#image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0@sha256:44d1d0e9f19c63f58b380c5fddaca7cf22c7cee564adeff365225a5df5ef3334
image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65532
seccompProfile:
type: RuntimeDefault
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
在master节点直接apply此文件进行部署
kubectl apply -f nginx-ingress-v1.10.0.yaml
查看并核实是否部署成功
kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-7gkvs 0/1 Completed 0 6d2h
ingress-nginx-admission-patch-lc9zc 0/1 Completed 0 6d2h
ingress-nginx-controller-dwzm4 1/1 Running 0 6d2h
ingress-nginx-controller-47hr2 1/1 Running 0 6d2h
在节点上查看是否生效
netstat -ntpl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 80515/nginx: master
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 80515/nginx: master
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 80515/nginx: master
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 80515/nginx: master
tcp 0 0 127.0.0.1:10010 0.0.0.0:* LISTEN 13145/containerd tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 80515/nginx: master
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 80515/nginx: master
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 80515/nginx: master
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 80515/nginx: master
............
注意nginx-ingress部署完毕后master节点因有污点,故默认无法暴露80与443端口,仅有集群节点可以使用。
至此nginx-ingress服务成功部署完毕。