使用Nginx-Ingress暴露TCP和udp服务

nginx-ingress在k8s中通常是作为7层的反向代理，其实nginx-ingress也是可以代理tcp和udp的服务，nginx-ingrss控制器的启动参数默认有提供–tcp-services-configmap和–udp-services-configmap这2个，nginx-ingres控制器会不断的watch这2个configmap的配置，然后配置到后端的/etc/nginx/nginx.conf。

tcp和udp配置方法

configmap的配置格式如下

port: <namespace/service name>:<service port>

下面我们配置下configmap来暴露集群的dns服务和thanos-query服务，dns服务是udp协议，thanos-query服务是tcp协议

tcp-services-configmap配置如下

apiVersion: v1
data:
  "9000": thanos/thanos-query:9090
kind: ConfigMap
metadata:
  labels:
    k8s-app: ingress-ingress-nginx-tcp
    qcloud-app: ingress-ingress-nginx-tcp
  name: ingress-ingress-nginx-tcp
  namespace: kube-system

udp-services-configmap配置如下

apiVersion: v1
data:
  "53": dnsmasq/dnsmasq:53
kind: ConfigMap
metadata:
  labels:
    k8s-app: ingress-ingress-nginx-udp
    qcloud-app: ingress-ingress-nginx-udp
  name: ingress-ingress-nginx-udp
  namespace: kube-system

配置好之后，看下pod内的nginx.conf是如何配置的

# TCP services                       
                                                   
 server {   
         preread_by_lua_block {       
                 ngx.var.proxy_upstream_name="tcp-thanos-thanos-query-9090";
         }     
                       
         listen                  9000;
                               
         proxy_timeout           600s;                                
         proxy_pass              upstream_balancer;
                               
 }                                                                    
          
 # UDP services                    
                                           
 server {                                          
         preread_by_lua_block {    
                 ngx.var.proxy_upstream_name="udp-dnsmasq-dnsmasq-53";
         }                                         
                                      
         listen                  53 udp;
                                        
         proxy_responses         1;     
         proxy_timeout           600s;
         proxy_pass              upstream_balancer;
 }                                

配置好configmap后，还需要在service加上端口配置，这里需要注意下，配置的udp和tcp只是在nginx进行了转发配置，但是不会自动在service加上端口映射，如果要通过service访问需要手动加上点开配置。

apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: ingress-ingress-nginx-controller
    qcloud-app: ingress-ingress-nginx-controller
  name: ingress-ingress-nginx-controller
  namespace: kube-system
spec:
  ports:
  - name: http
    nodePort: 32113
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    nodePort: 32452
    port: 443
    protocol: TCP
    targetPort: 443
  - name: 9000-tcp
    nodePort: 31491
    port: 9000
    protocol: TCP
    targetPort: 9000
  - name: 53-udp
    nodePort: 31492
    port: 53
    protocol: udp
    targetPort: 53
  selector:
    k8s-app: ingress-ingress-nginx-controller
    qcloud-app: ingress-ingress-nginx-controller
  type: NodePort

验证端口

这里可以通过service验证，也可以直接访问后端pod的9000端口和53端口是否能访问到后端服务。

[root@VM-0-13-centos ~]# k get pod -A -o wide | grep ngress-ingress-nginx-controller
kube-system                    ingress-ingress-nginx-controller-844b559b5d-hpcn2         1/1     Running     0          74d     10.0.1.237   10.0.0.2                <none>           1/1

[root@VM-0-13-centos ~]# nslookup  cls-xxxx.ccs.tencent-cloud.com 10.0.1.237
Server:         10.0.1.237
Address:        10.0.1.237#53

Name:   cls-xxxx.ccs.tencent-cloud.com
Address: 10.0.0.60

[root@VM-0-13-centos ~]# curl 10.0.1.237:9000
<a href="/graph">Found</a>.

经过而是直接通过pod的53端口可以解析我们的域名，通过9000端口可以访问到thanos-query服务，说明用nginx-ingress暴露tcp和udp服务配置成功。