系统整理K8S的配置管理实战-建议收藏系列

原文公众号链接:https://mp.weixin.qq.com/s/gZx-IIW1g9-Kl7yXw7tsEA
原文公众号链接:https://mp.weixin.qq.com/s/gZx-IIW1g9-Kl7yXw7tsEA

一、ConfigMap

应用部署的最佳实践都趋向于将应用所需要的配置信息和应用程序本身离开,以便实现通过不同的配置实现更灵活的功能。

我们都知道,K8S中运行的是容器,若不将配置文件抽离,每一次配置文件的变动,都需要重新制作镜像,这其实挺麻烦挺没必要的。

在K8S中提供了ConfigMap资源对象作为配置管理的统一管理方案,可以通过环境变量或者文件的方式,在创建容器的时候将配置信息动态注入到容器中~

下文开始对ConfigMap的实战

官网文档:https://kubernetes.io/zh/docs/concepts/configuration/configmap/

1.1、创建

1.1.1、from-file

准备目录和文件

# 创建目录及配置文件 [root@master01 yamls]# ls conf/ nginx.conf  redis.conf  [root@master01 yamls]# cat conf/nginx.conf app.name=nginx location.prefix=/app  [root@master01 yamls]# cat conf/redis.conf ip=10.10.10.101 port=2379 

创建cm

[root@master01 yamls]# kubectl create cm -h # --from-file参数既可以指定文件又可以指定目录 [root@master01 yamls]# kubectl create cm cm-from-file --from-file=conf/ configmap/cm-from-file created 

查看

[root@master01 yamls]# kubectl get cm NAME               DATA   AGE cm-from-file       2      38s kube-root-ca.crt   1      17d  [root@master01 yamls]# kubectl get cm cm-from-file -oyaml apiVersion: v1 data:   # 默认的data的key值就是文件名   # 可以在创建时通过参数 --from-file=abc=nginx.conf的方式将data.key改成abc   nginx.conf: |     app.name=nginx     location.prefix=/app   redis.conf: |     ip=10.10.10.101     port=2379 kind: ConfigMap metadata:   creationTimestamp: "2022-03-31T00:52:21Z"   name: cm-from-file   namespace: default   resourceVersion: "759788"   uid: 5a41bdbe-66d0-45fa-8a5e-b6f6b005d671 
1.1.2、from-env-file

env环境变量的格式为:key=value

指定参数:--from-env-file而不是--from-file

[root@master01 yamls]# kubectl create cm cm-env-file --from-env-file=conf/nginx.conf configmap/cm-env-file created  [root@master01 yamls]# kubectl get  cm cm-env-file -oyaml apiVersion: v1 data:   app.name: nginx   location.prefix: /app kind: ConfigMap metadata:   creationTimestamp: "2022-03-31T01:04:12Z"   name: cm-env-file   namespace: default   resourceVersion: "761496"   uid: 8e885d31-0a83-423c-a59f-6ea8b0473103 
1.1.3、from-literal

literal(翻译为:字面意义的)直接在命令行上指定好key和value

参数:--from-literal

[root@master01 ~]# kubectl create cm env-from-literal --from-literal=Level=INFO --from-literal=Name=Mac configmap/env-from-literal created  [root@master01 ~]# kubectl get cm env-from-literal -oyaml apiVersion: v1 data:   Level: INFO   Name: Mac kind: ConfigMap metadata:   creationTimestamp: "2022-04-01T00:31:13Z"   name: env-from-literal   namespace: default   resourceVersion: "762641"   uid: da2f8922-8385-46ba-82e9-b0dc197a1eb2 [root@master01 ~]# 
1.1.4、基于yaml文件创建
apiVersion: v1 kind: ConfigMap metadata:   name: game-demo data:   # 类属性键;每一个键都映射到一个简单的值   player_initial_lives: "3"   ui_properties_file_name: "user-interface.properties"    # 类文件键   game.properties: |     enemy.types=aliens,monsters     player.maximum-lives=5       user-interface.properties: |     color.good=purple     color.bad=yellow     allow.textmode=true     

1.2、Pod使用ConfigMap

1.2.1、valueFrom

推荐将下面这种kv格式的configMap注入为容器的配置文件

[root@master01 yamls]# kubectl get cm cm-env-file -oyaml apiVersion: v1 data:   app.name: nginx   location.prefix: /app kind: ConfigMap metadata:   creationTimestamp: "2022-03-31T01:04:12Z"   name: cm-env-file   namespace: default   resourceVersion: "761496"   uid: 8e885d31-0a83-423c-a59f-6ea8b0473103 

创建dp,并将cm挂在进去

[root@master01 yamls]#  kubectl create deployment dp-test-cm --image=nginx --dry-run=client -oyaml > dp-test-cm.yaml 

修改

apiVersion: apps/v1 kind: Deployment metadata:   labels:     app: dp-test-cm   name: dp-test-cm spec:   replicas: 1   selector:     matchLabels:       app: dp-test-cm   template:     metadata:       labels:         app: dp-test-cm     spec:       containers:       - image: nginx         name: nginx         env:           # 直接定义ENV的kv           - name: Level             value: Pro           # 将cm中的指定key的值作为AppName这个环境变量的值           - name: AppName             valueFrom:               configMapKeyRef:                 name: cm-env-file                 key: app.name 

验证:

[root@master01 yamls]# kubectl get po NAME                          READY   STATUS    RESTARTS         AGE dp-test-cm-7c5666c5c4-wnhtk   1/1     Running   0                111s [root@master01 yamls]# kubectl exec -ti dp-test-cm-7c5666c5c4-wnhtk -- sh # echo $AppName nginx # echo $Level Pro 
1.2.2、envFrom

一次性配置多个环境变量

注意点:如果key不存在,不会影响pod的运用,但是也不会创建这个环境变量

apiVersion: apps/v1 kind: Deployment metadata:   labels:     app: dp-test-cm   name: dp-test-cm spec:   replicas: 1   selector:     matchLabels:       app: dp-test-cm   template:     metadata:       labels:         app: dp-test-cm     spec:       containers:       - image: nginx         name: nginx         # 和env同级,将redis-cm中的所有kv均作为环境变量注入到容器中         envFrom:         - configMapRef:             name: redis-cm         env:           - name: Level             value: Pro           - name: AppName             valueFrom:               configMapKeyRef:                 name: cm-env-file                 key: app.name 

验证:

[root@master01 yamls]# kubectl exec -ti dp-test-cm-6b7d8dc675-wf2vm -- sh # echo $ip 10.10.10.101 
1.2.3、volumeMounts

将基于文件创建的configMap以文件的形式挂在Pod的指定目录中

关键字:volume、volumeMounts

apiVersion: apps/v1 kind: Deployment metadata:   labels:     app: dp-test-cm   name: dp-test-cm spec:   replicas: 1   selector:     matchLabels:       app: dp-test-cm   template:     metadata:       labels:         app: dp-test-cm     spec:       volumes:         - name: brm-config           configMap:             name: cm-from-file       containers:       - image: nginx         name: nginx         volumeMounts:         - name: brm-config           mountPath: "/etc/foo" 

验证:可以看到它会将ConfigMap.data中的每一项,都作为一个单独的文件挂载进Pod中。

[root@master01 yamls]# kubectl get cm cm-from-file -oyaml apiVersion: v1 data:   nginx.conf: |     app.name=nginx     location.prefix=/app   redis.conf: |     ip=10.10.10.101     port=2379 kind: ConfigMap metadata:   creationTimestamp: "2022-03-31T00:52:21Z"   name: cm-from-file   namespace: default   resourceVersion: "759788"   uid: 5a41bdbe-66d0-45fa-8a5e-b6f6b005d671    [root@master01 yamls]# kubectl exec -ti dp-test-cm-68d77c6c6-4jtqr -- sh # cd /etc/foo # ls nginx.conf  redis.conf # cat nginx.conf app.name=nginx location.prefix=/app 
1.2.4、自定义文件名称

自定义挂载进Pod容器的文件名关键字:spec.spec.volumes.configMap.items

apiVersion: apps/v1 kind: Deployment metadata:   labels:     app: dp-test-cm   name: dp-test-cm spec:   replicas: 1   selector:     matchLabels:       app: dp-test-cm   template:     metadata:       labels:         app: dp-test-cm     spec:       volumes:         - name: brm-config           configMap:             name: cm-from-file             items:             - key: redis.conf               path: redis.conf.cus       containers:       - image: nginx         name: nginx         volumeMounts:         - name: brm-config           mountPath: "/etc/foo" 
1.2.5、控制文件权限

关键字:spec.template.spec.vulumes.configMap.defaultModespec.template.spec.vulumes.configMap.items.mode

用在控制挂载进Pod的文件的权限,---分别对应421(八进制)

apiVersion: apps/v1 kind: Deployment metadata:   labels:     app: dp-test-cm   name: dp-test-cm spec:   replicas: 1   selector:     matchLabels:       app: dp-test-cm   template:     metadata:       labels:         app: dp-test-cm     spec:       volumes:         - name: brm-config           configMap:             name: cm-from-file             items:             - key: redis.conf               path: redis.conf.cus               mode: 0111             defaultMode: 0666         containers:       - image: nginx         name: nginx         volumeMounts:         - name: brm-config           mountPath: "/etc/foo" 
1.2.6、子目录-subPath

关键字:subPath

subpath的作用是解决spec.template.spec.containers.volumeMounts.mountPath的挂载覆盖问题

意思就是说,如果mountPath指定的是一个目录,那么k8s会用我们期望挂载进该目录的文件覆盖掉该目录中已存在的文件,从而导致意外的结果。

解决方式如下的subPath,就是将cm 挂载成指定目录+指定文件,覆盖也是对指定目录中的指定文件进行覆盖,也就是下例中:将brm-config这个cm挂载进容器的/etc/foo/nginx.conf文件(覆盖也不会影响其他文件)。

apiVersion: apps/v1 kind: Deployment metadata:   labels:     app: dp-test-cm   name: dp-test-cm spec:   replicas: 1   selector:     matchLabels:       app: dp-test-cm   template:     metadata:       labels:         app: dp-test-cm     spec:       volumes:         - name: brm-config           configMap:             name: cm-from-file             items:             - key: redis.conf               path: redis.conf.cus               mode: 0111             defaultMode: 0666         containers:       - image: nginx         name: nginx         volumeMounts:         - name: brm-config           mountPath: "/etc/foo/nginx.conf"           subPath: "nginx.conf" 

这里可以做一个实验,比如在挂在之前先搞一个/etc/foo/1.txt,然后分别用subpath和不用subpath两种方式做挂载,对比/etc/foo/1.txt的是否被覆盖的差异~

1.2.7、热更新

cm中的值被编辑修改后,Pod中对应的挂载文件的值也会保持同步。

特殊情况:

env-fromvalue-from这种以环境变量的形式挂载进pod的cm,无法做到热更新

subPath也无法热更新

1.2.8、不可变的cm

与Kind同级,设置参数immutable=true即可

1.3、限制条件

  • cm收NameSpace限制,只有在同一个NameSpace中的Pod才能引用她
  • cm必须先于使用它的pod前创建
  • 只有被apiserver管理的Pod的才能使用cm,静态pod无法使用cm

二、Secret

Secret主要用户保管私密数据,比如密码、token、ssh keys等信息。这些信息直接放在镜像中一是不方便动态变动,而是数据比较敏感,不安全。

可以通过三种方式使用Secret

  • 通过挂在方式挂在Secret到Pod中使用它
  • 通过

2.1、创建

2.1.1、from-file
# 创建文件 [root@master01 echo -n 'admin' > ./username.txt [root@master01 echo -n 'qwerqwer123123' > ./password.txt  # 查看帮助文档 [root@master01 secret-file]# kubectl create secret --help  # 基于file创建secret,使用generic创建出的Secret类型为Opaque [root@master01 secret-file]# kubectl create secret generic db-user-info --from-file=./username.txt --from-file=./password.txt secret/db-user-info created 

查看

[root@master01 secret-file]# kubectl get secrets db-user-info -oyaml apiVersion: v1 data:   password.txt: cXdlcnF3ZXIxMjMxMjM=   username.txt: YWRtaW4= kind: Secret metadata:   creationTimestamp: "2022-04-05T23:08:11Z"   name: db-user-info   namespace: default   resourceVersion: "837054"   uid: eabe607c-27fc-4611-92c6-fce4cc24f030 type: Opaque 

data中的数据经base64加密存储,可直接解密

[root@master01 secret-file]# echo "cXdlcnF3ZXIxMjMxMjM=" | base64  -d qwerqwer123123 
2.1.2、from-literal

基于字面意思上的命令行中指定好key和value创建Secret

[root@master01 secret-file]# kubectl create secret generic secret-from-literal --from-literal=username=admin --from-literal=passsword=123456 secret/secret-from-literal created 

查看

[root@master01 secret-file]# kubectl get secrets secret-from-literal -oyaml apiVersion: v1 data:   passsword: MTIzNDU2   username: YWRtaW4= kind: Secret metadata:   creationTimestamp: "2022-04-05T23:19:02Z"   name: secret-from-literal   namespace: default   resourceVersion: "838616"   uid: ebb95a99-8cbf-48fd-a651-60e12cd30110 type: Opaque 
2.1.3、基于yaml文件创建
  • 需要将data使用base64提前加密好,必须基于秘文创建
  • 也可以使用stringData基于明文创建secret
apiVersion: v1 kind: Secret metadata:   name: my-secret-from-yaml   namespace: default type: Opaque data:   username: YWRtaW4=   password: MWYyZDFlMmU2N2Rm 

创建

[root@master01 secret-file]# kubectl apply -f secret-user-info.yaml secret/my-secret-from-yaml created 

查看

[root@master01 secret-file]# kubectl get secrets my-secret-from-yaml -o yaml apiVersion: v1 data:   password: MWYyZDFlMmU2N2Rm   username: YWRtaW4= kind: Secret metadata:   annotations:     kubectl.kubernetes.io/last-applied-configuration: |       {"apiVersion":"v1","data":{"password":"MWYyZDFlMmU2NRm","username":"YWRW4="},"kind":"Secret","metadata":{"annotations":{},"name":"my-secret-from-yaml","namespace":"default"},"type":"Opaque"}   creationTimestamp: "2022-04-05T23:28:27Z"   name: my-secret-from-yaml   namespace: default   resourceVersion: "839971"   uid: eb3acb4d-edf2-4ac0-9373-e7a931a1a559 type: Opaque 

基于明文创建

使用stringData关键字可以基于明文创建,因为不安全所以不推荐。

apiVersion: v1 kind: Secret metadata:   name: secret-basic-auth type: kubernetes.io/basic-auth stringData:   username: admin      #  kubernetes.io/basic-auth 类型的必需字段   password: t0p-Secret # kubernetes.io/basic-auth 类型的必需字段 
2.1.4、from-env-file

用法和cm的from-env-file类似

[root@master01 secret-file]# cat ./secret-env-file.txt username=admin password=123456  [root@master01 secret-file]# kubectl get secrets secret-from-env -oyaml apiVersion: v1 data:   password: MTIzNDU2   username: YWRtaW4= kind: Secret metadata:   creationTimestamp: "2022-04-05T23:39:47Z"   name: secret-from-env   namespace: default   resourceVersion: "841608"   uid: efaf7ad6-2ded-4544-93f8-5efabd2ca449 type: Opaque 

2.2、实战

2.2.1、配置阿里云私有仓库密钥

系统整理K8S的配置管理实战-建议收藏系列

给kubelet配置私有云下载Docker镜像的账号密码

# 查看帮助文档 [root@master01 secret-file]# kubectl create secret docker-registry -h  # 将username、password、email修改成自己的 [root@master01 secret-file]# kubectl create secret docker-registry brm-alicloud-docker-secret --docker-username=xxxx --docker-password=xxx --docker-email=xxx@qq.com  --docker-server=registry.cn-hangzhou.aliyuncs.com 

查看(data中的dockerconfigjson) 同样可以使用base64解码看到一个json串

[root@master01 secret-file]# kubectl get secrets brm-alicloud-docker-secret -oyaml apiVersion: v1 data:   .dockerconfigjson: eyJhdXRocyI6eyJyZWdpc3RyeS5jbi1oYW5nemhvdS5hbGl5dW5jcy5jb20iOnsidXNlcm5hbWUiOiLmnLHmmIzmraYyMjMzIiwicGFzc3dvcmQiOiJxd2VyMTAxMC4uIiwiZW1haWwiOiI2NDY0NTAzMDhAcXEuY29tIiwiYXV0aCI6IjVweXg1cGlNNXEybU1qSXpNenB4ZDJWeU1UQXhNQzR1In19fQ== kind: Secret metadata:   creationTimestamp: "2022-04-06T00:04:17Z"   name: brm-alicloud-docker-secret   namespace: default   resourceVersion: "845134"   uid: 6e7bd04e-621c-4da8-8649-99c0ebffcee9 type: kubernetes.io/dockerconfigjson 

挂载,先搞一个模版dp

[root@master01 secret-file]# kubectl create deployment alicloud-private-nginx --image=registry.cn-hangzhou.aliyuncs.com/changwu/nginx:1.7.9-nettools --dry-run=client -oyaml 

修改添加imagePullSecrets配置

apiVersion: apps/v1 kind: Deployment metadata:   creationTimestamp: null   labels:     app: alicloud-private-nginx   name: alicloud-private-nginx spec:   replicas: 1   selector:     matchLabels:       app: alicloud-private-nginx   strategy: {}   template:     metadata:       creationTimestamp: null       labels:         app: alicloud-private-nginx     spec:       imagePullSecrets:       - name: brm-alicloud-docker-secret       containers:       - image: registry.cn-hangzhou.aliyuncs.com/changwu/nginx:1.7.9-nettools         name: nginx         resources: {} status: {} 

验证的话就去 kubectl get pod -owide即可

2.2.2、管理https证书

生成私钥和crt

[root@master01 https]# openssl req -x509 -nodes -days 3650  > -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=test.com" Generating a 2048 bit RSA private key ....................................................................+++ ...........+++ writing new private key to 'tls.key' ----- [root@master01 https]# ls tls.crt  tls.key 

将key和cry托管进secret

[root@master01 https]# kubectl create secret tls https-nginx-tls-test-secret -n default --key=tls.key --cert=tls.crt secret/https-nginx-tls-test-secret created 

查看

[root@master01 https]# kubectl get secrets https-nginx-tls-test-secret -oyaml apiVersion: v1 data:   tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1R1lrWTJRCjN4VUpQV1hVR0lqM2VVYnhTc0hFZ2lyZDlJalBZL1pwZEsxWittbmNHTWMyNW41aVhoUEs1UG1ZcjYrcAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==   tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1V3AzbUtRNHJkZG1CZGplMjJpMjR1bU5IUGd2STJBRGlEalAKNXBTb2hRYWRkZVZYRDJFZWdlWFRTTHY3Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0K kind: Secret metadata:   creationTimestamp: "2022-04-06T00:25:33Z"   name: https-nginx-tls-test-secret   namespace: default   resourceVersion: "848213"   uid: afa127bb-4a3a-4eb9-9d55-8cd50af5b3f4 type: kubernetes.io/tls 

配置到ingress中(不是很推荐直接在ingress中配置https,通常会将https配置在ingress的上一层的SLB代理中)

apiVersion: networking.k8s.io/v1 kind: Ingress metadata:   name: simple-tls-ingress spec:   ingressClassName: nginx   rules:   - host: https-test.com # 配置域名,可以不写,匹配*,或者写 *.bar.com     http:       paths: # 相当于nginx的location,同一个host可以配置多个path       - path: /         pathType: Prefix         backend:           service:             name: nginx-svc             port:               number: 80   tls:    - secretName: https-nginx-tls-test-secret 

验证

系统整理K8S的配置管理实战-建议收藏系列

2.2.3、不可变的secret

与Kind同级,设置参数immutable=true即可

发表评论

相关文章