目 录CONTENT

文章目录

containerd 配置从私有仓库拉取镜像

Seven
2022-10-13 / 0 评论 / 0 点赞 / 2796 阅读 / 14265 字 / 正在检测是否收录...

官方地址:

https://github.com/containerd/cri/blob/master/docs/registry.md
少走弯路,尽量看官方文档。

containerd 使用了类似 K8S 中 svc 与 endpoint 的概念,svc 可以理解为访问名称,这个名称会解析到对应的 endpoint 上。也可以理解 mirror 配置就是一个反向代理,它把客户端的请求代理到 endpoint 配置的后端镜像仓库。mirror 名称可以随意填写,但是必须符合IP或域名的定义规则。并且可以配置多个 endpoint,默认解析到第一个 endpoint,如果第一个 endpoint 没有返回数据,则自动切换到第二个 endpoint,以此类推。

比如以下配置示例:

mirrors:
  "172.31.6.200:5000":
    endpoint:
      - "http://172.31.6.200:5000"
  "rancher.ksd.top:5000":
    endpoint:
      - "http://172.31.6.200:5000"
  "docker.io":
    endpoint:
      - "https://fogjl973.mirror.aliyuncs.com"
      - "https://registry-1.docker.io"

可以通过

crictl pull 172.31.6.200:5000/library/alpine 
crictl pull rancher.ksd.top:5000/library/alpine

获取到镜像,但镜像都是从同一个仓库获取到的。

root@rancher-server:/etc/rancher/k3s# systemctl restart k3s.service
root@rancher-server:/etc/rancher/k3s# crictl pull 172.31.6.200:5000/library/alpine
Image is up to date for sha256:a24bb4013296f61e89ba57005a7b3e52274d8edd3ae2077d04395f806b63d83e
root@rancher-server:/etc/rancher/k3s# crictl pull rancher.ksd.top:5000/library/alpine
Image is up to date for sha256:a24bb4013296f61e89ba57005a7b3e52274d8edd3ae2077d04395f806b63d83e
root@rancher-server:/etc/rancher/k3s#

非安全(http)私有仓库配置

配置非安全(http)私有仓库,只需要在 endpoint 中指定 http 协议头的地址即可。

在没有 TLS 通信的情况下,需要为 endpoints 指定http:// ,否则将默认为 https。

无认证

如果你使用的是非安全(http)私有仓库,那么可以通过下面的参数来配置 K3s 连接私有仓库:

root@ip-172-31-13-117:~# cat >> /etc/rancher/k3s/registries.yaml <<EOF
mirrors:
  "172.31.6.200:5000":
    endpoint:
      - "http://172.31.6.200:5000"
EOF
systemctl restart k3s

然后可以通过 crictl 去 pull 镜像:

root@ip-172-31-13-117:~# crictl pull 172.31.6.200:5000/my-ubuntu
Image is up to date for sha256:9499db7817713c4d10240ca9f5386b605ecff7975179f5a46e7ffd59fff462ee

接下来,在看一下 containerd 的配置,可以看到文件末尾追加了如下配置:

root@ip-172-31-13-117:~# cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml
[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."172.31.6.200:5000"]
  endpoint = ["http://172.31.6.200:5000"]
[plugins.cri.registry.mirrors."rancher.ksd.top:5000"]
  endpoint = ["http://172.31.6.200:5000"]
有认证

如果你的非安全(http)私有仓库带有认证,那么可以通过下面的参数来配置 k3s 连接私有仓库:

mirrors:
  "35.182.134.80":
    endpoint:
      - "http://35.182.134.80"
configs:
  "35.182.134.80":
    auth:
      username: admin # this is the registry username
      password: Harbor12345 # this is the registry password
EOF
systemctl restart k3s

通过 crictl 去 pull 镜像:

root@ip-172-31-13-117:~# crictl pull 35.182.134.80/ksd/ubuntu:16.04
Image is up to date for sha256:9499db7817713c4d10240ca9f5386b605ecff7975179f5a46e7ffd59fff462ee

Containerd 配置文件末尾追加了如下配置:

[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."35.182.134.80"]
  endpoint = ["http://35.182.134.80"]
[plugins.cri.registry.configs."35.182.134.80".auth]
  username = "admin"
  password = "Harbor12345"

新版本containerd 1.5.13 配置文件和老版有一些改动

[plugins."io.containerd.grpc.v1.cri".registry]

  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
     endpoint = ["https://fogjl973.mirror.aliyuncs.com", "https://registry-1.docker.io"]

  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."47.96.74.152:5100"]
     endpoint = ["http://47.96.74.152:5100"]
[plugins."io.containerd.grpc.v1.cri".registry.configs]
  [plugins."io.containerd.grpc.v1.cri".registry.configs."47.96.74.152:5100".tls]
     insecure_skip_verify = true
  [plugins."io.containerd.grpc.v1.cri".registry.configs."47.96.74.152:5100".auth]
     username = "xxxxx"
     password = "xxxxx"

在最新的2.0版本中,配置又有了新的改动:

1、 添加新的私有仓库在config.toml里面只添加loadpath:

config.toml
    [plugins."io.containerd.grpc.v1.cri".registry]
      config_path = "/etc/containerd/certs.d" #这一步就是添加私有仓库的配置路径。

      [plugins."io.containerd.grpc.v1.cri".registry.auths]

2、到路径里去新建目录

mkdir certs.d
ls /etc/containerd
ubuntu@master:/etc/containerd$ ll
total 24
drwxr-xr-x   3 root root 4096 Feb 27 15:37 ./
drwxr-xr-x 115 root root 4096 Apr 24 06:54 ../
drwxr-xr-x   4 root root 4096 Feb 26 18:32 certs.d/
-rw-rw-r--   1 root root 8434 Feb 26 18:02 config.toml

3、新建公有或者是私有仓库目录

mkdir 47.96.74.152:5100
mkdir docker.io
ubuntu@master:/etc/containerd/certs.d$ ll
total 16
drwxr-xr-x 4 root root 4096 Feb 26 18:32 ./
drwxr-xr-x 3 root root 4096 Feb 27 15:37 ../
drwxr-xr-x 2 root root 4096 Feb 27 11:02 47.96.74.152:5100/
drwxr-xr-x 2 root root 4096 Feb 26 18:10 docker.io/

4、进入私有仓库目录47.96.74.152:5100配置-》hosts.toml

  • 这个文件名必须是hosts.toml(坑点)

  • 内容:

    server = "http://47.96.74.152:5100"
    
    [host."http://47.96.74.152:5100"]
      capabilities = ["pull","resolve","push"]
      skip_verify = true
      scheme = "http"
      [header]
        Authorization = "Basic amVua2lxxxxxxxxxxxxxxS2E0" #账号密码Base64编码。添加到header头进行验证。获取方法 echo -n "账号:密码" | base64
    
    

    到此私有仓库配置结束

  • 如果还想配置其它公共仓库比如:docker.io、阿里云

  • 在certs.d目录继续添加目录docker.io

  • 进入目录新建hosts.toml这个相当于docker 添加多个公共仓库源。

    server = "https://docker.io"
    
    [host."https://mirror.baidubce.com"]
      capabilities = ["pull","resolve"]
    [host."https://docker.m.daocloud.io"]
      capabilities = ["pull","resolve"]
    [host."https://hbv0b596.mirror.aliyuncs.com"]
      capabilities = ["pull","resolve"]
    [host."https://hub-mirror.c.163.com"]
      capabilities = ["pull","resolve"]
    [host."https://docker.mirrors.ustc.edu.cn"]
      capabilities = ["pull","resolve"]
    
    

    最后通过crictl进行镜像拉取测试。

安全(https)私有仓库配置

以下示例均启用了认证,所以每个示例都配置了configs.auth,如果实际环境未配置认证,删除configs.auth配置即可。

使用授信 ssl 证书
与非安全(http)私有仓库配置类似,只需要配置 endpoint 对应的仓库地址为 https 即可。

root@ip-172-31-13-117:~# cat >> /etc/rancher/k3s/registries.yaml <<EOF
mirrors:
  "harbor.kingsd.top":
    endpoint:
      - "https://harbor.kingsd.top"
configs:
  "harbor.kingsd.top":
    auth:
      username: admin # this is the registry username
      password: Harbor12345 # this is the registry password
EOF
systemctl restart k3s

通过 crictl 去 pull 镜像:

root@ip-172-31-13-117:~# crictl pull harbor.kingsd.top/ksd/ubuntu:16.04
Image is up to date for sha256:9499db7817713c4d10240ca9f5386b605ecff7975179f5a46e7ffd59fff462ee

Containerd 配置文件末尾追加了如下配置:

root@ip-172-31-13-117:~# cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml
[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."harbor.kingsd.top"]
  endpoint = ["https://harbor.kingsd.top"]
[plugins.cri.registry.configs."harbor.kingsd.top".auth]
  username = "admin"
  password = "Harbor12345"
使用自签 ssl 证书

如果后端仓库使用的是自签名的 ssl 证书,那么需要配置 CA 证书 用于 ssl 证书的校验。

mirrors:
  "harbor-ksd.kingsd.top":
    endpoint:
      - "https://harbor-ksd.kingsd.top"
configs:
  "harbor-ksd.kingsd.top":
    auth:
      username: admin # this is the registry username
      password: Harbor12345 # this is the registry password
    tls:
      ca_file: /opt/certs/ca.crt
EOF
systemctl restart k3s

通过 crictl 去 pull 镜像:

root@ip-172-31-13-117:~# crictl pull harbor-ksd.kingsd.top/ksd/ubuntu:16.04
Image is up to date for sha256:9499db7817713c4d10240ca9f5386b605ecff7975179f5a46e7ffd59fff462ee

Containerd 配置文件末尾追加了如下配置:

root@ip-172-31-13-117:~# cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml
[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."harbor-ksd.kingsd.top"]
  endpoint = ["https://harbor-ksd.kingsd.top"]
[plugins.cri.registry.configs."harbor-ksd.kingsd.top".auth]
  username = "admin"
  password = "Harbor12345"
[plugins.cri.registry.configs."harbor-ksd.kingsd.top".tls]
  ca_file = "/opt/certs/ca.crt"
ssl 双向认证

如果镜像仓库配置了双向认证,那么需要为 containerd 配置 ssl 证书用于 镜像仓库对 containerd 做认证。

root@ip-172-31-13-117:~# cat >> /etc/rancher/k3s/registries.yaml <<EOF
mirrors:
  "harbor-ksd.kingsd.top":
    endpoint:
      - "https://harbor-ksd.kingsd.top"
configs:
  "harbor-ksd.kingsd.top":
    auth:
      username: admin # this is the registry username
      password: Harbor12345 # this is the registry password
    tls:
      ca_file: /opt/certs/ca.crt # path to the ca file used in the registry
      cert_file: /opt/certs/harbor-ksd.kingsd.top.cert # path to the cert file used in the registry
      key_file: /opt/certs/harbor-ksd.kingsd.top.key # path to the key file used in the registry
EOF
systemctl restart k3s

通过 crictl 去 pull 镜像:

root@ip-172-31-13-117:~# crictl pull harbor-ksd.kingsd.top/ksd/ubuntu:16.04
Image is up to date for sha256:9499db7817713c4d10240ca9f5386b605ecff7975179f5a46e7ffd59fff462ee

Containerd 配置文件末尾追加了如下配置:

[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."harbor-ksd.kingsd.top"]
  endpoint = ["https://harbor-ksd.kingsd.top"]
[plugins.cri.registry.configs."harbor-ksd.kingsd.top".auth]
  username = "admin"
  password = "Harbor12345"
[plugins.cri.registry.configs."harbor-ksd.kingsd.top".tls]
  ca_file = "/opt/certs/ca.crt"
  cert_file = "/opt/certs/harbor-ksd.kingsd.top.cert"
  key_file = "/opt/certs/harbor-ksd.kingsd.top.key"

加速器配置

Containerd 与 docker 都有默认仓库,均为 docker.io 。如果配置中未指定 mirror 为 docker.io,containerd 后会自动加载 docker.io 配置。与 docker 不同的是,containerd 可以修改 docker.io 对应的 endpoint(默认为 https://registry-1.docker.io ) ,而 docker 无法修改。

Docker 中可以通过 registry-mirrors 设置镜像加速地址。如果 pull 的镜像不带仓库地址(项目名+镜像名:tag),则会从默认镜像仓库去拉取镜像。如果配置了镜像加速地址,会先访问镜像加速仓库,如果没有返回数据,再访问默认的镜像仓库。

Containerd 目前没有直接配置镜像加速的功能,但 containerd 中可以修改 docker.io 对应的 endpoint,所以可以通过修改 endpoint 来实现镜像加速下载。因为 endpoint 是轮询访问,所以可以给 docker.io 配置多个仓库地址来实现 加速地址+默认仓库地址。如下配置示例:

mirrors:
  "docker.io":
    endpoint:
      - "https://fogjl973.mirror.aliyuncs.com"
      - "https://registry-1.docker.io"
EOF
systemctl restart k3s

Containerd 配置文件末尾追加了如下配置:

root@ip-172-31-13-117:~# cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml
[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."docker.io"]
  endpoint = ["https://fogjl973.mirror.aliyuncs.com", "https://registry-1.docker.io"]

完整配置示例

  "192.168.50.119":
    endpoint:
      - "http://192.168.50.119"
  "docker.io":
    endpoint:
      - "https://fogjl973.mirror.aliyuncs.com"
      - "https://registry-1.docker.io"
configs:
  "192.168.50.119":
    auth:
      username: '' # this is the registry username
      password: '' # this is the registry password
    tls:
      cert_file: '' # path to the cert file used in the registry
      key_file: '' # path to the key file used in the registry
      ca_file: '' # path to the ca file used in the registry
  "docker.io":
    auth:
      username: '' # this is the registry username
      password: '' # this is the registry password
    tls:
      cert_file: '' # path to the cert file used in the registry
      key_file: '' # path to the key file used in the registry
      ca_file: '' # path to the ca file used in the registry
0

评论区