标题：[实践OK]CentOS7安装最新Docker，CentOS下的Docker DeviceMapper占用空间太大解决方案
出处：向东博客 专注WEB应用 构架之美 --- 构架之美，在于尽态极妍 | 应用之美，在于药到病除
时间：Wed, 05 Dec 2018 18:24:01 +0000
作者：jackxiang
地址：http://jackxiang.com/post/9978/

内容：
CentOS7安装最新Docker：

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo  #make clean all
yum remove docker  docker-common docker-selinux docker-engine
yum list docker-ce --showduplicates | sort -r  #docker-ce-20.10.14 最新
docker-ce.x86_64            3:20.10.2-3.el7                     docker-ce-stable
docker-ce.x86_64            3:20.10.14-3.el7                    docker-ce-stable

yum install docker-ce-20.10.14 -y


来自：https://www.cnblogs.com/hongshaozi/p/12744483.html
对于二的实践发现，刚开始目录里没有东西：/var/lib/docker/devicemapper/devicemapper ，但：systemctl  start docker后就有了：
/var/lib/docker/devicemapper/devicemapper
data  metadata
成功迁移到/data盘：/var/lib/docker/devicemapper/devicemapper -> /data/docker/devicemapper/devicemapper  #docker-ce-20.10.14版本

刚开始docker目录也没有，也得等systemctl  start docker后才会有，进而写入内容：/etc/docker/daemon.json

二、CentOS下的Docker DeviceMapper占用空间太大解决方案


步骤一：
rsync -avz /var/lib/docker /data/
systemctl stop docker

步骤二：
进入/etc/docker文件编辑daemon.json，若改文件不存在则创建改文件；
touch daemon.json 
将以下配置写入daemon.json
/etc/docker/daemon.json
{
  "data-root": "/var/lib/docker",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "512m",
    "max-file": "20",
    "compress": "true"
  }
}

#cat daemon.json 
{
  "data-root": "/data/docker",
  "storage-driver": "overlay2",
  "storage-opts": [
      "overlay2.override_kernel_check=true"
   ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "512m",
    "max-file": "20",
    "compress": "true"
  }
}

方式二成功：
vim /usr/lib/systemd/system/docker.service​
​ExecStart=/usr/bin/dockerd --graph=/data/docker​  #
ExecStart=/usr/bin/dockerd --data-root="/data/docker" -H fd:// --containerd=/run/containerd/containerd.sock
#systemctl start docker
Warning: The unit file, source configuration file or drop-ins of docker.service changed on disk. Run 'systemctl daemon-reload' to reload units.
systemctl daemon-reload
systemctl start docker

步骤三：
​systemctl daemon-reload​​
systemctl restart docker​
overlayfs: upper fs missing required features.  ZFS文件系统不行

msg="The \"graph\" config file option is deprecated. Please u
docker修改默认存储存储路径 The "graph" config file option is deprecated. Please use "data-root" instead.


docker info​​
看到 Docker Root Dir 就是docker存放数据的根目录
 Docker Root Dir: /var/lib/docker


验证日志大小，输入：
docker inspect -f {{.HostConfig.LogConfig}} docker-compose_FW_1
输出：
{json-file map[compress:true max-file:20 max-size:256m]}

docker下的log日志过大问题，日志太大的问题：
https://blog.csdn.net/czf19950601/article/details/125064452


/var/lib/docker/devicemapper/devicemapper/data
scp /var/lib/docker/devicemapper/devicemapper/data root@10.71.165.136:/var/lib/docker/devicemapper/devicemapper/data  #这个方法不行，看有2G，一拷贝有20G
步骤1）挂载盘：
rm -rf /var/lib/docker/
mkdir -p /var/lib/docker/devicemapper/
mkdir -p  /data/docker/devicemapper/devicemapper     
ln -sf /data/docker/devicemapper/devicemapper /var/lib/docker/devicemapper/devicemapper
ll /var/lib/docker/devicemapper/devicemapper
lrwxrwxrwx 1 root root 38 Dec  5 22:35 /var/lib/docker/devicemapper/devicemapper -> /data/docker/devicemapper/devicemapper

步骤2）卸载重装Docker：
yum remove docker-ce
yum install docker-ce
systemctl start docker 
docker images
cd /lib/systemd/system
mv docker.service docker.service.init0
mv docker.service.proxy.ok docker.service
systemctl daemon-reload 
systemctl restart docker 


步骤3）检查是否给把原来的大的挂载磁盘给软链接进来了：

#du -sh /var/lib/docker/devicemapper/devicemapper/data
11M     /var/lib/docker/devicemapper/devicemapper/data

[root@docker_build_bj_sjs_10_71_159_11:~]
#ls -lart /var/lib/docker/devicemapper/devicemapper
lrwxrwxrwx 1 root root 38 Dec  5 22:35 /var/lib/docker/devicemapper/devicemapper -> /data/docker/devicemapper/devicemapper

附录：docker.service

#cat /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
# 翻墙的测试Docker服务器@10.71.159.168  registry.qr.cntv.net #私有IP：10.71.165.137
Environment="HTTPS_PROXY=http://10.70.36.30:8080" "NO_PROXY=localhost,127.0.0.1,registry.qr.cntv.net"
#ExecStart=/usr/bin/dockerd
ExecStart=/usr/bin/dockerd --registry-mirror=http://gcr.io
#ExecStart=/usr/bin/dockerd   --insecure-registry registry.qr.cntv.net:80
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
#TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s

[Install]
WantedBy=multi-user.target




5.1G    /var/lib/docker/devicemapper/devicemapper/data  #这个可能38G，太大，40G盘存不下来。

都有可能。得按方法二，来处：https://blog.csdn.net/CHENYUFENG1991/article/details/79839497

解决方案2：成功
停止docker!!!这一步很关键，否则下面的设置会失败。
编辑以下文件：
vim /lib/systemd/system/docker.service

原内容如下：
ExecStart=/usr/bin/dockerd-current \
          --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current \
          --default-runtime=docker-runc \
          --exec-opt native.cgroupdriver=systemd \
          --userland-proxy-path=/usr/libexec/docker/docker-proxy-current \
          $OPTIONS \
          $DOCKER_STORAGE_OPTIONS \
          $DOCKER_NETWORK_OPTIONS \
          $ADD_REGISTRY \
          $BLOCK_REGISTRY \
          $INSECURE_REGISTRY\
          $REGISTRIES


然后修改为以下：
ExecStart=/usr/bin/dockerd-current \
          --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current \
          --default-runtime=docker-runc \
          --exec-opt native.cgroupdriver=systemd \
          --userland-proxy-path=/usr/libexec/docker/docker-proxy-current \
          --storage-opt dm.loopdatasize=8G \
          --storage-opt dm.loopmetadatasize=4G \
          --storage-opt dm.basesize=8G \
          $OPTIONS \
          $DOCKER_STORAGE_OPTIONS \
          $DOCKER_NETWORK_OPTIONS \
          $ADD_REGISTRY \
          $BLOCK_REGISTRY \
          $INSECURE_REGISTRY\
          $REGISTRIES


也就是多加以下3行：
          --storage-opt dm.loopdatasize=8G \
          --storage-opt dm.loopmetadatasize=4G \
          --storage-opt dm.basesize=8G \

设置devicemapper的data为8G，metadata为4G，镜像的大小不能大于8G。

删除原有docker，并使用dd命令进行空间分配
rm -rf /var/lib/docker 
mkdir -p /var/lib/docker/devicemapper/devicemapper/ 
dd if=/dev/zero of=/var/lib/docker/devicemapper/devicemapper/data bs=1M count=0 seek=8192
dd if=/dev/zero of=/var/lib/docker/devicemapper/devicemapper/metadata bs=1M count=0 seek=4096

完成上述步骤后
systemctl daemon-reload 
systemctl start docker 
docker info




Generated by Jackxiang's Bo-blog 2.1.1 Release