Apparently we had run out of disk space on the node. I couldn't recover from that even after clearing space and deleting /var/lib/docker. Had to throw away the node and start again. Wonder if there was a better way of handling this.
On 10 October 2016 at 12:12, Lionel Orellana <lione...@gmail.com> wrote: > Something broke docker. > > # docker run hello-world > docker: Error response from daemon: devicemapper: Error running > deviceCreate (createSnapDevice) dm_task_run failed. > See '/usr/bin/docker-current run --help'. > # > > # docker info > Containers: 1 > Running: 0 > Paused: 0 > Stopped: 1 > Images: 283 > Server Version: 1.10.3 > Storage Driver: devicemapper > Pool Name: docker-0:38-148510880-pool > Pool Blocksize: 65.54 kB > Base Device Size: 10.74 GB > Backing Filesystem: xfs > Data file: /dev/loop0 > Metadata file: /dev/loop1 > Data Space Used: 38.19 GB > Data Space Total: 107.4 GB > Data Space Available: 16.89 GB > Metadata Space Used: 45.18 MB > Metadata Space Total: 2.147 GB > Metadata Space Available: 2.102 GB > Udev Sync Supported: true > Deferred Removal Enabled: false > Deferred Deletion Enabled: false > Deferred Deleted Device Count: 0 > Data loop file: /var/lib/docker/devicemapper/devicemapper/data > WARNING: Usage of loopback devices is strongly discouraged for production > use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt > dm.no_warn_on_loop_devices=true` to suppress this warning. > Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata > Library Version: 1.02.107-RHEL7 (2016-06-09) > Execution Driver: native-0.2 > Logging Driver: json-file > Plugins: > Volume: local > Network: bridge null host > Kernel Version: 3.10.0-327.28.3.el7.x86_64 > Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo) > OSType: linux > Architecture: x86_64 > Number of Docker Hooks: 2 > CPUs: 1 > Total Memory: 3.703 GiB > Name: xxxxxx > ID: S7TX:CAUS:656Z:TLJN:IS2F:CICD:RISE:7N76:MRSA:HUBP:MINL:WU7M > Debug mode (server): true > File Descriptors: 16 > Goroutines: 24 > System Time: 2016-10-10T12:11:00.561344233+11:00 > EventsListeners: 0 > Init SHA1: 1f87ef7fa7fd7401b9aa61ca3a3e096b4fd2228e > Init Path: > Docker Root Dir: /var/lib/docker > WARNING: bridge-nf-call-iptables is disabled > WARNING: bridge-nf-call-ip6tables is disabled > Registries: docker.io (secure) > > > > On 10 October 2016 at 10:36, Lionel Orellana <lione...@gmail.com> wrote: > >> I deleted all containers with >> >> docker rm -v $(docker ps -q -f status=exited) >> >> The openvswitch service continous to fail claiming the name already >> exists. >> >> Oct 10 10:30:28 poc-docker02.aipo.gov.au openvswitch[17706]: docker: >> Error response from daemon: Conflict. The name "/openvswitch" is already in >> use by container b9a94ed6a5885c5b02f9fe456b27e4 >> 09c4ce2732a3e7be1bce19d8d69f397d2d. You have to remove (or rename) that >> container to be able to reuse that name.. >> Oct 10 10:30:28 poc-docker02.aipo.gov.au openvswitch[17706]: See >> '/usr/bin/docker-current run --help'. >> -- >> >> # docker ps -a >> CONTAINER ID IMAGE COMMAND CREATED >> STATUS PORTS NAMES >> # >> >> >> >> On 10 October 2016 at 09:44, Lionel Orellana <lione...@gmail.com> wrote: >> >>> Hi, >>> >>> I'm getting this error on my nodes after the host has rebooted. >>> >>> Oct 10 09:22:13 poc-docker02.aipo.gov.au systemd[1]: Job >>> origin-node.service/start failed with result 'dependency'. >>> Oct 10 09:22:23 poc-docker02.aipo.gov.au systemd[1]: Dependency failed >>> for origin-node.service. >>> -- Subject: Unit origin-node.service has failed >>> -- Defined-By: systemd >>> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel >>> -- >>> -- Unit origin-node.service has failed. >>> -- >>> -- The result is dependency. >>> >>> No container is running. (i.e. docker ps returns nothing but -a does). >>> >>> It seems to be failing to create openvswitch because it already exists? >>> Should I delete old containers? Any advice on getting a clean system >>> reboot? >>> >>> $journalct -xe >>> >>> Oct 10 09:37:47 poc-docker02.aipo.gov.au docker[7343]: >>> time="2016-10-10T09:37:47.353549643+11:00" level=debug msg="Calling >>> POST /v1.22/containers/create" >>> Oct 10 09:37:47 poc-docker02.aipo.gov.au docker[7343]: >>> time="2016-10-10T09:37:47.353603207+11:00" level=debug msg="POST >>> /v1.22/containers/create?name=openvswitch" >>> Oct 10 09:37:47 poc-docker02.aipo.gov.au docker[7343]: >>> time="2016-10-10T09:37:47.353898488+11:00" level=debug msg="form data: >>> {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\ >>> ":true,\"Cmd\":null,\"Domainname\":\"\",\"Entry >>> Oct 10 09:37:47 poc-docker02.aipo.gov.au docker[7343]: >>> time="2016-10-10T09:37:47.354052936+11:00" level=info >>> msg="{Action=create, LoginUID=4294967295, PID=9878}" >>> Oct 10 09:37:47 poc-docker02.aipo.gov.au docker[7343]: >>> time="2016-10-10T09:37:47.359950019+11:00" level=error msg="Handler for >>> POST /v1.22/containers/create returned error: Conflict. The name >>> \"/openvswitch\" is already in use by con >>> Oct 10 09:37:47 poc-docker02.aipo.gov.au openvswitch[9878]: *docker: >>> Error response from daemon: Conflict. The name "/openvswitch" is already in >>> use by container *497532ea92a4e14782b96b15f2763d >>> 3883d2843d5fdf473f720170756e4de815. You ha >>> Oct 10 09:37:47 poc-docker02.aipo.gov.au openvswitch[9878]: See >>> '/usr/bin/docker-current run --help'. >>> Oct 10 09:37:47 poc-docker02.aipo.gov.au systemd[1]: >>> openvswitch.service: main process exited, code=exited, status=125/n/a >>> Oct 10 09:37:50 poc-docker02.aipo.gov.au polkitd[847]: Registered >>> Authentication Agent for unix-process:9885:60099516 (system bus name >>> :1.3589 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path >>> /org/freedesktop/PolicyKit1/Au >>> Oct 10 09:37:50 poc-docker02.aipo.gov.au systemd[1]: Cannot add >>> dependency job for unit origin-master.service, ignoring: Unit >>> origin-master.service failed to load: No such file or directory. >>> Oct 10 09:37:50 poc-docker02.aipo.gov.au systemd[1]: Cannot add >>> dependency job for unit origin-master.service, ignoring: Unit >>> origin-master.service failed to load: No such file or directory. >>> Oct 10 09:37:50 poc-docker02.aipo.gov.au systemd[1]: Started >>> origin-node-dep.service. >>> >>> >>> thanks >>> >>> >>> Lionel >>> >> >> >
_______________________________________________ users mailing list users@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/users
