docker.service start failed

[EUROKEY]

Hello,

our netPi stopped working, the docker.service does not start up anymore. (docker.service: Failed with result 'protocol')
How can we fix this? WebInterface is working. Firmware version V1.2.0.0
Any help would be welcome. What may produce this? Power failure?
syslog of boot process attached.



Best
Manuel

[Armin@netPI]

Hello Manuel,

I informed the development team to look to the log file.

But could you tell me more when this effect started to happen. Like ... the unit was working all day long ... all of a sudden the device didn't start docker any more after a next power cycle. Or like ... I pull and started a new container and after that the docker service didn't start any more.

Thx
Armin

[EUROKEY]

Good Morning Armin,

We powered off the device by disconnecting the power source. On the next day we reconnected it and found the service no longer being functional.
We didn't pull the container itself, we only work remotely via ssh in the container.

HTH,
Manuel

[Armin@netPI]

Hello Manuel,

results from the development team are still pending. I looked into the log file myself and I see Docker not starting as well.

I see "unable to configure the Docker daemon with file /etc/docker/daemon.json: EOF" as first indication that something went wrong. Apparently this file is needed but it seems to be empty. The file format is officially explained here https://docs.docker.com/engine/reference...e/dockerd/ and configures the Docker Daemon.

I would expect nobody on the system is ever writing to this file, so I am wondering why it got empty. I guess my team is able to tell me more.

You asked me if an unintended power-off could be the reason. Well, usually in such cases only the file that was written the time of the power-off could get corrupted. But why the daemon.json if nobody is writing to it. And then "EOF" ... End of File, so the file is there, but empty. If a file got corrupt then it is not there and removed by Linux if recognized, but empty ... looks like a systematic problem.

I switched on and off my netPI since I operate it for 3 years nearly daily, I have never had such a problem. And I have not heard it from any customer yet.

The only chance I see right now for you is to recover it with a USB stick.

If you have a revision #2 device (small number "2" in the lower left corner of the bar code label) then the internal SD card can be remove and a binary image be taken with a windows tool for example. This image could be analyzed here. But I expect that we will not get really out what happened here.

Thx
Armin

[EUROKEY]

Good morning Armin,

I cannot find a bar code anywhere so I expect you're referring to the QR code? But the part number is 1321.510, same as our 2nd netPi, date code is 19/38.

So I think we need to ship it back?

Best,
Manuel

[Armin@netPI]

Hello Manuel,

sending back is not necessary at all. You can recover it via USB. The procedure is described in the FAQ: https://www.netiot.com/?id=2427

But you are right. I meant the QR-code of course. Sorry for misleading you. 1321.510 is the product code, correct. There is another digit "2" next to it with a space in between. Your device has been manufacturered week 38 year 2019, so it should be a version 2.

Thx
Armin

[EUROKEY]

Hello Armin,

thanks. We flashed the firmware and functionally is back.

Best,
Manuel

[student]

For imformation: Now I have the same problem with the Docker Management, it does not start. I tried to start the service by myself so the button turns from yellow in green. as soon as i leave the services the status returns to yellow. I did the same like Manuel. I powered off the netPI and after a restart the servie does not start again. I will recover the system...

Best regards,
Marcel

[Armin@netPI]

What is always possible with netPI is to look to the system log file that can be downloaded from the netPI Control panel system dialog. You can post it here in the forum if you want and I can have a look. This should give clearance what happened to the system.

Thx
Armin

[Armin@netPI]

I have seen such an effect myself already.

What I did was just unpowering my netPI instead of shutting it down properly using the "shutdown" function in the control panel. To be honest I am doing this always by just removing the 24V power, but this time it seems to corrupted my netPI file system.

The system log file was showing me something like that

Code:

<37>1 2020-07-20T07:56:15+00:00 ab-lte-netpi audit 1750 - - SYSCALL arch=40000028 syscall=322 per=800000 success=yes exit=6 a0=ffffff9c a1=139c14a0 a2=a0000 a3=0 items=1 ppid=1 pid=1750 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="dockerd" exe="/usr/bin/dockerd" key="docker"
<37>1 2020-07-20T07:56:15+00:00 ab-lte-netpi audit - - - CWD cwd="/"
<37>1 2020-07-20T07:56:15+00:00 ab-lte-netpi audit - - - PATH item=0 name="/etc/docker/daemon.json" inode=8820 dev=00:13 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
<37>1 2020-07-20T07:56:15+00:00 ab-lte-netpi audit - - - PROCTITLE proctitle=2F7573722F62696E2F646F636B657264002D480066643A2F2F
<30>1 2020-07-20T07:56:15+00:00 ab-lte-netpi dockerd 1750 - - unable to configure the Docker daemon with file /etc/docker/daemon.json: EOF
<29>1 2020-07-20T07:56:15+00:00 ab-lte-netpi systemd 1 - - docker.service: Main process exited, code=exited, status=1/FAILURE
<27>1 2020-07-20T07:56:15+00:00 ab-lte-netpi systemd 1 - - Failed to start Docker Application Container Engine.
<37>1 2020-07-20T07:56:15+00:00 ab-lte-netpi audit 1 - - SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=docker comm="systemd" exe="/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
<29>1 2020-07-20T07:56:15+00:00 ab-lte-netpi systemd 1 - - docker.service: Unit entered failed state.
<28>1 2020-07-20T07:56:15+00:00 ab-lte-netpi systemd 1 - - docker.service: Failed with result 'protocol'.
<30>1 2020-07-20T07:56:16+00:00 ab-lte-netpi systemd 1 - - docker.service: Service hold-off time over, scheduling restart.

so the docker daemon reports that it can't be started cause the daemon.json file got corrupt.

And indeed ... I have a very special netPI where I was able to look to the host's file system and to th /etc/docker/deamon.json file and I saw that it was totally empty which I cannot explain why.

Usually this file content is  

Code:

{
 "storage-driver": "overlay2",
 "data-root": "/run/.system_part/docker",
 "live-restore": true,
 "disable-legacy-registry": true,
 "userland-proxy": false,
 "userland-proxy-path": "/usr/bin/docker-proxy",
 "log-driver": "journald"
}

So I really have no clue where this effect was coming from other than I removed the power from the netPI device instantly.