Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Sven Hartge]

On 22.01.19 18:28, Marc Haber wrote:
> On Mon, Jan 21, 2019 at 03:31:53PM +0100, Marc Haber wrote:
>> On Mon, Jan 21, 2019 at 11:20:52AM +0100, Marc Haber wrote:

>>> I am downgrading this to important though, since I consider the
>>> statistics writing an auxillary feature, and I suspect that the issue
>>> will fix itself next midnight.
>>
>> Can you check whether:
>> mv /var/log/atop/atop_$CURDAY /var/log/atop/atop_oldformat
>> atopconvert /var/log/atop/atop_oldformat /var/log/atop/atop_$CURDAY
>> will allow your atop to run again?
> 
> Do you have a possibility to check whether atop 2.4.0-2 in experimental
> behaves any better? I have added some postinst magic to convert the
> latest raw file.

I had to grab the package from incoming.debian.org to test it, but your
change fixes the problem for me. The old format is detected, converted
and then the new daemon starts without problems.

Grüße,
Sven.




signature.asc
Description: OpenPGP digital signature

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Marc Haber]

On Mon, Jan 21, 2019 at 03:31:53PM +0100, Marc Haber wrote:
> On Mon, Jan 21, 2019 at 11:20:52AM +0100, Marc Haber wrote:
> > I am downgrading this to important though, since I consider the
> > statistics writing an auxillary feature, and I suspect that the issue
> > will fix itself next midnight.
> 
> Can you check whether:
> mv /var/log/atop/atop_$CURDAY /var/log/atop/atop_oldformat
> atopconvert /var/log/atop/atop_oldformat /var/log/atop/atop_$CURDAY
> will allow your atop to run again?

Do you have a possibility to check whether atop 2.4.0-2 in experimental
behaves any better? I have added some postinst magic to convert the
latest raw file.

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Sven Hartge]

On 21.01.19 15:31, Marc Haber wrote:
> On Mon, Jan 21, 2019 at 11:20:52AM +0100, Marc Haber wrote:

>> I am downgrading this to important though, since I consider the
>> statistics writing an auxillary feature, and I suspect that the issue
>> will fix itself next midnight.
> 
> Can you check whether:
> mv /var/log/atop/atop_$CURDAY /var/log/atop/atop_oldformat
> atopconvert /var/log/atop/atop_oldformat /var/log/atop/atop_$CURDAY
> will allow your atop to run again?

I can confirm: this works.

Grüße,
Sven.



signature.asc
Description: OpenPGP digital signature

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Sven Hartge]

On 21.01.19 15:43, Sven Hartge wrote:
> On 21.01.19 15:31, Marc Haber wrote:
>> On Mon, Jan 21, 2019 at 11:20:52AM +0100, Marc Haber wrote:
> 
>>> I am downgrading this to important though, since I consider the
>>> statistics writing an auxillary feature, and I suspect that the issue
>>> will fix itself next midnight.
>>
>> Can you check whether:
>> mv /var/log/atop/atop_$CURDAY /var/log/atop/atop_oldformat
>> atopconvert /var/log/atop/atop_oldformat /var/log/atop/atop_$CURDAY
>> will allow your atop to run again?
> 
> I don't have the old files any more unfortunately, sorry.

But I can temporarily downgrade to the version from testing and then do
the upgrade again.

Grüße,
Sven.



signature.asc
Description: OpenPGP digital signature

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Sven Hartge]

On 21.01.19 15:31, Marc Haber wrote:
> On Mon, Jan 21, 2019 at 11:20:52AM +0100, Marc Haber wrote:

>> I am downgrading this to important though, since I consider the
>> statistics writing an auxillary feature, and I suspect that the issue
>> will fix itself next midnight.
> 
> Can you check whether:
> mv /var/log/atop/atop_$CURDAY /var/log/atop/atop_oldformat
> atopconvert /var/log/atop/atop_oldformat /var/log/atop/atop_$CURDAY
> will allow your atop to run again?

I don't have the old files any more unfortunately, sorry.

Grüße,
Sven.




signature.asc
Description: OpenPGP digital signature

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Marc Haber]

On Mon, Jan 21, 2019 at 11:20:52AM +0100, Marc Haber wrote:
> I am downgrading this to important though, since I consider the
> statistics writing an auxillary feature, and I suspect that the issue
> will fix itself next midnight.

Can you check whether:
mv /var/log/atop/atop_$CURDAY /var/log/atop/atop_oldformat
atopconvert /var/log/atop/atop_oldformat /var/log/atop/atop_$CURDAY
will allow your atop to run again?

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Marc Haber]

On Mon, Jan 21, 2019 at 03:10:28PM +0100, Sven Hartge wrote:
> On 21.01.19 13:27, Marc Haber wrote:
> > On Mon, Jan 21, 2019 at 01:15:09PM +0100, Sven Hartge wrote:
> 
> >> I think you can just delete all old statistics files, since they are
> >> useless with the new atop. sysstat does this all the time (after a
> >> debconf prompt).
> > 
> > I'd rather go without debconf and gazillions of translations.
> 
> Easiest solution would be to run the rotate script on upgrades from
> <2.4.0 before starting the newly installed version. That way the old
> files are still there, if one needs them but the new daemon starts with
> a clean slate.

This is no direct rotate script, atop gets restarted at midnight with -w
"$LOGPATH"/atop_"$CURDAY", so just restarting the script won't work.

Take a look at /usr/share/atop/atop.daily.

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Sven Hartge]

On 21.01.19 13:27, Marc Haber wrote:
> On Mon, Jan 21, 2019 at 01:15:09PM +0100, Sven Hartge wrote:

>> I think you can just delete all old statistics files, since they are
>> useless with the new atop. sysstat does this all the time (after a
>> debconf prompt).
> 
> I'd rather go without debconf and gazillions of translations.

Easiest solution would be to run the rotate script on upgrades from
<2.4.0 before starting the newly installed version. That way the old
files are still there, if one needs them but the new daemon starts with
a clean slate.

Grüße,
Sven.



signature.asc
Description: OpenPGP digital signature

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Marc Haber]

On Mon, Jan 21, 2019 at 01:42:38PM +0100, Vincent Lefevre wrote:
> If too complex, it should be sufficient to announce what will be
> done in NEWS.Debian (with apt-listchanges, the user could do a
> backup first, and if he doesn't use apt-listchanges, he probably
> doesn't care) and choose one of the solutions (not 3, because it
> could use too much disk space). Probably 2, as the user probably
> won't downgrade and won't have to remove the old files manually.

atopconvert will also back-convert, if I believe the docs.

Yuck, there is so much to do wrong in that postinst. I already hate it.

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Vincent Lefevre]

On 2019-01-21 12:48:59 +0100, Vincent Lefevre wrote:
> On 2019-01-21 12:28:40 +0100, Vincent Lefevre wrote:
> > If not, another (better?) possibility is to ask the user if he
> > wishes to convert these old files to the new format.
> 
> After thinking more about it, I think that the choice should be given
> between:
> 
> 1. Move the atop_* files under /var/log/atop to /var/log/atop.old
>(to be created if it doesn't exist yet).
> 
> 2. Convert these files to the new format with atopconvert.
> 
> 3. Both (just in case there would be a bug in atopconvert).

If too complex, it should be sufficient to announce what will be
done in NEWS.Debian (with apt-listchanges, the user could do a
backup first, and if he doesn't use apt-listchanges, he probably
doesn't care) and choose one of the solutions (not 3, because it
could use too much disk space). Probably 2, as the user probably
won't downgrade and won't have to remove the old files manually.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Vincent Lefevre]

On 2019-01-21 13:15:09 +0100, Sven Hartge wrote:
> I think you can just delete all old statistics files, since they are
> useless with the new atop. sysstat does this all the time (after a
> debconf prompt).

I disagree. They may still be useful to analyze a past problem,
even with the new atop since they can be converted.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Marc Haber]

On Mon, Jan 21, 2019 at 12:28:40PM +0100, Vincent Lefevre wrote:
> On 2019-01-21 11:20:52 +0100, Marc Haber wrote:
> > What do you think, would it be an acceptable workaround to move the
> > current day file away in postinst and document that fact?
> 
> Are the old files still readable by the new atop?

No :-(

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Marc Haber]

On Mon, Jan 21, 2019 at 01:15:09PM +0100, Sven Hartge wrote:
> I think you can just delete all old statistics files, since they are
> useless with the new atop. sysstat does this all the time (after a
> debconf prompt).

I'd rather go without debconf and gazillions of translations.

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Sven Hartge]

On 21.01.19 11:20, Marc Haber wrote:

> I am downgrading this to important though, since I consider the
> statistics writing an auxillary feature, and I suspect that the issue
> will fix itself next midnight.

But until then, the daemon will be non-functional and since there is no
message *why* this is the case, the user/admin is left in the dark.

> What do you think, would it be an acceptable workaround to move the
> current day file away in postinst and document that fact?

I think you can just delete all old statistics files, since they are
useless with the new atop. sysstat does this all the time (after a
debconf prompt).

Grüße,
Sven.



signature.asc
Description: OpenPGP digital signature

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Vincent Lefevre]

On 2019-01-21 12:28:40 +0100, Vincent Lefevre wrote:
> On 2019-01-21 11:20:52 +0100, Marc Haber wrote:
> > What do you think, would it be an acceptable workaround to move the
> > current day file away in postinst and document that fact?
> 
> Are the old files still readable by the new atop?

Since there is a new program atopconvert, I suppose that they are not.

> If not, another (better?) possibility is to ask the user if he
> wishes to convert these old files to the new format.

After thinking more about it, I think that the choice should be given
between:

1. Move the atop_* files under /var/log/atop to /var/log/atop.old
   (to be created if it doesn't exist yet).

2. Convert these files to the new format with atopconvert.

3. Both (just in case there would be a bug in atopconvert).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Vincent Lefevre]

On 2019-01-21 11:20:52 +0100, Marc Haber wrote:
> What do you think, would it be an acceptable workaround to move the
> current day file away in postinst and document that fact?

Are the old files still readable by the new atop?

If not, another (better?) possibility is to ask the user if he
wishes to convert these old files to the new format.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Marc Haber]

tags #919922 confirmed upstream
severity #919922 important
thanks

Hi Sven,

thanks for spotting this. Strangely, this didn't happen on my test
box, but I can confirm this behavior for my workstation.

I am downgrading this to important though, since I consider the
statistics writing an auxillary feature, and I suspect that the issue
will fix itself next midnight.

What do you think, would it be an acceptable workaround to move the
current day file away in postinst and document that fact?

Greetings
Marc


On Sun, Jan 20, 2019 at 07:17:54PM +0100, Sven Hartge wrote:
> From: Sven Hartge 
> Subject: Bug#919922: Does not start anymore, existing file
>  /var/log/atop/atop_20190120 has incompatible header
> To: Debian Bug Tracking System 
> Reply-To: Sven Hartge , 919...@bugs.debian.org
> Date: Sun, 20 Jan 2019 19:17:54 +0100
> X-Mailer: reportbug 7.5.1
> 
> Package: atop
> Version: 2.4.0-1
> Severity: grave
> 
> Hello!
> 
> After the update tpo 2.4.0-1 atop does not start anymore for me.
> TL;DR: 
> 
> existing file /var/log/atop/atop_20190120 has incompatible header
> (created by version 2.3 - current version 2.4)
> 
> Here the whole debugging session:
> 
> server:~# systemctl start atop; systemctl status atop; sleep 2; systemctl 
> status atop
> ● atop.service - Atop advanced performance monitor
>Loaded: loaded (/lib/systemd/system/atop.service; enabled; vendor preset: 
> enabled)
>Active: active (running) since Sun 2019-01-20 19:08:50 CET; 11ms ago
>  Docs: man:atop(1)
>  Main PID: 550488 (atop.daily)
> Tasks: 5 (limit: 4691)
>Memory: 1.5M
>CGroup: /system.slice/atop.service
>├─550488 /bin/bash /usr/share/atop/atop.daily
>├─550491 ps -p 547913
>└─550492 grep atop$
> ● atop.service - Atop advanced performance monitor
>Loaded: loaded (/lib/systemd/system/atop.service; enabled; vendor preset: 
> enabled)
>Active: failed (Result: exit-code) since Sun 2019-01-20 19:08:50 CET; 1s 
> ago
>  Docs: man:atop(1)
>   Process: 550488 ExecStart=/usr/share/atop/atop.daily (code=exited, status=7)
>  Main PID: 550488 (code=exited, status=7)
> 
> As one can see, the unit starts /usr/share/atop/atop.daily but no daemon
> remains at the end.
> 
> Even when running the shell script directly, nothing happens:
> 
> server:~# ps auwwwx | grep [a]top; rm /run/atop.pid; bash -x 
> /usr/share/atop/atop.daily; sleep 5; ps auwwwx | grep [a]top
> user  547330  0.2  0.8  51748 33624 pts/2S+   19:06   0:00 
> /usr/bin/python3 /usr/bin/reportbug atop
> user  549910  0.0  0.0   2512   508 pts/2S+   19:07   0:00 sh -c vim 
> -c :6 '/tmp/user/1000/reportbug-atop-20190120-547330-uof401oa'
> user  549911  0.1  0.1  12340  7324 pts/2S+   19:07   0:00 vim -c :6 
> /tmp/user/1000/reportbug-atop-20190120-547330-uof401oa
> + LOGOPTS=-R
> + LOGINTERVAL=600
> + LOGGENERATIONS=28
> + DEFAULTSFILE=/etc/default/atop
> + '[' -e /etc/default/atop ']'
> + . /etc/default/atop
> + case "$LOGGENERATIONS" in
> ++ date +%Y%m%d
> + CURDAY=20190120
> + LOGPATH=/var/log/atop
> + BINPATH=/usr/bin
> + PIDFILE=/run/atop.pid
> + '[' -e /run/atop.pid ']'
> + sleep 3
> + echo 555663
> + exec /usr/bin/atop -R -w /var/log/atop/atop_20190120 600
> + find /var/log/atop -name 'atop_*' -mtime +28 -exec rm '{}' ';'
> user  547330  0.2  0.8  51748 33624 pts/2S+   19:06   0:00 
> /usr/bin/python3 /usr/bin/reportbug atop
> user  549910  0.0  0.0   2512   508 pts/2S+   19:07   0:00 sh -c vim 
> -c :6 '/tmp/user/1000/reportbug-atop-20190120-547330-uof401oa'
> user  549911  0.1  0.1  12340  7324 pts/2S+   19:07   0:00 vim -c :6 
> /tmp/user/1000/reportbug-atop-20190120-547330-uof401oa
> 
> (You can see the open reportbug where I type this in right now.)
> 
> Running atop directly shows the problem:
> 
> server:~# /usr/bin/atop -R -w /var/log/atop/atop_20190120 600
> existing file /var/log/atop/atop_20190120 has incompatible header
> (created by version 2.3 - current version 2.4)
> 
> Deleting all files in /var/log/atop/ makes it work again.
> 
> Side not: the error message does *not* appear in the journal nor the syslog.
> 
> Grüße,
> Sven.
> 
> -- System Information:
> Debian Release: buster/sid
>   APT prefers unstable-debug
>   APT policy: (500, 'unstable-debug'), (500, 'testing-debug'), (500, 
> 'unstable'), (500, 'testing'), (200, 'experimental'), (1, 
> 'experimental-debug')
> Architecture: i386 (x86_64)
> Foreign Architectures: amd64
> 
> Kernel: Linux 4.19.0-1-amd64 (SMP w/4 CPU cores)
> Locale: LANG=de_DE.utf8, LC_CTYPE=de_DE.utf8 (charmap=UTF-8), 
> LANGUAGE=de_DE.utf8 (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/d

Bug#919922: Does not start anymore, existing file /var/log/atop/atop_20190120 has incompatible header

[Sven Hartge]

Package: atop
Version: 2.4.0-1
Severity: grave

Hello!

After the update tpo 2.4.0-1 atop does not start anymore for me.
TL;DR: 

existing file /var/log/atop/atop_20190120 has incompatible header
(created by version 2.3 - current version 2.4)

Here the whole debugging session:

server:~# systemctl start atop; systemctl status atop; sleep 2; systemctl 
status atop
● atop.service - Atop advanced performance monitor
   Loaded: loaded (/lib/systemd/system/atop.service; enabled; vendor preset: 
enabled)
   Active: active (running) since Sun 2019-01-20 19:08:50 CET; 11ms ago
 Docs: man:atop(1)
 Main PID: 550488 (atop.daily)
Tasks: 5 (limit: 4691)
   Memory: 1.5M
   CGroup: /system.slice/atop.service
   ├─550488 /bin/bash /usr/share/atop/atop.daily
   ├─550491 ps -p 547913
   └─550492 grep atop$
● atop.service - Atop advanced performance monitor
   Loaded: loaded (/lib/systemd/system/atop.service; enabled; vendor preset: 
enabled)
   Active: failed (Result: exit-code) since Sun 2019-01-20 19:08:50 CET; 1s ago
 Docs: man:atop(1)
  Process: 550488 ExecStart=/usr/share/atop/atop.daily (code=exited, status=7)
 Main PID: 550488 (code=exited, status=7)

As one can see, the unit starts /usr/share/atop/atop.daily but no daemon
remains at the end.

Even when running the shell script directly, nothing happens:

server:~# ps auwwwx | grep [a]top; rm /run/atop.pid; bash -x 
/usr/share/atop/atop.daily; sleep 5; ps auwwwx | grep [a]top
user  547330  0.2  0.8  51748 33624 pts/2S+   19:06   0:00 
/usr/bin/python3 /usr/bin/reportbug atop
user  549910  0.0  0.0   2512   508 pts/2S+   19:07   0:00 sh -c vim -c 
:6 '/tmp/user/1000/reportbug-atop-20190120-547330-uof401oa'
user  549911  0.1  0.1  12340  7324 pts/2S+   19:07   0:00 vim -c :6 
/tmp/user/1000/reportbug-atop-20190120-547330-uof401oa
+ LOGOPTS=-R
+ LOGINTERVAL=600
+ LOGGENERATIONS=28
+ DEFAULTSFILE=/etc/default/atop
+ '[' -e /etc/default/atop ']'
+ . /etc/default/atop
+ case "$LOGGENERATIONS" in
++ date +%Y%m%d
+ CURDAY=20190120
+ LOGPATH=/var/log/atop
+ BINPATH=/usr/bin
+ PIDFILE=/run/atop.pid
+ '[' -e /run/atop.pid ']'
+ sleep 3
+ echo 555663
+ exec /usr/bin/atop -R -w /var/log/atop/atop_20190120 600
+ find /var/log/atop -name 'atop_*' -mtime +28 -exec rm '{}' ';'
user  547330  0.2  0.8  51748 33624 pts/2S+   19:06   0:00 
/usr/bin/python3 /usr/bin/reportbug atop
user  549910  0.0  0.0   2512   508 pts/2S+   19:07   0:00 sh -c vim -c 
:6 '/tmp/user/1000/reportbug-atop-20190120-547330-uof401oa'
user  549911  0.1  0.1  12340  7324 pts/2S+   19:07   0:00 vim -c :6 
/tmp/user/1000/reportbug-atop-20190120-547330-uof401oa

(You can see the open reportbug where I type this in right now.)

Running atop directly shows the problem:

server:~# /usr/bin/atop -R -w /var/log/atop/atop_20190120 600
existing file /var/log/atop/atop_20190120 has incompatible header
(created by version 2.3 - current version 2.4)

Deleting all files in /var/log/atop/ makes it work again.

Side not: the error message does *not* appear in the journal nor the syslog.

Grüße,
Sven.

-- System Information:
Debian Release: buster/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'testing-debug'), (500, 
'unstable'), (500, 'testing'), (200, 'experimental'), (1, 'experimental-debug')
Architecture: i386 (x86_64)
Foreign Architectures: amd64

Kernel: Linux 4.19.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=de_DE.utf8, LC_CTYPE=de_DE.utf8 (charmap=UTF-8), 
LANGUAGE=de_DE.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages atop depends on:
ii  libc62.28-5
ii  libncurses6  6.1+20181013-1
ii  libtinfo66.1+20181013-1
ii  lsb-base 10.2018112800
ii  zlib1g   1:1.2.11.dfsg-1

Versions of packages atop recommends:
ii  cron [cron-daemon]  3.0pl1-130

atop suggests no packages.

-- debconf-show failed