Send inn-workers mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/inn-workers
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of inn-workers digest..."
Today's Topics:
1. Re: ovsqlite - bug? (Julien ?LIE)
2. [Resolved?] Re: ovsqlite - bug? (Kamil Jo?ca)
3. Re: [Resolved?] Re: ovsqlite - bug? (Julien ?LIE)
4. Re: [Resolved?] ovsqlite - bug? (Bo Lindbergh)
----------------------------------------------------------------------
Message: 1
Date: Sun, 20 Nov 2022 22:01:26 +0100
From: Julien ?LIE <[email protected]>
To: [email protected]
Subject: Re: ovsqlite - bug?
Message-ID: <[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed
Hi Kamil,
>> Are there plenty of these messages or only a very few?
> makehistory: cannot write overview data
> "@03FF30000000000000000000004400000008@"
> makehistory: cannot write overview data
> "@03FF31000000000000000000004400000008@"
> makehistory: cannot write overview data
> "@03FF32000000000000000000004400000008@"
> makehistory: cannot write overview data
> "@03FF33000000000000000000004400000008@"
> makehistory: cannot write overview data
> "@03FF35000000000000000000004400000007@"
> makehistory: cannot write overview data
> "@03FF36000000000000000000004400000007@"
> makehistory: cannot write overview data
> "@03FF37000000000000000000004400000007@"
>
> 7 ( for 4.8 million articles in spool)
OK, I thought there were far more errors than that.
As you noticed, the regularity in the storage tokens is strange.
@03FF30000000000000000000004400000008@
=> CNFS method (0x03), class 255 (0xFF), buffer named "0" (0x30), block
68 (0x44), cycle number 8 (0x08)
Same block and cycle number 8 for buffers named "1", "2" and "3".
Same block but cycle number 7 for buffers named "5", "6" and "7" (which
I assumed still have not wrapped, and you are currently filling buffer "4").
>> Do you happen to have the same errors if you try to convert your
>> overview to tradindexed (running the same makehistory command)?
> I rebuilt with ovdb - no messages.
> then I rebuilt witho ovsqlite - results above.
Strange that only ovsqlite reports an error when fed the overview lines
related to these articles in block 68.
makehistory generates the overview lines; it is not specific to an
overview method.
> Do you still suggest to test with tradindexed?
It may be interesting to see whether tradindexed has the same errors,
yes. Notably when ovdb does not.
--
Julien ?LIE
??? Par Thor?!
? Par Odin?!
? Par exemple?!?? (Ast?rix)
------------------------------
Message: 2
Date: Sun, 20 Nov 2022 22:48:43 +0100
From: Kamil Jo?ca <[email protected]>
To: [email protected]
Subject: [Resolved?] Re: ovsqlite - bug?
Message-ID: <[email protected]>
Content-Type: text/plain; charset=iso-8859-2
Julien ?LIE <[email protected]> writes:
> Hi Kamil,
>
>>> Are there plenty of these messages or only a very few?
>> makehistory: cannot write overview data
>> "@03FF30000000000000000000004400000008@"
>> makehistory: cannot write overview data
>> "@03FF31000000000000000000004400000008@"
>> makehistory: cannot write overview data
>> "@03FF32000000000000000000004400000008@"
>> makehistory: cannot write overview data
>> "@03FF33000000000000000000004400000008@"
>> makehistory: cannot write overview data
>> "@03FF35000000000000000000004400000007@"
>> makehistory: cannot write overview data
>> "@03FF36000000000000000000004400000007@"
>> makehistory: cannot write overview data
>> "@03FF37000000000000000000004400000007@"
>> 7 ( for 4.8 million articles in spool)
>
> OK, I thought there were far more errors than that.
Well, it seems that problem is with spool not with sqlite.
I dig somewhat deeper into overview database (with sqlite3 cli tool) and I
have some suspections about duplicates of (group,artnum) in spool
These suspections makes me to issue command:
--8<---------------cut here---------------start------------->8---
sudo -u news env PATH=$PATH:/usr/lib/news/bin makehistory -O -x
-S|grep'\(linux.debian.user:464806\|gmane.linux.debian.packages.ocaml.scm:21155\|gmane.comp.file-systems.gluster.user:28781\|gmane.comp.emulators.qemu:799173\|gmane.comp.java.jenkins.user:48616\)'|tee
/tmp/spool
--8<---------------cut here---------------end--------------->8---
As a result I have:
--8<---------------cut here---------------start------------->8---
@03FF32000000000000000000004400000008@ 1643431380 0 Antipatterns and Handcuffs
[was: Re: Resolved (without understanding ...] <[email protected]> Fri,
24 Jul 2020 11:30:01 +0200 <[email protected]>
<[email protected]> <[email protected]>
<[email protected]> <[email protected]>
<[email protected]> <[email protected]>
<[email protected]> <[email protected]> 4818 57
Xref: news.kjonca linux.debian.user:464806
@03FF32000000000000000000004400000008@ 1643431380 0 Antipatterns and Handcuffs
[was: Re: Resolved (without understanding ...] <[email protected]> Fri,
24 Jul 2020 11:30:01 +0200 <[email protected]>
<[email protected]> <[email protected]>
<[email protected]> <[email protected]>
<[email protected]> <[email protected]>
<[email protected]> <[email protected]> 4818 57
Xref: news.kjonca linux.debian.user:464806
@03FF33000000000000000000004400000008@ 1643433853 0 [PATCH 1/4] fuzz: adjust
timeout to allow for longer inputs Alexander Bulekov <[email protected]> Thu,
17 Jun 2021 15:53:50 -0400 <[email protected]>
<[email protected]> 12813 68 Xref: news.kjonca
gmane.comp.emulators.qemu:799173
@03FF33000000000000000000004400000008@ 1643433853 0 [PATCH 1/4] fuzz: adjust
timeout to allow for longer inputs Alexander Bulekov <[email protected]> Thu,
17 Jun 2021 15:53:50 -0400 <[email protected]>
<[email protected]> 12813 68 Xref: news.kjonca
gmane.comp.emulators.qemu:799173
@03FF35000000000000000000004400000007@ 1643412102 0 [ocaml-ctypes] branch
master updated (e9b7f58 -> d0083e4) =?UTF-8?Q?St=C3=A9phane_Glondu?=
<[email protected]> Tue, 14 Jun 2016
09:46:35 +0000 <[email protected]>
15918 200 Xref: news.kjonca gmane.linux.debian.packages.ocaml.scm:21155
@03FF35000000000000000000004400000007@ 1643412102 0 [ocaml-ctypes] branch
master updated (e9b7f58 -> d0083e4) =?UTF-8?Q?St=C3=A9phane_Glondu?=
<[email protected]> Tue, 14 Jun 2016
09:46:35 +0000 <[email protected]>
15918 200 Xref: news.kjonca gmane.linux.debian.packages.ocaml.scm:21155
@03FF36000000000000000000004400000007@ 1643417459 0 Re: volume start: data0:
failed: Commit failed on localhost. Deepak Naidu
<[email protected]> Fri, 24 Feb 2017 17:36:03
+0000 <[email protected]>
<[email protected]>
<[email protected]> 277268 3740 Xref:
news.kjonca gmane.comp.file-systems.gluster.user:28781
@03FF36000000000000000000004400000007@ 1643417459 0 Re: volume start: data0:
failed: Commit failed on localhost. Deepak Naidu
<[email protected]> Fri, 24 Feb 2017 17:36:03
+0000 <[email protected]>
<[email protected]>
<[email protected]> 277268 3740 Xref:
news.kjonca gmane.comp.file-systems.gluster.user:28781
@03FF37000000000000000000004400000007@ 1643421414 0 Re: location of email-ext
template file Chris T
<[email protected]> Wed, 6 Dec 2017
10:42:10 -0800 (PST)
<753c223c-90cd-4770-9c75-6de9afbc0446__19462.2635994621$1512588446$gmane$o...@googlegroups.com>
<[email protected]>
<CAPiUgVchzFuyQHWuw4-CsxqZ-tjSuGxteE9xZkMkWaJX=ac...@mail.gmail.com>
<CACKaJRPcdnhiQwU0DevOju-5U=djyxwhpko97zt1uttnzuu...@mail.gmail.com>
<capiugvdavcnstwympfdemozjxgyf0jfpxayudf1hvuz3b+u...@mail.gmail.com>
<[email protected]>
<capiugvf2s_2oygpskmgoc1cyw-4afemaeyh+cz0p5l_xxqu...@mail.gmail.com> 215351
3638 Xref: news.kjonca gmane.comp.java.jenkins.user:48616
@03FF37000000000000000000004400000007@ 1643421414 0 Re: location of email-ext
template file Chris T
<[email protected]> Wed, 6 Dec 2017
10:42:10 -0800 (PST)
<753c223c-90cd-4770-9c75-6de9afbc0446__19462.2635994621$1512588446$gmane$o...@googlegroups.com>
<[email protected]>
<CAPiUgVchzFuyQHWuw4-CsxqZ-tjSuGxteE9xZkMkWaJX=ac...@mail.gmail.com>
<CACKaJRPcdnhiQwU0DevOju-5U=djyxwhpko97zt1uttnzuu...@mail.gmail.com>
<capiugvdavcnstwympfdemozjxgyf0jfpxayudf1hvuz3b+u...@mail.gmail.com>
<[email protected]>
<capiugvf2s_2oygpskmgoc1cyw-4afemaeyh+cz0p5l_xxqu...@mail.gmail.com> 215351
3638 Xref: news.kjonca gmane.comp.java.jenkins.user:48616
--8<---------------cut here---------------end--------------->8---
So makehistory sometimes generate two lines for spool entry.
It is worth noting, that greping /var/lib/news/history for these storage
tokesn returns nothing.
(for example)
--8<---------------cut here---------------start------------->8---
kjonca@alfa:~%grep 03FF32000000000000000000004400000008 /var/lib/news/history
kjonca@alfa:~%
--8<---------------cut here---------------end--------------->8---
So the first thigs I should do is to make history consistent with spool
and then test further.
But for now I am pretty sure, that sqlite worked properly as, it forces
uniqueness of (group,artnum ) in overview (and other methods not) -
and this causes error messages.
Sorry for bothering you.
KJ
--
http://wolnelektury.pl/wesprzyj/teraz/
------------------------------
Message: 3
Date: Sun, 20 Nov 2022 23:30:37 +0100
From: Julien ?LIE <[email protected]>
To: [email protected]
Subject: Re: [Resolved?] Re: ovsqlite - bug?
Message-ID: <[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed
Hi Kamil,
> Well, it seems that problem is with spool not with sqlite.
Oh, great, thanks for your investigation!
ovsqlite appears to be more robust than ovdb as it did an integrity
check before accepting new overview data (maybe by primary key in
database, I have not checked).
> It is worth noting, that greping /var/lib/news/history for these storage
> tokens returns nothing.
>
> So the first thigs I should do is to make history consistent with spool
> and then test further.
I don't think you should bother.
You won't be able to make your history file consistent with your
self-expiring CNFS buffers. I bet the articles have already expired
from your current overview. They are just still here in the buffer,
just waiting for being overwritten by the next cycle.
What happens is that makehistory parses your spool and finds these old
articles. It generates overview entries for them.
Tomorrow, after the run of expireover (by news.daily), your overview
date will be purged of these entries.
> But for now I am pretty sure, that sqlite worked properly as, it forces
> uniqueness of (group,artnum ) in overview (and other methods not)
Glad to know!
--
Julien ?LIE
??J'ai six loyaux serviteurs, ils m'ont appris tout ce que je sais?: ils
s'appellent quoi, pourquoi, quand, comment, o? et qui.?? (Rudyard
Kipling)
------------------------------
Message: 4
Date: Mon, 21 Nov 2022 00:00:12 +0100
From: Bo Lindbergh <[email protected]>
To: [email protected]
Cc: Kamil Jo?ca <[email protected]>
Subject: Re: [Resolved?] ovsqlite - bug?
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8
2022-11-20 kl. 22:48 skrev Kamil Jo?ca <[email protected]>:
> But for now I am pretty sure, that sqlite worked properly as, it forces
> uniqueness of (group,artnum ) in overview (and other methods not) -
> and this causes error messages.
It looks like ovdb handles groupname:artnum duplicates by silently replacing
the old record with the new one. (I checked the ovdb source right now,
but I can't remember doing so two years ago while writing ovsqlite.)
Anyway, I call seven out of several million too rare to worry about.
/Bo Lindbergh
------------------------------
Subject: Digest Footer
_______________________________________________
inn-workers mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/inn-workers
------------------------------
End of inn-workers Digest, Vol 145, Issue 4
*******************************************
