Send inn-workers mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/inn-workers
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of inn-workers digest..."
Today's Topics:
1. Re: ovsqlite (Julien ?LIE)
2. Re: silently ignore duplicate inews via mailpost (Julien ?LIE)
3. Re: ovsqlite (Bo Lindbergh)
4. Re: ovsqlite (Bo Lindbergh)
5. Re: ovsqlite (Julien ?LIE)
----------------------------------------------------------------------
Message: 1
Date: Mon, 21 Dec 2020 21:58:24 +0100
From: Julien ?LIE <[email protected]>
To: [email protected]
Subject: Re: ovsqlite
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8; format=flowed
Hi Bo,
>> Commits are done by default after 10 000 transactions. Maybe a
>> silly question but how does ovsqlite behaves if INN (or better
>> say>> ovsqlite-server) crashes or is killed without exiting
>> properly? (do we loose up to 10 000 overview changes since last
>> commit?) >
> You'd lose at most the last 10000 articles, or the last 10 seconds of
> changes, whichever is smaller. (I have no idea how many articles a
> typical INN site needs to store each second.)
>
> If it seems too high for daily use,
> we could lower it and add a note in the documentation about temporarily
> raising it when rebuilding overview data.
Interesting information; I haven't still read that part of code about 10
seconds. I'll mention this in the man page.
>> With ovsqlite, are article counts in newsgroups always accurate? (I think so)
>
> That's what the code attempts to do, at least. :-)
That's great!
> Manual undeletion would be a lot easier with ovsqlite than with ovdb,
> since you get a free SQL command line tool with each SQLite installation.
So true!
Here are a few gcc warnings I had with GCC 10.1.0:
* In ovsqlite/ovsqlite.c, the ovsqlite_open() function does not use its
argument "mode". Is it expected? (for read-only or writable mode)
* fill_search_buffer() should be static. (Fixed.)
* I'm facing 5 "cast increases required alignment of target type"
errors. Do you know how I could fix them?
ovsqlite/ovsqlite.c: Dans la fonction ??finish_request??:
ovsqlite/ovsqlite.c:143:6: attention: le transtypage augmente
l'alignement requis pour le type cibl? [-Wcast-align]
143 | *(uint32_t *)request->data = request->left;
| ^
ovsqlite/ovsqlite.c: Dans la fonction ??read_response??:
ovsqlite/ovsqlite.c:225:34: attention: le transtypage augmente
l'alignement requis pour le type cibl? [-Wcast-align]
225 | response_size = *(uint32_t *)response->data;
| ^
ovsqlite/ovsqlite.c: Dans la fonction ??fill_search_buffer??:
ovsqlite/ovsqlite.c:584:20: attention: le transtypage augmente
l'alignement requis pour le type cibl? [-Wcast-align]
584 | rh->overview = (char **)store;
| ^
ovsqlite/ovsqlite.c:587:19: attention: le transtypage augmente
l'alignement requis pour le type cibl? [-Wcast-align]
587 | rh->arrived = (time_t *)store;
| ^
ovsqlite/ovsqlite.c:590:18: attention: le transtypage augmente
l'alignement requis pour le type cibl? [-Wcast-align]
590 | rh->artnum = (ARTNUM *)store;
| ^
I do not manage to link innd:
21:47 iulius@denver ~/work/inn/trunk/innd% ../libtool --mode=link
/home/iulius/work/gcc/gcc-10.1.0/bin/gcc -pie -Wl,-z,relro -Wl,-z,now -o
innd art.o cc.o chan.o icd.o innd.o keywords.o lc.o nc.o newsfeeds.o
ng.o perl.o proc.o python.o rc.o site.o status.o util.o wip.o
/home/iulius/work/inn/trunk/storage/libstorage.la
/home/iulius/work/inn/trunk/history/libinnhist.la
/home/iulius/work/inn/trunk/lib/libinn.la -ldb -lz
/home/iulius/work/inn/trunk/lib/perl.o -Wl,-E -fstack-protector
-L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.20/CORE -lperl -ldl
-lm -lpthread -lcrypt -L/usr/lib -lpython2.7 -lpthread -ldl -lutil -lm
-L/usr/lib -lz -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
-Wl,-z,relro
/home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined
reference to `unpack_now'
/home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined
reference to `unpack_later'
/home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined
reference to `pack_later'
/home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined
reference to `pack_now'
My Makefile.global only has SQLITE3_LIBS set:
SQLITE3_CPPFLAGS =
SQLITE3_LDFLAGS =
SQLITE3_LIBS = -lsqlite3
Is there something I mislooked?
--
Julien ?LIE
??Ce que j'aime dans les voyages, c'est l'?tonnement du retour.??
(Stendhal)
------------------------------
Message: 2
Date: Mon, 21 Dec 2020 22:49:50 +0100
From: Julien ?LIE <[email protected]>
To: [email protected], Harald Dunkel
<[email protected]>
Subject: Re: silently ignore duplicate inews via mailpost
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8; format=flowed
Hi Harald,
> I have modified mailpost as suggested, but the problem is not that
> easy to reproduce. Obviously there is a race condition between
> sendmail/mailpost and inews. I would guess that usually sendmail
> is faster and that inews dropped the duplicate article without
> complaining.
As you're using "-c 15" in the call to mailpost, it sleeps during 15
seconds:
CUSTOMER_BOSS: "|/usr/lib/news/bin/mailpost -c 15 -b /var/tmp -x To:CC
example.projects.CUSTOMER_BOSS"
From: [email protected] (Anja)
Newsgroups: example.projects.CUSTOMER_BOSS
Cc: CUSTOMER_BOSS <[email protected]>
Message-ID: <[email protected]>
Thunderbird sends both an article and an e-mail.
So either you have in that order:
1/ a/ message directly posted to the newsgroup
b/ message posted to the newsgroup by mailpost 15s after having
received the e-mail (but then you would have a Duplicate error)
or:
2/ a/ message posted to the newsgroup by mailpost 15s after having
received the e-mail
b/ message directly posted to the newsgroup (but then Anja receives
an error in Thunderbird because of a Duplicate message-ID)
The first case 1/ would be the most frequent, wouldn't it?
Or I'm missing something in the explanation; what would the race
condition be?
--
Julien ?LIE
??Quand on retire tout ce qu'on a dit, il reste tout ce qu'on n'a pas
dit.?? (Philippe Geluck)
------------------------------
Message: 3
Date: Tue, 22 Dec 2020 04:32:18 +0100
From: Bo Lindbergh <[email protected]>
To: [email protected]
Subject: Re: ovsqlite
Message-ID: <[email protected]>
Content-Type: text/plain; charset="utf-8"
Quoth Julien ?LIE <[email protected]>:
> * In ovsqlite/ovsqlite.c, the ovsqlite_open() function does not use its
> argument "mode". Is it expected? (for read-only or writable mode)
The current version doesn't care about the mode. The right thing to do here
is to include it in the initial handshake and let the server process ignore it.
> * fill_search_buffer() should be static. (Fixed.)
Correct.
> * I'm facing 5 "cast increases required alignment of target type" errors. Do
> you know how I could fix them?
The way to silence this warning is apparently to first cast to void *.
> ovsqlite/ovsqlite.c: Dans la fonction ? finish_request ?:
> ovsqlite/ovsqlite.c:143:6: attention: le transtypage augmente l'alignement
> requis pour le type cibl? [-Wcast-align]
> 143 | *(uint32_t *)request->data = request->left;
> | ^
> ovsqlite/ovsqlite.c: Dans la fonction ? read_response ?:
> ovsqlite/ovsqlite.c:225:34: attention: le transtypage augmente l'alignement
> requis pour le type cibl? [-Wcast-align]
> 225 | response_size = *(uint32_t *)response->data;
> | ^
> ovsqlite/ovsqlite.c: Dans la fonction ? fill_search_buffer ?:
> ovsqlite/ovsqlite.c:584:20: attention: le transtypage augmente l'alignement
> requis pour le type cibl? [-Wcast-align]
> 584 | rh->overview = (char **)store;
> | ^
The first three are definitely safe since the input pointer comes
from the start of a malloc-ed buffer.
> ovsqlite/ovsqlite.c:587:19: attention: le transtypage augmente l'alignement
> requis pour le type cibl? [-Wcast-align]
> 587 | rh->arrived = (time_t *)store;
> | ^
The fourth one is safe as long as time_t does not have stricter alignment
than char *.
> ovsqlite/ovsqlite.c:590:18: attention: le transtypage augmente l'alignement
> requis pour le type cibl? [-Wcast-align]
> 590 | rh->artnum = (ARTNUM *)store;
> | ^
The fifth one is safe as long as ARTNUM does not have stricter alignment
than time_t.
> I do not manage to link innd:
>
> 21:47 iulius@denver ~/work/inn/trunk/innd% ../libtool --mode=link
> /home/iulius/work/gcc/gcc-10.1.0/bin/gcc -pie -Wl,-z,relro -Wl,-z,now -o innd
> art.o cc.o chan.o icd.o innd.o keywords.o lc.o nc.o newsfeeds.o ng.o perl.o
> proc.o python.o rc.o site.o status.o util.o wip.o
> /home/iulius/work/inn/trunk/storage/libstorage.la/home/iulius/work/inn/trunk/history/libinnhist.la/home/iulius/work/inn/trunk/lib/libinn.la
> -ldb -lz /home/iulius/work/inn/trunk/lib/perl.o -Wl,-E -fstack-protector
> -L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.20/CORE -lperl -ldl -lm
> -lpthread -lcrypt -L/usr/lib -lpython2.7 -lpthread -ldl -lutil -lm -L/usr/lib
> -lz -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro
> /home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined reference
> to `unpack_now'
> /home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined reference
> to `unpack_later'
> /home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined reference
> to `pack_later'
> /home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined reference
> to `pack_now'
That is strange. The pack_* and unpack_* functions are defined
in ovsqlite-private.c, which should be in METHOD_SOURCES.
I've collected fixes for the issues discussed so far:
* #ifdef HAVE_SQLITE3 in ovsqlite-private.c
* unused mode in ovsqlite_open
* static fill_search_buffer
* cast alignment warnings
Delta patch attached to this message (in git diff format, complain if that
doesn't work for you).
Full patch based on SVN revision 10457 here:
https://www.stacken.kth.se/~blgl/inn-ovsqlite-take-2.diff.gz
/Bo Lindbergh
-------------- next part --------------
A non-text attachment was scrubbed...
Name: delta.diff.gz
Type: application/x-gzip
Size: 1964 bytes
Desc: not available
URL:
<https://lists.isc.org/pipermail/inn-workers/attachments/20201222/436e4480/attachment-0001.bin>
------------------------------
Message: 4
Date: Tue, 22 Dec 2020 05:09:59 +0100
From: Bo Lindbergh <[email protected]>
To: [email protected]
Subject: Re: ovsqlite
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
Quoth Russ Allbery <[email protected]>:
>
> ovdb has historically required some porting to new versions of BerkelyDB,
> although as the development pace of BerkeleyDB has stalled, it's come up
> less.
Well, SQLite ought to do better there, since its stated goals include
long-term stability and compatibility.
The need for multiple ovdb_monitor processes always looked odd to me.
It's a weird design if it allows things to go so wrong that the next
regular client to access the database can't fix it.
/Bo Lindbergh
------------------------------
Message: 5
Date: Tue, 22 Dec 2020 09:08:09 +0100
From: Julien ?LIE <[email protected]>
To: Bo Lindbergh <[email protected]>,
"[email protected]" <[email protected]>
Subject: Re: ovsqlite
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8; format=flowed
Hi Bo,
>> * In ovsqlite/ovsqlite.c, the ovsqlite_open() function does not use
>> its argument "mode". Is it expected? (for read-only or writable
>> mode)
>
> The current version doesn't care about the mode. The right thing to
> do here is to include it in the initial handshake and let the server
> process ignore it.
OK, thanks for the addition.
>> I do not manage to link innd:
>>
>> 21:47 iulius@denver ~/work/inn/trunk/innd% ../libtool --mode=link
>> /home/iulius/work/gcc/gcc-10.1.0/bin/gcc -pie -Wl,-z,relro -Wl,-z,now -o
>> innd art.o cc.o chan.o icd.o innd.o keywords.o lc.o nc.o newsfeeds.o ng.o
>> perl.o proc.o python.o rc.o site.o status.o util.o wip.o
> /home/iulius/work/inn/trunk/storage/libstorage.la/home/iulius/work/inn/trunk/history/libinnhist.la/home/iulius/work/inn/trunk/lib/libinn.la
>
> -ldb -lz /home/iulius/work/inn/trunk/lib/perl.o -Wl,-E
> -fstack-protector -L/usr/local/lib
> -L/usr/lib/x86_64-linux-gnu/perl/5.20/CORE -lperl -ldl -lm -lpthread
> -lcrypt -L/usr/lib -lpython2.7 -lpthread -ldl -lutil -lm -L/usr/lib -lz
> -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro
>> /home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined reference
>> to `unpack_now'
>> /home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined reference
>> to `unpack_later'
>> /home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined reference
>> to `pack_later'
>> /home/iulius/work/inn/trunk/storage/.libs/libstorage.so: undefined reference
>> to `pack_now'
>
> That is strange.? The pack_* and unpack_* functions are defined
> in ovsqlite-private.c, which should be in METHOD_SOURCES.
Works fine now... I don't know why it did not work yesterday, but well,
let's consider it's now fixed.
> I've collected fixes for the issues discussed so far:
> ?* #ifdef HAVE_SQLITE3 in ovsqlite-private.c
> ?* unused mode in ovsqlite_open
> ?* static fill_search_buffer
> ?* cast alignment warnings
>
> Delta patch attached to this message (in git diff format, complain if that
> doesn't work for you).
Applied without problem, thanks!
Also, a minor patch to silent other warnings:
--- storage/ovsqlite/ovsqlite-server.c.OLD 2020-12-22 08:33:16.513653482
+0100
+++ storage/ovsqlite/ovsqlite-server.c 2020-12-22 08:35:06.608337788 +0100
@@ -185,7 +185,7 @@
sqlite3_clear_bindings(stmt);
}
-client_t *add_client(
+static client_t *add_client(
int sock)
{
client_t *result;
@@ -215,7 +215,7 @@
return result;
}
-void del_client(
+static void del_client(
client_t *client)
{
size_t ix;
@@ -1766,12 +1766,10 @@
static void do_finish_expire(
client_t *client)
{
- buffer_t *reqbuf;
int changes;
int64_t groupid = 0;
failvar_stmt_savepoint;
- reqbuf = client->request;
if (!finish_request(client))
fail(response_bad_request);
I'm now encountering two shift-negative-value errors (building with
"make warnings"):
ovsqlite/ovsqlite-server.c: Dans la fonction ??pack_length??:
ovsqlite/ovsqlite-server.c:482:27: erreur: d?calage vers la gauche d'une
valeur n?gative [-Werror=shift-negative-value]
482 | *--walk = length | (~0<<(9-lenlen));
| ^~
ovsqlite/ovsqlite-server.c: Dans la fonction ??unpack_length??:
ovsqlite/ovsqlite-server.c:501:20: erreur: d?calage vers la gauche d'une
valeur n?gative [-Werror=shift-negative-value]
501 | length = c&~(~0<<(8-lenlen));
| ^~
ovsqlite/ovsqlite-server.c: Dans la fonction ??open_db??:
ovsqlite/ovsqlite-server.c:591:9: erreur: ??SQLITE_PREPARE_PERSISTENT??
non d?clar? (premi?re utilisation dans cette fonction)
591 | SQLITE_PREPARE_PERSISTENT,
| ^~~~~~~~~~~~~~~~~~~~~~~~~
For this last one, it seems that SQLITE_PREPARE_PERSISTENT was
introduced in SQLite 3.20.0.
Should INN require that recent version (August 2017) at configure time?
Or #if SQLITE_VERSION_NUMBER >= 3020000 added there?
And also around sqlite3_prepare_v3 in sqlite-helper.c and use
sqlite3_prepare_v2 before 3.20.0?
If I do not take into account the first 2 warnings, and manually use
sqlite3_prepare_v2, the remaining of the build works fine with SQLite
3.8.7.1 from Debian Jessie. Good news!
--
Julien ?LIE
??? Ouvre l'?il, et le bon?!
? L'autre, je peux pas encore l'ouvrir, je risque pas de me
tromper?!?? (Ast?rix)
------------------------------
Subject: Digest Footer
_______________________________________________
inn-workers mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/inn-workers
------------------------------
End of inn-workers Digest, Vol 126, Issue 14
********************************************
