Send inn-workers mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/inn-workers
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of inn-workers digest..."
Today's Topics:
1. Re: ovsqlite (Julien ?LIE)
2. Re: ovsqlite (Bo Lindbergh)
----------------------------------------------------------------------
Message: 1
Date: Sun, 20 Dec 2020 00:14:33 +0100
From: Julien ?LIE <[email protected]>
To: [email protected]
Subject: Re: ovsqlite
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8; format=flowed
Hi Bo,
> All the actual code in ovsqlite-private.c should of course have
> #ifdef HAVE_SQLITE3 / #endif around it.
Added, thanks for the confirmation.
> I use a single compile-link-and-run test to probe for a sufficient
> version of SQLite. Feel free to refine this if you have more
> patience with autoconf than I.
I'll certainly integrate sqlite.m4 from rra-c-util:
https://github.com/rra/rra-c-util/tree/master/m4
and merge inside the specific test for HAVE_INTTYPES_H, PRIu64 and an
SQLite version >= 3.8.2.
I see that compression is disabled by default in ovsqlite.conf; is it
still nowadays the best configuration? (ovdb also disables it by
default but we may have a different behaviour, if better, for ovsqlite)
Commits are done by default after 10 000 transactions. Maybe a silly
question but how does ovsqlite behaves if INN (or better say
ovsqlite-server) crashes or is killed without exiting properly? (do we
loose up to 10 000 overview changes since last commit?)
With ovsqlite, are article counts in newsgroups always accurate? (I
think so)
I see that the removal of a newsgroup is deferred until the next
expiration, which is a good thing (preventing an error in a control
message or manually with ctlinnd).
In order to make searches faster (responses to HDR and XPAT when
requesting header fields stored in overview), wouldn't it be useful to
also store each header body (at least the mandatory ones in
overview.fmt) in a separate column? or would the gain be meaningless?
Thanks again for your very great work on ovsqlite!
--
Julien ?LIE
??Sol lucet omnibus.??
------------------------------
Message: 2
Date: Sun, 20 Dec 2020 11:00:16 +0100
From: Bo Lindbergh <[email protected]>
To: [email protected]
Subject: Re: ovsqlite
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8
Quoth Julien ?LIE <[email protected]>:
> I see that compression is disabled by default in ovsqlite.conf; is it still
> nowadays the best configuration? (ovdb also disables it by default but we
> may have a different behaviour, if better, for ovsqlite)
No idea. I just copied that from ovdb.
> Commits are done by default after 10 000 transactions. Maybe a silly
> question but how does ovsqlite behaves if INN (or better say ovsqlite-server)
> crashes or is killed without exiting properly? (do we loose up to 10 000
> overview changes since last commit?)
You'd lose at most the last 10000 articles, or the last 10 seconds of changes,
whichever is smaller. (I have no idea how many articles a typical INN site
needs to store each second.)
The problem is that committing a transaction takes several hundred
times as long as inserting a single row within a transaction,
so throughput correlates strongly with transaction size.
I chose the default after measuring how long "makehistory -O -x" took
using various settings. If it seems too high for daily use,
we could lower it and add a note in the documentation about temporarily
raising it when rebuilding overview data.
> With ovsqlite, are article counts in newsgroups always accurate? (I think so)
That's what the code attempts to do, at least. :-)
> I see that the removal of a newsgroup is deferred until the next expiration,
> which is a good thing (preventing an error in a control message or manually
> with ctlinnd).
Actually, it's because removing thousands of rows from the article table
could take some time and shouldn't be allowed to block regular operations.
Adding a new group does NOT undelete a previous one with the same name.
I checked carefully what ovdb does in this situation and implemented
the closest equivalent.
Manual undeletion would be a lot easier with ovsqlite than with ovdb,
since you get a free SQL command line tool with each SQLite installation.
> In order to make searches faster (responses to HDR and XPAT when requesting
> header fields stored in overview), wouldn't it be useful to also store each
> header body (at least the mandatory ones in overview.fmt) in a separate
> column? or would the gain be meaningless?
The current overview method interface has no way to specify a particular
overview field; it's all or nothing. I did look into storing the subset
needed for expiration separately, but since the helper functions
(OVhisthasmsgid and OVgroupbasedexpire) expect the full overview data
as their input, I abandoned that idea.
Using separate columns would negate the advantage of compression too.
/Bo Lindbergh
------------------------------
Subject: Digest Footer
_______________________________________________
inn-workers mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/inn-workers
------------------------------
End of inn-workers Digest, Vol 126, Issue 12
********************************************
