On Jun 13, 2018, at 3:12 PM, Richard Hipp <d...@sqlite.org> wrote:
>
> I have found is that forum
> and ticket traffic far exceeds the amount of source code.
I just exported my local 6-year archive of the SQLite mailing list traffic to
an mbox file, then gzipped it, yielding a 31 MB file. The current
sqlite.fossil file is 51 MB.
If we do a linear projection back to the start of the mailing list — August
2002, according to MARC — then we’d expect the full ML archive to be roughly
100 MB, not counting the -dev list and such.
That estimate includes the peak in traffic many years ago:
https://imgur.com/a/UdQ5ziH
That chart shows that the SQLite-Users mailing list is “dying,” in that its
message volume is currently lower than near the mailing list’s beginning,
despite the much greater popularity of the library these days. I ascribe that
to an increasing percentage of questions having web-searchable answers now,
plus a decreasing willingness to put up with mailing lists.
(I just corrected a funny autocorrect: malign lists. :) )
My gzipped mbox test is worse than a Fossil-based solution, which:
1. would not have all of those redundant RFC 822 mail header labels;
2. would not need many of the message header lines at all (e.g. X-Abuse-Info or
List-Unsubscribe); and
3. would not store many of the fields in inefficient plain text forms within
the SQLite DB. (e.g. Delivery-Date might be a time_t stored in an INTEGER
column rather than the verbose text RFC 822 format, and if not, then it would
probably be the more compact ISO 8601 format so as to make the SQLite date/time
routines happy.)
There would be no extra disk space cost on sqlite.org, since I assume you’re
already keeping a mailing list archive there.
The extra space on the wire during cloning can be solved in the same way /uv is
handled today, if need be.
> this kind of traffic does not lend itself well to delta compression.
In fact, delta compression would benefit two use cases of the Fossil Forum
Feature:
1. The post editing case, just as with Fossil wiki articles today.
2. Quotations in replies.
Granted, a forum feature would lower the current /stat compression ratios, but
we’re not really after high compression ratios for their own sake, are we?
We’re using compression to solve problems: inter-version redundancy, the low
information density of prose and high-level language code, etc.
Forum posts would be highly compressible by zlib. The uncompressed mbox file
in the tests above is about 5x the size of the gzipped version.
> extraneous and noisy forum traffic.
One man’s noise is another man’s signal.
It would be very nice to be able to search the mailing list from within Fossil
UI, getting only one copy of each matching message, not one from
mail-archive.com, one from MARC, one from Nabble, one from …
I happen know where we can get a pretty good FTS implementation. ;)
This feature would be even more useful on private projects, where we cannot
rely on the likes of mail-archive.com to gather and index our discussions.
> This is especially true if attachments are
> allowed on forum posts
You can fob that off on third-party services like Imgur, for the most part,
just as Stack Exchange does.
If a post really really needs a repo-embedded attachment, it can use an
embedded doc link, just as you’d do for a Fossil wiki article today.
>> 3. Forum posts can show up in the timeline.
>
> Yikes. I think I would certainly want that to be turned off by default.
The timeline search type field defaults to “Check-ins”, does it not? You’d
only see forum posts with “Any Type” selected or with a new “Forum Posts”
filter selected.
Even then, it should be quite manageable. With the default 100 event search
window size, you’d still be seeing several days of activity with the current
level of SQLite mailing list traffic, most of the time. SQLite’s mailing list
traffic must be somewhere over the 90th percentile in terms of daily traffic,
so most Fossil-hosted software projects would show even more days of activity
in that 100 event window.
To clarify, I don’t mean that the whole forum post would show up in the
Timeline, just a brief summary, as currently happens with tickets and wiki
article events.
_______________________________________________
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
