Re: [sqlite] Best way to store key,value pairs

2020-01-14 Thread Peter da Silva
Another thing to consider is that RFC-822/1036 (et seq) headers are not inherently unique and some are repeated, especially since there's software that treats Usenet and Mail headers interchangeably. Also, debugging may require being able to see the exact layout of the headers as received. The

Re: [sqlite] Best way to store key,value pairs

2020-01-14 Thread Dominique Devienne
On Tue, Jan 14, 2020 at 9:35 AM Wout Mertens wrote: > On Mon, Jan 13, 2020 at 10:45 PM James K. Lowden > This is a trade-off between schema simplicity, storage layout and speed of > some operations. I'd > argue that in this particular case, a JSON field is beneficial for > simplicity, speed

Re: [sqlite] Best way to store key,value pairs

2020-01-14 Thread Wout Mertens
On Mon, Jan 13, 2020 at 10:45 PM James K. Lowden wrote: > You're making an efficiency argument here, or maybe > ease-of-implementation assertion. For me, inserting one header row or > 20 is the same coding effort (still need a loop). I think transaction > throughput would be about the same if

Re: [sqlite] Best way to store key,value pairs

2020-01-13 Thread Jens Alfke
> On Jan 13, 2020, at 1:45 PM, James K. Lowden wrote: > > So, basically, a nomalized design requires too much use of INSERT? > You're making an efficiency argument here, or maybe > ease-of-implementation assertion. For me, inserting one header row or > 20 is the same coding effort (still

Re: [sqlite] Best way to store key,value pairs

2020-01-13 Thread James K. Lowden
On Mon, 13 Jan 2020 10:37:57 -0800 Jens Alfke wrote: > > On Jan 12, 2020, at 4:12 PM, James K. Lowden > > wrote: > > > > What is the motivation behind this advice? It's completely > > unnecessary. > > Thanks for your opinion, James! I disagree. > ... > IMHO there are too many of them to

Re: [sqlite] Best way to store key,value pairs

2020-01-13 Thread Petite Abeille
> On Jan 13, 2020, at 19:37, Jens Alfke wrote: > >> What is the motivation behind this advice? It's completely unnecessary. > > Thanks for your opinion, James! I disagree. Arnt Gulbrandsen, of Archiveopteryx fame, would disagree with you, Jens :) https://archiveopteryx.org/schema In

Re: [sqlite] Best way to store key,value pairs

2020-01-13 Thread Jens Alfke
> On Jan 12, 2020, at 4:12 PM, James K. Lowden wrote: > > What is the motivation behind this advice? It's completely unnecessary. Thanks for your opinion, James! I disagree. RFC822 headers are schemaless, and in a Usenet or email database they have rather high volume (probably 20+ per

Re: [sqlite] Best way to store key,value pairs

2020-01-12 Thread James K. Lowden
On Tue, 7 Jan 2020 17:11:45 -1000 Jens Alfke wrote: > Consider encoding the headers as JSON and storing them in a single > column. SQLite has a JSON extension that makes it easy to access > values from JSON data in a query. What is the motivation behind this advice? It's completely

Re: [sqlite] Best way to store key,value pairs

2020-01-11 Thread Jens Alfke
> On Jan 8, 2020, at 3:13 AM, Simon Slavin wrote: > > I believe Jens' point is valid, as long as you don't have to search/scan > headers. You can even do that — to search for a specific header’s value, just create an index on json_extract(headers, ‘$Header-Name’), then in a query use that

Re: [sqlite] Best way to store key,value pairs

2020-01-08 Thread Simon Slavin
On 8 Jan 2020, at 3:11am, Jens Alfke wrote: > Consider encoding the headers as JSON and storing them in a single column. > SQLite has a JSON extension that makes it easy to access values from JSON > data in a query. You can even index them. > > Simon’s suggestion (a row per header) is correct

Re: [sqlite] Best way to store key,value pairs

2020-01-07 Thread Jens Alfke
Consider encoding the headers as JSON and storing them in a single column. SQLite has a JSON extension that makes it easy to access values from JSON data in a query. You can even index them. Simon’s suggestion (a row per header) is correct in theory, but the large numbers of headers you’ll be

Re: [sqlite] Best way to store key,value pairs

2020-01-07 Thread Simon Slavin
On 7 Jan 2020, at 7:13pm, Andy wrote: > But it is variable number this fields and column are strictly defined. Maybe > all in one Sqlite text field or define for example 10 field which often will > empty? Parent Child database. One table has one entry per message. The other table has one

[sqlite] Best way to store key,value pairs

2020-01-07 Thread Andy
I am trying using Sqlite to news reader and local small NNTP server. NNTP message has header and body. Some special headers field needed to XOVER I keep in one table, body I will compress and store as blob, some other required header fields I will store in own columns, but also exists not