[sqlite] Any way to disable transactional behavior?

2016-10-09 Thread Hayden Livingston
I have a program that writes hundreds of thousands of statements that are logically unrelated, i.e. each is "transactional". The problem is if I don't do a BEGIN TRANSACTION and do my inserts, it takes absolutely forever for my program to finish (we're talking hours). If instead I do it in a

[sqlite] Comparing same data in two tables in two separate databases

2015-07-21 Thread Hayden Livingston
Hello, I would like to compare two tables (of the same name) in two separate database files. I could conceivably change the situation such that I have 2 tables within the same database but different names. My table definition is: CREATE TABLE (SchoolId INTEGER, SchoolName TEXT, EnrollmentCount

[sqlite] Adding PRAGMA=UTF-8 makes INSERT slower?

2015-07-18 Thread Hayden Livingston
er things that might be involved, > we have to guess, and as you have (no doubt) noticed from many of the > replies - we often guess very wrong. > > > > On 2015-07-18 07:37 AM, Hayden Livingston wrote: >> >> I was getting garble in my SQLite database, so I switched

[sqlite] Adding PRAGMA=UTF-8 makes INSERT slower?

2015-07-17 Thread Hayden Livingston
I was getting garble in my SQLite database, so I switched PRAGMA=UTF-16 on for my INSERT statements. These are getting prepared. I noticed my total time dramatically increased. I then switched to UTF-8 thinking it's the increased writes causing it, no noticeable difference, i.e. it's just as

[sqlite] storing data across multiple tables

2015-07-17 Thread Hayden Livingston
'll be string parsing all the rows that their type belongs to. On Fri, Jul 17, 2015 at 9:46 AM, Simon Slavin wrote: > > On 17 Jul 2015, at 5:27pm, Hayden Livingston > wrote: > >> Sorry, they do not have the same definition. They are different >> definitions. I just want

[sqlite] storing data across multiple tables

2015-07-17 Thread Hayden Livingston
at 3:48 AM, Simon Slavin wrote: > > On 17 Jul 2015, at 3:22am, Hayden Livingston > wrote: > >> So, In my application code I'm going to through loop through all the >> tables. The table count is usually high hundreds (~600-800) > > Why do you have so many tables whic

[sqlite] Improving performance by removing MapParameter calls?

2015-07-17 Thread Hayden Livingston
In my program I'm spending most of my time in this function: System.Data.SQLite!SQLiteStatement.MapParameter This function is doing string comparisons to map the correct parameter. I do put my SQL command text into a statement and prepare it. I was wondering if there is a way to do so for my

[sqlite] storing data across multiple tables

2015-07-16 Thread Hayden Livingston
Thanks, Keith for correcting my usage. I'm referring to tables when I mean database schema, so pardon my technical terminology. I have multiple tables of data already, say TableA, TableB. These tables have different representations, one may contain a TEXT column, the other may contain and

[sqlite] storing data across multiple tables

2015-07-16 Thread Hayden Livingston
I have different schema data that I'd like to store across tables, and it is sorted in chronological order. But then I also want to be able to make queries across tables when I do a query that says after time X, I want it to show me all rows -- obviously different schema doesn't make this

[sqlite] why does SQLite support VARCHAR(X) when the doc says it ignores it?

2015-07-16 Thread Hayden Livingston
Thanks! On Thu, Jul 16, 2015 at 2:50 PM, Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 07/16/2015 02:46 PM, Hayden Livingston wrote: >> It seems that >> >> CREATE TABLE A( something varchar(255) ) >> >> it just >> &g

[sqlite] why does SQLite support VARCHAR(X) when the doc says it ignores it?

2015-07-16 Thread Hayden Livingston
It seems that CREATE TABLE A( something varchar(255) ) it just CREATE TABLE A( something text ) Why have it at all?

[sqlite] Schema-less JSON SQLite DB?

2015-07-15 Thread Hayden Livingston
I'm beginning to understand some of the implementation challenges here. Eric, it'd be great to get that open sourced. I can tell you at least I will jump and prototype on it. Sergej and Roger, I do like this potential idea as well. Of course, now it's a custom sqlite, but at least the job gets

[sqlite] Schema-less JSON SQLite DB?

2015-07-14 Thread Hayden Livingston
including the querying and indexing. -- Darren Duncan > > > On 2015-07-13 5:43 PM, Hayden Livingston wrote: >> >> Is there a concept of a schema-less JSON SQLite DB? >> >> My reason is simple: versioning. We have lot of business metrics that >> get updated let's

[sqlite] Schema-less JSON SQLite DB?

2015-07-14 Thread Hayden Livingston
a document I can read about it'd be great. On Mon, Jul 13, 2015 at 8:34 PM, Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 07/13/2015 08:00 PM, Hayden Livingston wrote: >> Does your code also map object hierarchies in json? > > Yes, but

[sqlite] Schema-less JSON SQLite DB?

2015-07-13 Thread Hayden Livingston
gt; On 07/13/2015 05:43 PM, Hayden Livingston wrote: >> Is there a concept of a schema-less JSON SQLite DB? > > What exactly is it you want? A schema-less database? A JSON > database? Using SQLite to store JSON? (It is unclear if you mean > something like SQLite, or SQLite itsel

[sqlite] Schema-less JSON SQLite DB?

2015-07-13 Thread Hayden Livingston
Is there a concept of a schema-less JSON SQLite DB? My reason is simple: versioning. We have lot of business metrics that get updated let's say once a month, and we need to be agile to get them. Right now, we just put the version in the SQLite file, and then make sure no queries cross the

[sqlite] Storing opaque data, but would like sorted indexes on some fields

2014-05-30 Thread Hayden Livingston
I have a binary format that is effectively structured data. I currently have multiple indexes but none of them are sorted, because in my toy system ORDER BY's are not supported, an implicit ORDER BY time of record inserted exists because it is a single threaded application. My indexing story is

Re: [sqlite] Using SQLite for write only workload, splitting databases into smaller files, but need to query it as a whole

2014-05-03 Thread Hayden Livingston
Ok. But I'm wondering how GROUPBY and stuff will work? On Sat, May 3, 2014 at 7:13 AM, Petite Abeille <petite.abei...@gmail.com>wrote: > > On May 3, 2014, at 3:40 PM, Hayden Livingston <halivings...@gmail.com> > wrote: > > > This looks promising. I sooo wish it

Re: [sqlite] Using SQLite for write only workload, splitting databases into smaller files, but need to query it as a whole

2014-05-03 Thread Hayden Livingston
to do an N way JOIN? On Sat, May 3, 2014 at 6:16 AM, Petite Abeille <petite.abei...@gmail.com>wrote: > > On May 3, 2014, at 2:59 PM, Hayden Livingston <halivings...@gmail.com> > wrote: > > > Thoughts? > > Take a look at ‘ATTACH’, it might help: > &

Re: [sqlite] Using SQLite for write only workload, splitting databases into smaller files, but need to query it as a whole

2014-05-03 Thread Hayden Livingston
org> wrote: > > On 3 May 2014, at 1:59pm, Hayden Livingston <halivings...@gmail.com> > wrote: > > > My workload is quite straightforward, I have a multi-threaded application > > that logs to this file but from within any single thread at any given > time. > >

[sqlite] Using SQLite for write only workload, splitting databases into smaller files, but need to query it as a whole

2014-05-03 Thread Hayden Livingston
I have a situation where I store a data in a text file with tabs, and it's becoming really quite unwieldy to use this. My workload is quite straightforward, I have a multi-threaded application that logs to this file but from within any single thread at any given time. So from SQLite's perspective

Re: [sqlite] Selecting rows as "groups" -- "pivots"?

2013-12-05 Thread Hayden Livingston
ers per hour rate. On Thu, Dec 5, 2013 at 4:29 PM, Igor Tandetnik <i...@tandetnik.org> wrote: > On 12/5/2013 7:16 PM, Hayden Livingston wrote: > >> I suppose I should have stated my goal even further. I'm putting them in >> sql so that I can query it row by row (

Re: [sqlite] Selecting rows as "groups" -- "pivots"?

2013-12-05 Thread Hayden Livingston
Tandetnik <i...@tandetnik.org> wrote: > On 12/5/2013 4:31 PM, Hayden Livingston wrote: > >> --> End Goal: >> >> A SQL Table: >> >> FooId,SomeStringValue,SomeIntValue,SomeOtherColumn, >> WhateverColumnNameItNeedsTo, >> > > You do not want

Re: [sqlite] Selecting rows as "groups" -- "pivots"?

2013-12-05 Thread Hayden Livingston
ou > wish to deduce from the stored data? Maybe we can come up with more > helpful suggestions (as opposed to just looking puzzled!). > > Cheers, > Ryan > > > On 2013/12/05 20:54, Igor Tandetnik wrote: > >> On 12/5/2013 1:43 PM, Hayden Livingston wrote: >> >

Re: [sqlite] Selecting rows as "groups" -- "pivots"?

2013-12-05 Thread Hayden Livingston
Id | RelationalIdentifier | ColC 1aZ 2bC How can I adapt your query to this new table schema? On Thu, Dec 5, 2013 at 10:33 AM, Igor Tandetnik <i...@tandetnik.org> wrote: > On 12/5/2013 1:15 PM, Hayden Livings

[sqlite] Selecting rows as "groups" -- "pivots"?

2013-12-05 Thread Hayden Livingston
I have a table schema such like ID | Col1 | Col2 | Col3 | Value 1 anull nullX 2 null a nullY 3 null null a Z 4 b nullnull A 5 null b nullB 6 null null b C Right now these are in

Re: [sqlite] Percentile function in SQLite 3.8.0 (for the .NET version)

2013-12-02 Thread Hayden Livingston
sfied with the responses? > > > > On 2013/12/02 19:30, Hayden Livingston wrote: > >> Is there any documentation or use case for percentile queries? I've only >> seen the source code checked in by "drh". >> >> Is it like the EXCEL 2013 function? >> >> I h

Re: [sqlite] Percentile function in SQLite 3.8.0 (for the .NET version)

2013-12-02 Thread Hayden Livingston
Ok great. Do you know if the 1.0.90.0 will be out anytime soon? That is, this month or so ...? On Mon, Dec 2, 2013 at 11:54 AM, Joe Mistachkin <sql...@mistachkin.com>wrote: > > Hayden Livingston wrote: > > > > (A) How do I create a Percentile Query? > > >

[sqlite] Percentile function in SQLite 3.8.0 (for the .NET version)

2013-12-02 Thread Hayden Livingston
Is there any documentation or use case for percentile queries? I've only seen the source code checked in by "drh". Is it like the EXCEL 2013 function? I have a table {operationId, latency) with a set of rows that have floating values for latencies of a given operation. I want to find the 25th

[sqlite] Percentile function in SQLite 3.8.0 (for the .NET version)

2013-11-30 Thread Hayden Livingston
Is there any documentation or use case for percentile queries? I've only seen the source code checked in by "drh". Is it like the EXCEL 2013 function? I have a table {operationId, latency) with a set of rows that have floating values for latencies of a given operation. I want to find the 25th