Re: [sqlite] LevelDB benchmark

2011-07-27 Thread Stephan Wehner
On Wed, Jul 27, 2011 at 7:00 PM, Simon Slavin <slav...@bigfraud.org> wrote:
>
> On 28 Jul 2011, at 2:53am, Stephan Wehner wrote:
>
>> On Wed, Jul 27, 2011 at 6:44 PM, Simon Slavin <slav...@bigfraud.org> wrote:
>>>
>>> On 28 Jul 2011, at 2:22am, Stephan Wehner wrote:
>>>
>>>> There are some benchmark's at
>>>> http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html
>>>>
>>>> I don't have anything to point to, but I thought sqlite3 does better
>>>> than stated there.
>>>
>>> i looked through their source code, trying to see if they defined 
>>> transactions. But I couldn't even find an INSERT command.
>>
>> Well, LevelDB is much simpler than sqlite3: it's a key-value store.
>
> Okay, but if they include their source code for testing SQLite I should be 
> able to find the word 'INSERT' somewhere in it, right ?
>

Sorry, I misunderstood -- S

> Simon.
> ___________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] LevelDB benchmark

2011-07-27 Thread Stephan Wehner
On Wed, Jul 27, 2011 at 6:44 PM, Simon Slavin <slav...@bigfraud.org> wrote:
>
> On 28 Jul 2011, at 2:22am, Stephan Wehner wrote:
>
>> There are some benchmark's at
>> http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html
>>
>> I don't have anything to point to, but I thought sqlite3 does better
>> than stated there.
>
> i looked through their source code, trying to see if they defined 
> transactions.  But I couldn't even find an INSERT command.
>

Well, LevelDB is much simpler than sqlite3: it's a key-value store.

Stephan


> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] LevelDB benchmark

2011-07-27 Thread Stephan Wehner
There are some benchmark's at 
http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html

I don't have anything to point to, but I thought sqlite3 does better 
than stated there.

In particular, 26,900 sequential writes per second and 420 random writes 
per second from section "1. Baseline Performance" look suspicious.

What you say?

Stephan
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Hidding records from the application

2011-07-16 Thread Stephan Wehner
On Fri, Jul 15, 2011 at 6:19 PM, san long <kerneltrap...@gmail.com> wrote:
> Dear all,
> I have an idea related to the safety of the records in a table: if it is
> possible to hide some records in a table so the upper user application could
> not see them?
> For example:
> table food has content:
> 1, "food A"
> 2, "food B"
> I want to hide the record whose rowid is 2, so:
> sqlite> SELECT  * from food;
> --
> 1, "food A"
> ---
>
> Is there any good solution to satisfy my requirement?


Some queries may depend on the hidden items. For example, table food
has column price.

select sum(price) from food;

-> Is the hidden food's price included?

Other queries may depend on entries not being present.

delete from customers where "customer has not ordered hidden foods"

-> Is the customer record gone?

Can you look further up in your requirements? A different level of the
application
may find it easier (Check before "displaying" any column value whether
user has read-access)

Stephan

> Thanks in advance.
> long san
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] Pretty schema ?

2011-04-13 Thread Stephan Wehner
Hello everybody,

is there a way to get a pretty-print version of

sqlite> .schema

Would be nice.

Stephan

-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] how to get a reliable Integer from a Real?

2011-02-02 Thread Stephan Wehner
On Wed, Feb 2, 2011 at 10:25 AM, Oliver Peters <oliver@web.de> wrote:
> Hello,
>
> I'm on Win XP with sqlite 3.7.5 and trying to get cents from euro but 
> sometimes
> I get a strange result:
>
> SQLite version 3.7.5
> Enter ".help" for instructions
> Enter SQL statements terminated with a "
> sqlite> SELECT CAST(2.1*100 AS INTEGER);
> 210
> sqlite> SELECT CAST(2.2*100 AS INTEGER);
> 220
> sqlite> SELECT CAST(2.3*100 AS INTEGER);
> 229 <- S T R A N G E result
> sqlite> SELECT CAST(2.4*100 AS INTEGER);
> 240
> sqlite> SELECT CAST(2.5*100 AS INTEGER);
> 250
> sqlite> SELECT CAST(2.6*100 AS INTEGER);
> 260
> sqlite> SELECT CAST(2.7*100 AS INTEGER);
> 270
> sqlite> SELECT CAST(2.8*100 AS INTEGER);
> 280
> sqlite> SELECT CAST(2.9*100 AS INTEGER);
> 290
> sqlite> SELECT CAST(3.0*100 AS INTEGER);
> 300
> sqlite> SELECT CAST(3.1*100 AS INTEGER);
> 310
> sqlite> SELECT CAST(3.2*100 AS INTEGER);
> 320
> sqlite> SELECT CAST(3.3*100 AS INTEGER);
> 330
> sqlite> SELECT CAST(3.4*100 AS INTEGER);
> 340
> sqlite>

Can you store the euro's as cents? Then you'll have the opposite
problem of converting cents to euro's but that is much safer.

Quick reference for further reading,
http://stackoverflow.com/questions/618056/what-is-the-best-way-to-store-a-money-value-in-the-database

Stephan

>
> Is the result for 2.3 expected behaviour (strange floating point arithmetic
> reasons?) and is
>
> sqlite> SELECT CAST(2.3*1000 AS INTEGER)/10;
> 230
> sqlite>
>
> a reliable way to get rid of my problem (even if the Real has 2 digits behind
> the .)?
>
> greetings
> Oliver
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Speed up DELETE of a lot of records

2010-10-08 Thread Stephan Wehner
On Fri, Oct 8, 2010 at 9:49 AM, Simon Slavin <slav...@bigfraud.org> wrote:
>
> On 8 Oct 2010, at 5:48pm, Stephan Wehner wrote:
>
>> On Fri, Oct 8, 2010 at 7:14 AM, Michele Pradella
>> <michele.prade...@selea.com> wrote:
>>>  "science fiction?" was a rhetorically question. I'm only wondering
>>> about what is the best and fastest way to DELETE a lot of records from
>>> huge DB. I know and understand physical limit of data moving: anyway for
>>> now I'm trying to split the BIG DELETE in some smaller DELETE to spread
>>> the time used. It's the only way I can figure out at the moment.
>>
>> Is a soft-delete faster? Then you could add a slow-moving delete
>> (mentioned earlier by Aldes Rossi, for example)
>> for the soft-deleted records.
>
> Soft-delete ?  Is that having another column which is a '1' if the record is 
> supposed to exist and a '0' if it's supposed to be deleted ?
>

Yes, that's what I meant. Usually call the column 'deleted'; 1 means
deleted, 0 means not-deleted.

Stephan

> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Speed up DELETE of a lot of records

2010-10-08 Thread Stephan Wehner
On Fri, Oct 8, 2010 at 7:14 AM, Michele Pradella
<michele.prade...@selea.com> wrote:
>  "science fiction?" was a rhetorically question. I'm only wondering
> about what is the best and fastest way to DELETE a lot of records from
> huge DB. I know and understand physical limit of data moving: anyway for
> now I'm trying to split the BIG DELETE in some smaller DELETE to spread
> the time used. It's the only way I can figure out at the moment.
>

Is a soft-delete faster? Then you could add a slow-moving delete
(mentioned earlier by Aldes Rossi, for example)
for the soft-deleted records.

Stephan

> Il 08/10/2010 15.55, Jay A. Kreibich ha scritto:
>> On Fri, Oct 08, 2010 at 09:09:09AM +0200, Michele Pradella scratched on the 
>> wall:
>>>    I was thinking this too, but I take this for last chance: my hope is I
>>> can delete 5 millions of records in few seconds, science fiction? :)
>>    Science fiction of the worst B-grade sort.
>>
>>    Think about the numbers.  You're talking about updating a significant
>>    chunk of a multi-gigabyte file.  The WAL file tells you the changes
>>    amount to ~600MB of writes.  That's a whole CDs worth of data.  These
>>    days that might not be much for storage, but it is still a lot of
>>    data to move around.  Even if your storage system has a continuous,
>>    sustained write ability of 20MB/sec, that's a half minute.  How fast
>>    can your disk copy 600MB worth of data?
>>
>>    But you're not just writing.  You're doing a lot of reads from all
>>    over the file in an attempt to figure out what to modify and write.
>>    Both the reads and the writes (the integration, at least) are
>>    scattered and small, so you're not going to get anywhere near the
>>    sustained performance levels.  10x slower would be extremely good.
>>
>>    Or think of it in more physical numbers... If you're using a single
>>    vanilla disk, it likely spins at 7200 RPMs.  If it takes five minutes
>>    to update 5,000,000 records, that's an average of almost 140 records
>>    per disk revolution.  That's pretty good, considering everything else
>>    that is going on!
>>
>>
>>
>>    The only possible way to manipulate that much data in a "few seconds"
>>    is to load up on RAM, get a real operating system, and throw the
>>    whole database into memory.  Or spend many, many, many thousands of
>>    dollars on a very wide disk array with a very large battery-backed
>>    cache and a huge pipe between your host and the array.
>>
>>    Big storage is cheap.  Fast storage is not.  Don't confuse the two.
>>
>>     -j
>>
>>
>
>
> --
> Selea s.r.l.
>
>
>        Michele Pradella R
>
>
>        SELEA s.r.l.
>
> Via Aldo Moro 69
> Italy - 46019 Cicognara (MN)
> Tel +39 0375 889091
> Fax +39 0375 889080
> *michele.prade...@selea.com* <mailto:michele.prade...@selea.com>
> *http://www.selea.com*
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Getting unique years from a timestamp column

2010-09-29 Thread Stephan Wehner
On Wed, Sep 29, 2010 at 9:34 AM, J. Bobby Lopez <j...@jbldata.com> wrote:
> Hello,
>
> I'm looking to do something completely in SQLite, without the assistance of
> Perl.
>
> The problem is this:
>
> I have a table with a text column, and the column contains unix timestamps.
> I would like to get a list of the unique years from that timestamp column.
>
> Here's what the raw dataset looks like:
>
> 1|blopez|somekinda.log|2010-07-10 13:21:10|10.100.0.1|make_db.pl
> |usage_reporter()
> 2|blopez|somekinda.log|2010-09-28 06:18:51|10.100.0.1|make_db.pl
> |usage_reporter()
> 3|blopez|somekinda.log|2010-06-28 17:58:37|10.100.0.1|make_db.pl
> |usage_reporter()
> 4|blopez|somekinda.log|2011-06-28 17:58:37|10.100.0.1|make_db.pl
> |usage_reporter()
>
> What I'd like to do is write a query that would return "2010" and "2011",
> the two unique years in the listing.
>
> It's pretty easy to get all records which match a single year, for example:
>
> SELECT id FROM data WHERE datetime(date) LIKE '2010%';
>

This might work for you

  select distinct strftime('%Y', date)  from data;

(Assuming your table is named "data", and the column with the time
stamp is called "date")

Stephan

> I'm sure I could use a BEGIN/COMMIT block and test for each year
> individually, but I don't want to hard-code the year that I'm looking for,
> if you get my meaning.
>
> Any assistance on this would be appreciated.  Thanks!
>
> Bobby
> _______
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Question concerning foreign keys across databases

2010-08-18 Thread Stephan Wehner
On Wed, Aug 18, 2010 at 1:39 PM, Pavel Ivanov <paiva...@gmail.com> wrote:
>> Summary: how can I use foreign keys across database boundaries? Is it at
>> all possible?
>
> No. It's logically incorrect action, so it's impossible. If you want
> consistency of your tables to be automatically checked by database
> engine you need to allow that engine to see those tables at all times.
> For SQLite it means that you need to keep those tables in one
> database. If you insist on keeping tables in different databases then
> your application should check consistency itself because it's the only
> one knowing how to keep track of different files.
>

So attaching the file1 before creating the table in file2
is going to fail? (Then sqlite would know about the {texts} table)

Stephan

>
> Pavel
>
> On Wed, Aug 18, 2010 at 4:26 PM, Oliver Schneider <oli...@f-prot.com> wrote:
>> Hello,
>>
>> when trying to use foreign keys I'm running into a problem. It could be
>> that I hit some general limitation, but then again the error that I'm
>> seeing could also be issued if the database with the table of the
>> referenced foreign key is not attached.
>>
>> Here's what I'm trying to do.
>>
>> I have some meta-data that rarely changes (e.g. text strings with
>> accompanying IDs) but is rather big. Suppose we have:
>>
>> -
>> pragma foreign_keys = on;
>>
>> create table texts ( id integer primary key autoincrement, str text
>> unique not null on conflict fail );
>>
>> /* With the following data */
>> insert into texts(str) values('foo');
>> insert into texts(str) values('bar');
>> insert into texts(str) values('baz');
>> -
>>
>> I'll call this table {texts} from now on. It's contained in file1. Now
>> comes some more variable data where I would prefer to use one database
>> per "data set", so I create another database contained in file2 (which
>> becomes {main}):
>>
>> -
>> pragma foreign_keys = on;
>>
>> attach database file1 as text;
>>
>> /* This fails right away:
>>
>> create table main.result ( id integer primary key autoincrement, strid
>> integer, details text default null, foreign key(strid) references
>> text.texts(id) );
>>
>> ... so I use: */
>>
>> create table main.result ( id integer primary key autoincrement, strid
>> integer, details text default null, foreign key(strid) references
>> texts(id) );
>>
>> /* Which succeeds for the moment, until ... */
>>
>> insert into result(strid,details) values (1,'some foo value');
>> -
>>
>> This last line gives me "Error: no such table: main.texts", which, I
>> suppose, is due to the constraint check for the foreign key "strid".
>>
>> The given error message obviously makes sense. However, since the
>> constraint check is done upon insertion referencing an attached database
>> shouldn't really pose a problem, right? At worst - e.g. if I hadn't
>> attached "file1 as text" - I would get the that error with a slight
>> variation: "Error: no such table: text.texts" ...
>>
>> Summary: how can I use foreign keys across database boundaries? Is it at
>> all possible?
>>
>>
>> Thanks,
>>
>> // Oliver
>>
>> PS: I'm using SQLite version 3.6.22
>> PPS: Tried to send it with PGP/MIME signature, but it didn't get through
>> according to the list archive, so sending without signature.
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Encryption of sqlite DB

2010-05-04 Thread Stephan Wehner
On Tue, May 4, 2010 at 9:19 AM, Kavita Raghunathan
<kavita.raghunat...@skyfiber.com> wrote:
> Hi,
> What’s the simplest way to encrypt only certain rows in an sqlite DB? If 
> there is no way to do this (for storing passwords etc), I would like to know 
> the best way to encrypt the whole sqlite DB. (Prefer only encrypting some 
> rows, but if this introduces complexity, I’m willing to encrypt the whole 
> database)

Do you mean you want to encrypt only some columns ?

Stephan

> kavita
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Computing day of week the using strftime() function - Feature or bug ?

2010-03-23 Thread Stephan Wehner
On Tue, Mar 23, 2010 at 12:25 PM, Bernie Reiter
<bernie_on_the_road_ag...@yahoo.co.uk> wrote:
> Dear Dr. Hipp,

> SELECT strftime('%w',2010-03-22 22:12:40) ; => SQL error near "22": syntax 
> error
Add quotes:

SELECT strftime('%w','2010-03-22 22:12:40');

See the section Examples at http://www.sqlite.org/lang_datefunc.html


Stephan

>
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Parsing create statements

2010-01-20 Thread Stephan Wehner
On Wed, Jan 20, 2010 at 6:13 AM, BareFeet <list@tandb.com.au> wrote:
> Hi All,
>
> This question seems to come up every now and then, but I've yet to see any 
> solution. The best phrasing of the question I found was back in 2004:
> http://www.mail-archive.com/sqlite-users@sqlite.org/msg02649.html
> Is there any way to parse a create statement (eg create table, create view, 
> create trigger) into its components?



Does this help?

  pragma table_info([table-name])

The output looks easier to parse than the create-table statement.

See http://www.sqlite.org/pragma.html#schema

Stephan


>
> Since SQLite does this internally using the Lemon parser, surely there's a 
> simple way to get the parser results?
>
> For example, given:
>
> create table "People"
> (       ID integer primary key
> ,       Name text collate nocase
> ,       "Family ID" integer
>                references Family(ID)
> )
>
> break it up into something like:
>
> table name: People
>
> table columns:
>  1:
>    name: ID
>    type: integer
>    constraints:
>      1:
>         name:
>         detail: primary key
>  2:
>    name: Name
>    type: text
>    constraints:
>      1:
>         name:
>         detail: collate nocase
>  3:
>    name: Family ID
>    type: integer
>    constraints:
>      1:
>         name:
>         detail: references Family(ID)
>         foreign table: Family
>         foreign column: ID
>
> I'm writing in Objective-C, so anything accessible to it would be helpful.
>
> Thanks,
> Tom
> BareFeet
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
-> http://twitter.com/stephanwehner / @stephanwehner
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] command line does not accept arrow keys

2009-11-20 Thread Stephan Wehner
On Fri, Nov 20, 2009 at 6:02 PM, Qianqian Fang
<fa...@nmr.mgh.harvard.edu> wrote:
> hi
>
> I can not type arrow keys in the sqlite3 command line, all the arrow
> keys (as well as other keys in the small keyboard) will be shown as
> escape sequence "^]]A".

Could it be a problem with readline? See

http://www.sqlite.org/cvstrac/wiki?p=ReadLine

Stephan

> I am using Terminator/gnome-terminal, is there a setting I need to
> twig in order to get this working?
>
> thanks
>
> Qianqian
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] is ticket 4037 not a bug?

2009-11-17 Thread Stephan Wehner
On Mon, Nov 16, 2009 at 9:49 PM, Roger Binns <rog...@rogerbinns.com> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Bogdan Ureche wrote:
>> I just noticed the ticket was closed without being fixed, with the following
>> remark:
>
> I was the one who closed it and added that remark.
>
>> Now, I have to admit that English is not my first language, nor the second,
>> but the way I see it this remark means something like: "This is not a bug,
>> and if you want to find out why please ask this question in the mailing
>> list. "
>
> Not quite :-) The CVSTRAC tracker has been closed and the replacement
> mechanism is described at http://www.sqlite.org/src/wiki?name=Bug+Reports
>
> In cases where it was obvious that the issue was a bug then I copied it over
> to the new one http://www.sqlite.org/src/timeline?n=200=t
>
> In cases like this one it is not necessarily a bug and I don't have enough
> knowledge to make the call.  That is why the suggestion to take it up with
> this group where there are people familiar with the standard and the
> behaviour of other databases.
>
>> So here I am asking the question here: is this a bug or not?
>
> Which is exactly what we want to know :-)  If the SQL standard specifies it
> then it is a very strong candidate for fixing.  If all other databases do it
> then that is also strong.  When only a subset do then that is not good.

Here's what happens with mysql:

$ mysql -u mysqluser -p

Enter password:

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 1

Server version: 5.1.30 MySQL Community Server (GPL)



mysql> select databasename.tablename.* from databasename.tablename;

ERROR 1146 (42S02): Table 'databasename.tablename' doesn't exist

mysql> create database databasename1;

Query OK, 1 row affected (0.01 sec)



mysql> connect databasename1;

Connection id:2

Current database: databasename1



mysql> create table tablename1(data varchar(2));

Query OK, 0 rows affected (0.01 sec)



mysql> select databasename1.tablename1.* from databasename1.tablename1;

Empty set (0.00 sec)



mysql>
quit;

Note -- no "syntax error", instead, and plausibly: "ERROR 1146","Table
'databasename.tablename' doesn't exist".

Stephan

If
> it alters the behaviour compared to previous SQLite versions then breaking
> that backwards compatibility would require exceptionally compelling evidence.
>
> Roger
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAksCOYAACgkQmOOfHg372QSt6ACgjAAwTucstMRp950Ce4dFjxs5
> /hgAoNtrs/2oAPab4vjaZbi1ZNkCE56E
> =9tn8
> -END PGP SIGNATURE-
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] in-memory database concept

2009-11-05 Thread Stephan Wehner
On Wed, Nov 4, 2009 at 4:57 AM, Serdar Genc <serdar.g...@gmail.com> wrote:
> Yes I am aware of that but I need to access an in-memory database from two
> processes which were written in Java and C++.
> I am just looking for a way to do so..

Hey Serdar,

do you have any update on this ? I found your problem quite
interesting, although also difficult.

Stephan

> On Wed, Nov 4, 2009 at 2:50 PM, Jay A. Kreibich <j...@kreibi.ch> wrote:
>
>> On Wed, Nov 04, 2009 at 10:27:57AM +0200, Serdar Genc scratched on the
>> wall:
>>
>> > But when I try to do this for in-memory, everything is getting chunky. A
>> new
>> > database is being created in memory for every attempt to open
>> > database in memory. Now I am trying to switch the database handler
>> between
>> > C++ and Java. It requires some efforts.
>>
>>   Yes, that's how it is supposed to work.  A ":memory:" database can
>>  only be accessed by the database handle that created it.  They're not
>>  designed to be shared.
>>
>>  http://www.sqlite.org/inmemorydb.html
>>
>>   -j
>>
>> --
>> Jay A. Kreibich < J A Y  @  K R E I B I.C H >
>>
>> "Our opponent is an alien starship packed with atomic bombs.  We have
>>  a protractor."   "I'll go home and see if I can scrounge up a ruler
>>  and a piece of string."  --from Anathem by Neal Stephenson
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] in-memory database concept

2009-11-04 Thread Stephan Wehner
On Wed, Nov 4, 2009 at 12:51 AM, Serdar Genc <serdar.g...@gmail.com> wrote:
> The OS that I am implementing this is SunOS and I have already tried doing
> it by creating the file in /tmp but
> but because of file I/O operations to reach /tmp. The speed of sqlite is
> significantly slow (memory is 5 times faster than file system). Speed of
> Sqlite is important for me. That's why I still working on the memory issue.
>

Hi there,

/tmp is not the same as tmpfs -- tmpfs is a file system that stores
files in RAM, but looks otherwise like any other file system.

On SunOS it should be straightforward; that wiki page says "SunOS, and
later Solaris, include probably the earliest implementations of
tmpfs".

Stephan

> Thanks for the answer,
> Serdar Genc
>
> On Wed, Nov 4, 2009 at 10:45 AM, Stephan Wehner 
> <stephanweh...@gmail.com>wrote:
>
>> On Wed, Nov 4, 2009 at 12:27 AM, Serdar Genc <serdar.g...@gmail.com>
>> wrote:
>> > Hello everyone,
>> >
>> > I have a little bit problem about in-memory database feature of SQLite. I
>> > would like to use two programming languages in my application .
>> > These languages will be Java and C++. I would like to reach the same
>> > database in memory by using both. I used Xerial for JDBC for Java.
>> > Everything seems fine if I am using a regular file on harddrive. I can
>> write
>> > data from C++ process and read these data from Java.
>> > But when I try to do this for in-memory, everything is getting chunky. A
>> new
>> > database is being created in memory for every attempt to open
>> > database in memory. Now I am trying to switch the database handler
>> between
>> > C++ and Java. It requires some efforts.
>> >
>> > What I want to try to reach the same database in memory from two threads
>> or
>> > processes and these would be in different programming languages.
>> > Any different idea how to implement this is appreciated..
>>
>> Can you use a RAM based file system?
>>
>>  http://en.wikipedia.org/wiki/Tmpfs
>>
>> Stephan
>>
>> > Thanks in advance,
>> > Serdar Genc
>> > ___
>> > sqlite-users mailing list
>> > sqlite-users@sqlite.org
>> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>> >
>>
>>
>>
>> --
>> Stephan Wehner
>>
>> -> http://stephan.sugarmotor.org (blog and homepage)
>> -> http://loggingit.com
>> -> http://www.thrackle.org
>> -> http://www.buckmaster.ca
>> -> http://www.trafficlife.com
>> -> http://stephansmap.org -- http://blog.stephansmap.org
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] in-memory database concept

2009-11-04 Thread Stephan Wehner
On Wed, Nov 4, 2009 at 12:27 AM, Serdar Genc <serdar.g...@gmail.com> wrote:
> Hello everyone,
>
> I have a little bit problem about in-memory database feature of SQLite. I
> would like to use two programming languages in my application .
> These languages will be Java and C++. I would like to reach the same
> database in memory by using both. I used Xerial for JDBC for Java.
> Everything seems fine if I am using a regular file on harddrive. I can write
> data from C++ process and read these data from Java.
> But when I try to do this for in-memory, everything is getting chunky. A new
> database is being created in memory for every attempt to open
> database in memory. Now I am trying to switch the database handler between
> C++ and Java. It requires some efforts.
>
> What I want to try to reach the same database in memory from two threads or
> processes and these would be in different programming languages.
> Any different idea how to implement this is appreciated..

Can you use a RAM based file system?

  http://en.wikipedia.org/wiki/Tmpfs

Stephan

> Thanks in advance,
> Serdar Genc
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sparse matrix in scientific computing with sqlite

2009-10-16 Thread Stephan Wehner
gt; >
>> >
>> > create temp view rowwiseB as
>> > select count(rowid) as ia from matrix
>> > where value != 0
>> > group by rowid
>> > order by rowid
>> > ;
>> >
>> >
>> >
>> > --Best regards,
>> > --Michael Chen
>> > --Google Voice Phone.: 847-448-0647
>> > ___
>> > sqlite-users mailing list
>> > sqlite-users@sqlite.org
>> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>> >
>>
>>
>>
>> --
>> Puneet Kishor http://www.punkish.org
>> Carbon Model http://carbonmodel.org
>> Charter Member, Open Source Geospatial Foundation http://www.osgeo.org
>> Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor
>> Nelson Institute, UW-Madison http://www.nelson.wisc.edu
>> ---
>> Assertions are politics; backing up assertions with evidence is science
>> ===
>> Sent from Madison, WI, United States
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>
>
>
> --
> Best regards,
> Michael Chen
> Google Voice Phone.: 847-448-0647
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://loggingit.com
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] feature request - field exclusion list in select list

2009-09-27 Thread Stephan Wehner
On Sun, Sep 27, 2009 at 5:19 PM, Darren Duncan <dar...@darrenduncan.net> wrote:
> Following my previous message to the sqlite-users list, I've done a bit more
> research and decided to escalate my reply to a formal feature request.
>
> I was initially going to file a ticket, but it seems that non-registered 
> SQLite
> developers can't do that anymore, and we're supposed to do it on sqlite-users
> where a non-registered developer would then distill list chatter to real
> tickets.  So here I go ...
>
> -
>
> I propose that SQLite (and other DBMSs, and the SQL standard) add support for
> explicitly naming a list of result fields that they do *not* want, intended 
> for
> use in situations where users do want most of the fields from a source but 
> don't
> want a few.  It would make for much more robust code if users can explicitly
> encode their intentions, meaning say that they want all fields except field 
> foo,
> and have them get exactly that (with the expectation that if non-foo fields 
> are
> added or removed to the source, they get or don't get those automatically, 
> same
> as if they did "SELECT *").  I have seen evidence from both first hand
> experience and from many other developers, that this feature would be very
> helpful to them.  And arguably it shouldn't be too terribly complicated for a
> DBMS to implement.

Its a nice idea, but I feel if you use some other interface to SQLite you can
get the same effect.

On the one hand, I think if you are writing a script/program to fetch
the results, then your
script can assemble the effective columns in the SELECT list (all
columns minus the ones
you want to leave out). Meaning you can write your own C function/Perl
method or what-have-you to
get that list, given the database connection, table name and the
columns-to-leave-out.

On the other hand I am not sure, but you seem to be interested in
having this feature
so that you get the functionality at the sqlite3 prompt. In which case
I think you can
use a script that has the columns you're interested in, and save
typing that way (type script once,
load many times / type script once, load, change script, load etc.)

I feel there is great value in not changing software.

Stephan

> Now I know the SQL 2008 standard doesn't have the feature, at least in its
> Foundation; I checked; see section "7.12 " of the SQL
> standard, which deals with the relevant area.  Similarly, the SQLite SQL 
> grammar
> doesn't include it, as seen at
> http://www.sqlite.org/syntaxdiagrams.html#select-core .
>
> What I propose is extending the syntax of what the standard calls " list>".  The old SQL 2008 definition is:
>
>    ::=
>       
>     |  [ {   }... ]
>
> ... and noting that the definition of "" is:
>
>    ::=
>       
>     | 
>
> ... so my proposed redefinition is:
>
>    ::=
>      [ EXCEPT  ]
>
>    ::=
>       
>     |  [ {   }... ]
>
>    ::=
>      [ {   }... ]
>
> So my proposed "" is identical to the old " sublist>", and my addition is the optional EXCEPT plus list of not derived 
> columns.
>
> Note that I'm not stuck on the keyword EXCEPT, but it should be a word that
> reads similarly.
>
> Examples of use:
>
>   SELECT * EXCEPT col4 FROM tbl
>
>   SELECT foo.*, bar.col6 EXCEPT foo.col3 FROM tbl1 INNER JOIN tbl2 USING (id)
>
> The semantics of the change are as if someone wrote the original " sublist>" in normal SQL with extra detail that spelled out all the fields
> individually, and excluded the ones after the EXCEPT.  However, the semantics
> are also that this interpretation is done in the context of when the SQL
> statement is executed, not just when it is written; if the underlying database
> schema changes meanwhile, the result's column list would be affected.
>
> I can also cite prior art in that Chris Date's and Hugh Darwen's Tutorial D 
> has
> the feature I propose, where it is used to modify the relational projection
> operation, and it is spelled with the "ALL BUT" prefix (example "tbl { ALL BUT
> col4 }".  Similarly, my Muldis D language has that feature, spelled with a "!"
> prefix (example "$...@{!col4}"), or as the long-hand "complementary 
> projection"
> operator separate to the "projection" operator.
>
> Note that especially if this proposal is treated favorably by SQLite, I'll go 
> on
> and propose it to other DBMS groups too, starting with Postgres.
>
> Thank you in advance for the consideration.
>
> -- Darren Duncan
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Strange File Import Error

2009-09-22 Thread Stephan Wehner
On Tue, Sep 22, 2009 at 3:04 PM, Simon Slavin
<slav...@hearsay.demon.co.uk> wrote:
>
> On 22 Sep 2009, at 10:49pm, Simon Davies wrote:
>
> If you're going to need to import such files frequently, you might
> need to write some code to do the import properly.

Hey there,

Its not necessarily difficult to use a scripting language, such as
Perl, Ruby, or Python,
to parse the CSV file and write to your database.

For example, Ruby has a "gem" called fastercsv. Such a script would
have around 10 lines of code.

Would also be more robust in the long run.

See you,

Stephan


> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Running test after misc7-6.1.2

2009-09-18 Thread Stephan Wehner
On Fri, Sep 18, 2009 at 7:51 AM, Stephan Wehner <stephanweh...@gmail.com> wrote:
> On Fri, Sep 18, 2009 at 2:35 AM, D. Richard Hipp <d...@hwaci.com> wrote:
>>
>> On Sep 17, 2009, at 10:56 PM, Stephan Wehner wrote:
>>
>>> I downloaded the sqlite3 source ( sqlite-3.6.18.tar.gz ) and managed
>>> to complete "make".
>>>
>>> Now with  "make fulltest", there is no progress for over  almost two
>>> hours. The present output is
>>> misc7-6.1.1... Ok
>>> misc7-6.1.2... Ok
>>>
>>> There is still activity with "top" showing memory/CPU usage for the
>>> "testfixture" process going up and down.
>>> I am guessing the machine is working on misc7-6.1.3.
>>>
>>> Is this a very demanding test? What does it test for?
>>>
>>> How can I run the tests that come next in "fulltest"?
>>>
>>
>> The next test (misc7-7.0) is testing the sqlite3_busy_timeout()
>> function.  It is suppose to use usleep() (or the equivalent, depending
>> on your OS) to sleep for short intervals and retry the connection
>> until 2.0 seconds have elapsed, then it should give up and report an
>> error.  My first guess would be that usleep() is not working quite
>> right on your system.
>
> I see.
>
>>
>> What OS are you running?  What compiler?
>
> This is on a (Dreamhost) virtual server:
>
> $ uname -a
> Linux ps12345 2.6.22.19-grsec2.1.11-vs2.2.0.7 #10 SMP Tue Aug 4
> 14:56:37 PDT 2009 x86_64 GNU/Linux
> $ cc --version
> gcc (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
>
> Thanks!

Would you have a tip how to continue here?

Stephan

>
>
> Stephan
>
>
>> D. Richard Hipp
>> d...@hwaci.com
>>
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>
>
>
> --
> Stephan Wehner
>
> -> http://stephan.sugarmotor.org (blog and homepage)
> -> http://www.thrackle.org
> -> http://www.buckmaster.ca
> -> http://www.trafficlife.com
> -> http://stephansmap.org -- http://blog.stephansmap.org
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Running test after misc7-6.1.2

2009-09-18 Thread Stephan Wehner
On Fri, Sep 18, 2009 at 2:35 AM, D. Richard Hipp <d...@hwaci.com> wrote:
>
> On Sep 17, 2009, at 10:56 PM, Stephan Wehner wrote:
>
>> I downloaded the sqlite3 source ( sqlite-3.6.18.tar.gz ) and managed
>> to complete "make".
>>
>> Now with  "make fulltest", there is no progress for over  almost two
>> hours. The present output is
>> misc7-6.1.1... Ok
>> misc7-6.1.2... Ok
>>
>> There is still activity with "top" showing memory/CPU usage for the
>> "testfixture" process going up and down.
>> I am guessing the machine is working on misc7-6.1.3.
>>
>> Is this a very demanding test? What does it test for?
>>
>> How can I run the tests that come next in "fulltest"?
>>
>
> The next test (misc7-7.0) is testing the sqlite3_busy_timeout()
> function.  It is suppose to use usleep() (or the equivalent, depending
> on your OS) to sleep for short intervals and retry the connection
> until 2.0 seconds have elapsed, then it should give up and report an
> error.  My first guess would be that usleep() is not working quite
> right on your system.

I see.

>
> What OS are you running?  What compiler?

This is on a (Dreamhost) virtual server:

$ uname -a
Linux ps12345 2.6.22.19-grsec2.1.11-vs2.2.0.7 #10 SMP Tue Aug 4
14:56:37 PDT 2009 x86_64 GNU/Linux
$ cc --version
gcc (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)

Thanks!


Stephan


> D. Richard Hipp
> d...@hwaci.com
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] Running test after misc7-6.1.2

2009-09-17 Thread Stephan Wehner
I downloaded the sqlite3 source ( sqlite-3.6.18.tar.gz ) and managed
to complete "make".

Now with  "make fulltest", there is no progress for over  almost two
hours. The present output is

misc6-1.4... Ok
Memory used:  now 16  max 361240  max-size 10
Page-cache used:  now  0  max  0  max-size   4096
Page-cache overflow:  now  0  max  59136
Scratch memory used:  now  0  max  0
Scratch overflow: now  0  max   9104  max-size   9104
misc7-1-misuse... Ok
misc7-2... Ok
misc7-3... Ok
misc7-4... Ok
misc7-5... Ok
misc7-6.1.1... Ok
misc7-6.1.2... Ok

There is still activity with "top" showing memory/CPU usage for the
"testfixture" process going up and down.
I am guessing the machine is working on misc7-6.1.3.

Is this a very demanding test? What does it test for?

How can I run the tests that come next in "fulltest"?

Stephan



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Multi-master replication through UPDATE and INSERT logging?

2009-09-17 Thread Stephan Wehner
On Thu, Sep 17, 2009 at 9:56 AM, Kelly Jones
<kelly.terry.jo...@gmail.com> wrote:
> On 9/17/09, Simon Slavin <slav...@hearsay.demon.co.uk> wrote:
>>
>> On 17 Sep 2009, at 4:54pm, Kelly Jones wrote:
>>
>>> I want to do multi-master sqlite3 replication by editing sqlite3.c to
>>> log UPDATE/INSERT queries with timestamps, and then using another
>>> program to run those queries on the other masters.

Quick comment; timestamps are not likely to be robust ... I would add
a "version" column and synchronize against that.

When it comes to backups, one likes reliability; I personally wouldn't
be confident with a  recompiled sqlite.
Plus how do you distribute the new binaries to the, presumably up and
running, replication
system without hick-ups? Most likely you are going to change your
implementation (schema changes not mentioned yet)

Stephan

>> Doesn't work.  Consider:  You have a database with three records:
>>
>> ID    object  colour
>> --    --  --
>> 1     car     red
>> 2     coat    black
>> 3     hat     green
>>
>> In database copy number 1 someone does
>>
>> UPDATE objects SET colour='green' WHERE id=1
>>
>> In database copy number 2 someone does
>>
>> UPDATE objects SET colour='purple' WHERE colour='green'
>>
>> Then you try to merge the databases by running everyone else's UPDATE
>> commands against each copy.  Under your system copy 1 will still end
>> up different copy 2.
>>
>> A better way is to hold just one 'master' copy of the database
>> centrally, use everyone's queries to update that, then distribute
>> copies of the master.  But even that won't have the results you might
>> expect at first glance.  First you need to work out what you'd want to
>> happen under those circumstances.  Then work out how to implement it.
>
> Hmmm, good point. I always thought this was how MySQL did replication,
> but maybe that's why MySQL's replication is unidirectional. Two comments:
>
>  % 99+% of the time, there won't be two updates "at the same time". In
>  other words, copy 1's change will almost always propagate to copy 2
>  before copy 2 does another update.
>
>  % In my app, almost all the UPDATEs are of the form "WHERE rowid IN
>  (...)". I don't do global updates like "WHERE foo='bar'".
>
> I read your other post as well, and still think this is worth
> pursuing. Is there a place I can read up on proposed solutions and why
> they fail?
>
> I've considered other solutions (eg, triggers and iwatch/rsync), but
> the query-logging one seems to have certain advantages.
>
> One thought: backup the db before each INSERT/UPDATE (too ugly?) +
> keep the backups for 5m (or whatever). If you get an update query from
> a remote master w/ a timestamp older than your own most-recent update
> query, restore the backup and re-run the queries in the correct order.
>
> I'm trying to implement a redundant system of 4 VPSs, and want to
> avoid having a single master.
>
> --
> We're just a Bunch Of Regular Guys, a collective group that's trying
> to understand and assimilate technology. We feel that resistance to
> new ideas and technology is unwise and ultimately futile.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Stephan Wehner

-> http://stephan.sugarmotor.org (blog and homepage)
-> http://www.thrackle.org
-> http://www.buckmaster.ca
-> http://www.trafficlife.com
-> http://stephansmap.org -- http://blog.stephansmap.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users