Re: [sqlite] Lemon parser

[Dan Kennedy]

On Apr 7, 2010, at 3:23 AM, Chris verBurg wrote:

> Hehe, okay, here I go.  :)
>
>
> I'm trying to replace an existing flex/bison parser with an re2c/lemon
> parser, but I'm running into a methodological problem.  I have a
> hypothetical grammar like this:
>
>  file ::= FOO str .
>  file ::= BAR str .

See the use of the "%fallback" directive in the parse.y
file that is part of SQLite sources. I think you want
something like this:

   %fallback str BAR.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite on Windows 2003

[Simon Slavin]

On 6 Apr 2010, at 9:03pm, Burnett, Joe wrote:

> Yes, I was able to open the DB file using the command line tool.
> I am unable to do so programmatically using C# .NET 2008.

Okay, there are some other steps in this test.

Rename that database file or move it somewhere safe.
Use the sqlite3 command-line tool to make a new database, and put just one 
table in it.
See if you can use your program to open this new database.

What I'm trying to do is establish if there's some problem with your required 
database.  Perhaps it's corrupt in some way.

If you get the same fault with this dummy database, the fault is with your code 
or your programming environment.  If the dummy database works fine, the fault 
is with the database you're trying to work with.

Also ...

On 6 Apr 2010, at 8:30pm, Burnett, Joe wrote:

> I have a SQLite application running just fine on Windows XP SP3. I am
> attempting to move the application to Windows 2003
> Enterprise Edition. However, the connection open method returns null.
> The connection string is good. The database file is there.

What value does sqlite3_open return ?

And are you actually using sqlite3_ function calls or are you using a framework 
or library ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite on Windows 2003

[Burnett, Joe]

Hi Teg,

UTF-8, no special characters in the file name WorkData.s3db.

Thanks,

Joe 

Joe Burnett | Principal Software Engineer | FIDELITY INSTITUTIONAL
TECHNOLOGY
2 Contra Way
Merrimack, NH 03054
603.791.5113
cell: 603.289.0481


-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Teg
Sent: Tuesday, April 06, 2010 4:51 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] SQLite on Windows 2003

Hello Joe,

UTF8? What's the filename? If it has any accent characters, you need
to convert the filename to UTF8 before you pass the filename to the
SQLite. At least in C++ you do.

C

Tuesday, April 6, 2010, 4:03:24 PM, you wrote:

BJ> Hi Simon,

BJ> Yes, I was able to open the DB file using the command line tool.
BJ> I am unable to do so programmatically using C# .NET 2008.

BJ> Thanks,

BJ> Joe 

BJ> Joe Burnett | Principal Software Engineer | FIDELITY INSTITUTIONAL
BJ> TECHNOLOGY
BJ> 2 Contra Way
BJ> Merrimack, NH 03054
BJ> 603.791.5113
BJ> cell: 603.289.0481


BJ> -Original Message-
BJ> From: sqlite-users-boun...@sqlite.org
BJ> [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Simon Slavin
BJ> Sent: Tuesday, April 06, 2010 3:33 PM
BJ> To: General Discussion of SQLite Database
BJ> Subject: Re: [sqlite] SQLite on Windows 2003


BJ> On 6 Apr 2010, at 8:30pm, Burnett, Joe wrote:

>> I have a SQLite application running just fine on Windows XP SP3. I am
>> attempting to move the application to Windows 2003
>> Enterprise Edition. However, the connection open method returns null.
>> The connection string is good. The database file is there.
>> I have the appropriate permissions on the file. The event viewer is
no
>> help.

BJ> Are you using the sqlite3_ library calls in a C program you've
written,
BJ> or are you using some sort of 3rd Party framework or library ?

BJ> If you try to open the same database using the sqlite3 command-line
BJ> tool, does it work ?

BJ> Simon.
BJ> ___
BJ> sqlite-users mailing list
BJ> sqlite-users@sqlite.org
BJ> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

BJ> ___
BJ> sqlite-users mailing list
BJ> sqlite-users@sqlite.org
BJ> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



-- 
Best regards,
 Tegmailto:t...@djii.com

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite on Windows 2003

[Teg]

Hello Joe,

UTF8? What's the filename? If it has any accent characters, you need
to convert the filename to UTF8 before you pass the filename to the
SQLite. At least in C++ you do.

C

Tuesday, April 6, 2010, 4:03:24 PM, you wrote:

BJ> Hi Simon,

BJ> Yes, I was able to open the DB file using the command line tool.
BJ> I am unable to do so programmatically using C# .NET 2008.

BJ> Thanks,

BJ> Joe 

BJ> Joe Burnett | Principal Software Engineer | FIDELITY INSTITUTIONAL
BJ> TECHNOLOGY
BJ> 2 Contra Way
BJ> Merrimack, NH 03054
BJ> 603.791.5113
BJ> cell: 603.289.0481


BJ> -Original Message-
BJ> From: sqlite-users-boun...@sqlite.org
BJ> [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Simon Slavin
BJ> Sent: Tuesday, April 06, 2010 3:33 PM
BJ> To: General Discussion of SQLite Database
BJ> Subject: Re: [sqlite] SQLite on Windows 2003


BJ> On 6 Apr 2010, at 8:30pm, Burnett, Joe wrote:

>> I have a SQLite application running just fine on Windows XP SP3. I am
>> attempting to move the application to Windows 2003
>> Enterprise Edition. However, the connection open method returns null.
>> The connection string is good. The database file is there.
>> I have the appropriate permissions on the file. The event viewer is no
>> help.

BJ> Are you using the sqlite3_ library calls in a C program you've written,
BJ> or are you using some sort of 3rd Party framework or library ?

BJ> If you try to open the same database using the sqlite3 command-line
BJ> tool, does it work ?

BJ> Simon.
BJ> ___
BJ> sqlite-users mailing list
BJ> sqlite-users@sqlite.org
BJ> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

BJ> ___
BJ> sqlite-users mailing list
BJ> sqlite-users@sqlite.org
BJ> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



-- 
Best regards,
 Tegmailto:t...@djii.com

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Chris verBurg]

Hehe, okay, here I go.  :)


I'm trying to replace an existing flex/bison parser with an re2c/lemon
parser, but I'm running into a methodological problem.  I have a
hypothetical grammar like this:

  file ::= FOO str .
  file ::= BAR str .

Where my keywords are FOO and BAR, and str is any ol' generic string.

I'm giving it the input "FOO BAR" in the hopes that it will match the first
rule and give str the value "BAR".  To keep Lemon from choking on a syntax
error when it sees BAR, I want to tell the lexer (re2c) to force the next
token after "FOO" to be a string.  Thus I broke the rule up like so:

  file ::= fooDefinition .

  fooDefinition ::= fooStart fooEnd .
  fooStart ::= FOO . { lexer->forceNextThingToBeAString(); }
  fooEnd ::= str .

However, the call to forceNextThingToBeAString() happens after re2c reads
"BAR", and by that time it's already identified it as the keyword BAR
instead of as a string that happens to be set to "BAR".  It seems like Lemon
always reads a lookahead token, even though it doesn't need to.

I considered using the lexer-only hack of having the lexer force a string
after "FOO", but that only works if every FOO in the grammar is followed by
str.  In my real-world problem, that's not the case.  The CFG of the lexer
is not adequate to know when to force strings; it requires control from the
parser.


Would you have any comments/ideas?

Thanks!
-Chris


On Tue, Apr 6, 2010 at 1:12 PM, Wilson, Ronald  wrote:

> > I'm using the Lemon parser and running into a methodological problem
> that
> > I
> > wanted to ask the user base about.  Is there a mailing list or forum
> > specifically for Lemon, or is this it?  :)
> >
> > Thanks,
> > -Chris
>
> There is no mailing list specifically for lemon.  Some of us are
> familiar to various degrees with lemon, so feel free to ask here.
> Though I must say, lemon questions tend languish for a while before
> someone replies.  Don't be afraid to bump your own message if it isn't
> answered.  I think it's safe to say that most sqlite users aren't
> initimately familiar with lemon.
>
> RW
>
> Ron Wilson, Engineering Project Lead
> (o) 434.455.6453, (m) 434.851.1612, www.harris.com
>
> HARRIS CORPORATION   |   RF Communications Division
> assuredcommunications(tm)
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Wilson, Ronald]

> I'm using the Lemon parser and running into a methodological problem
that
> I
> wanted to ask the user base about.  Is there a mailing list or forum
> specifically for Lemon, or is this it?  :)
> 
> Thanks,
> -Chris

There is no mailing list specifically for lemon.  Some of us are
familiar to various degrees with lemon, so feel free to ask here.
Though I must say, lemon questions tend languish for a while before
someone replies.  Don't be afraid to bump your own message if it isn't
answered.  I think it's safe to say that most sqlite users aren't
initimately familiar with lemon.

RW

Ron Wilson, Engineering Project Lead
(o) 434.455.6453, (m) 434.851.1612, www.harris.com

HARRIS CORPORATION   |   RF Communications Division
assuredcommunications(tm)
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] ?bug causing sigsegv

[Pavel Ivanov]

> I did consider this as the issue. However tracing through with gdb the
> sqlite3 "handle" (ie sqlite3*) is the same memory location from Qt calls
> and the returned handle from

Sure thing, it will be the same because you requested pointer from
inside Qt. But it's not a static object which I talked about.

> I also thought that sqlite3_initialize shouldn't need to be called and 
> looking through the Qt code it isn't.

It is called automatically from inside some of SQLite functions
including sqlite3_open*(), but not from sqlite3_exec().

> Also just to be clear I open the sqlite3 database with a Qt call. I then use 
> the returned handle to execute sqlite3 functions directly. This is to extend 
> the Qt sqlite3 implementation.

That's exactly why problem of several instances of static objects can appear.

> So I can't see how I would have two instances. But willing to be corrected!

If you can trace with gdb then enter into sqlite3_exec, then enter
into sqlite3_mutex_enter and then look on the address of
sqlite3GlobalConfig.mutex that is used there (or to the same extent on
the address of just sqlite3GlobalConfig). I guess db.exec() will also
enter into sqlite3_exec so you'll be able to go the same path and look
at the address of sqlite3GlobalConfig.mutex there (if the path is
different then just go through all SQLite functions until you see
sqlite3GlobalConfig somewhere). I bet you'll discover that addresses
of this variable are different inside db.exec() and inside
sqlite3_exec().


Pavel

On Tue, Apr 6, 2010 at 3:51 PM, Neville Dastur  wrote:
> On 05/04/2010 22:04, Pavel Ivanov wrote:
>
> Thank you for reply.
>> Looking at this output from gdb:
>>
>>
>>> #0  0x in ?? ()
>>> #1  0x08058f8e in sqlite3_mutex_enter (p=0x90d98c0) at
>>> 3rdparty/sqlite3/sqlite3.c:14549
>>>
>> I can say that you don't have any mutex module installed (even a no-op
>> mutex module). It means that sqlite3_intialize() wasn't called because
>> mutex system is always initialized in sqlite3_initialize(). But you
>> work with SQLite via Qt API without problems, i.e. Qt called
>> sqlite3_initialize() which seems to be contradictory. But
>> sqlite3_initialize() initializes static structures which means that
>> SQLite API used inside Qt and SQLite API that you use yourself see the
>> same static variables as different memory locations. I suspect this
>> can happen if Qt is some dynamically linked library which has SQLite
>> library statically linked inside. And you link SQLite statically with
>> your application, so you've got 2 completely different instances of
>> SQLite API working with completely different static variables.
>>
> I did consider this as the issue. However tracing through with gdb the
> sqlite3 "handle" (ie sqlite3*) is the same memory location from Qt calls
> and the returned handle from
>
> sqlite3 *handle = *static_cast(v.data());
>
> I also thought that sqlite3_initialize shouldn't need to be called and 
> looking through the Qt code it isn't. Also just to be clear I open the 
> sqlite3 database with a Qt call. I then use the returned handle to execute 
> sqlite3 functions directly. This is to extend the Qt sqlite3 implementation. 
> So I can't see how I would have two instances. But willing to be corrected!
>
>> If everything is indeed how I said then I doubt the code where Qt API
>> and SQLite API are used interchangeably will work properly. I guess
>> you can fix your SIGSEGV by calling sqlite3_initialize() somewhere at
>> the beginning of your application but I think inconsistency in static
>> variables of SQLite can surface in some other errors along the way.
>>
>>
>> Pavel
>>
>> On Sun, Apr 4, 2010 at 4:48 PM, Neville Dastur  wrote:
>>
>>> |Hi I am using Qt to connect to a Sqlite3 database (3.6.23-1). Source is
>>> just included with my app, not a dll / lib.
>>> System: Mac OSX 10.6 and Ubuntu 9.10 (GCC 4.4)
>>> The method of getting a sqlite* is described in the Qt docs and I have
>>> checked the memory addresses in gdb and the returned sqlite3* really is
>>> valid. sqlite.magic also = OPEN.
>>> Other ref:
>>> |http://www.mail-archive.com/sqlite-users@sqlite.org/msg25807.html
>>> |
>>> The issues is I get a SIGSEGV on the sqlite_exec call
>>>
>>>     QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
>>>     db.setDatabaseName("test.db");
>>>     if ( db.open() ) {
>>>
>>>        // If uncommented this works fine
>>>        //db.exec("CREATE TABLE normalMethod1 ( id INT(64), value
>>> CHAR(32) )");
>>>
>>>         // This is taken from the Qt Docs
>>>         QVariant v = db.driver()->handle();
>>>         if (v.isValid()&&  qstrcmp(v.typeName(), "sqlite3*")==0) {
>>>             // v.data() returns a pointer to the handle
>>>             sqlite3 *handle = *static_cast(v.data());
>>>             if (handle != 0) { // check that it is not NULL
>>>
>>>                 qint64 id = sqlite3_last_insert_rowid((sqlite3
>>> *)handle); //<-- This works fine
>>>
>>> 

Re: [sqlite] SQLite on Windows 2003

[Burnett, Joe]

Hi Simon,

Yes, I was able to open the DB file using the command line tool.
I am unable to do so programmatically using C# .NET 2008.

Thanks,

Joe 

Joe Burnett | Principal Software Engineer | FIDELITY INSTITUTIONAL
TECHNOLOGY
2 Contra Way
Merrimack, NH 03054
603.791.5113
cell: 603.289.0481


-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Simon Slavin
Sent: Tuesday, April 06, 2010 3:33 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] SQLite on Windows 2003


On 6 Apr 2010, at 8:30pm, Burnett, Joe wrote:

> I have a SQLite application running just fine on Windows XP SP3. I am
> attempting to move the application to Windows 2003
> Enterprise Edition. However, the connection open method returns null.
> The connection string is good. The database file is there.
> I have the appropriate permissions on the file. The event viewer is no
> help.

Are you using the sqlite3_ library calls in a C program you've written,
or are you using some sort of 3rd Party framework or library ?

If you try to open the same database using the sqlite3 command-line
tool, does it work ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon parser

[Chris verBurg]

Hey guys,

I'm using the Lemon parser and running into a methodological problem that I
wanted to ask the user base about.  Is there a mailing list or forum
specifically for Lemon, or is this it?  :)

Thanks,
-Chris
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] ?bug causing sigsegv

[Neville Dastur]

On 05/04/2010 22:04, Pavel Ivanov wrote:

Thank you for reply.
> Looking at this output from gdb:
>
>
>> #0  0x in ?? ()
>> #1  0x08058f8e in sqlite3_mutex_enter (p=0x90d98c0) at
>> 3rdparty/sqlite3/sqlite3.c:14549
>>  
> I can say that you don't have any mutex module installed (even a no-op
> mutex module). It means that sqlite3_intialize() wasn't called because
> mutex system is always initialized in sqlite3_initialize(). But you
> work with SQLite via Qt API without problems, i.e. Qt called
> sqlite3_initialize() which seems to be contradictory. But
> sqlite3_initialize() initializes static structures which means that
> SQLite API used inside Qt and SQLite API that you use yourself see the
> same static variables as different memory locations. I suspect this
> can happen if Qt is some dynamically linked library which has SQLite
> library statically linked inside. And you link SQLite statically with
> your application, so you've got 2 completely different instances of
> SQLite API working with completely different static variables.
>
I did consider this as the issue. However tracing through with gdb the 
sqlite3 "handle" (ie sqlite3*) is the same memory location from Qt calls 
and the returned handle from

sqlite3 *handle = *static_cast(v.data());

I also thought that sqlite3_initialize shouldn't need to be called and looking 
through the Qt code it isn't. Also just to be clear I open the sqlite3 database 
with a Qt call. I then use the returned handle to execute sqlite3 functions 
directly. This is to extend the Qt sqlite3 implementation. So I can't see how I 
would have two instances. But willing to be corrected!

> If everything is indeed how I said then I doubt the code where Qt API
> and SQLite API are used interchangeably will work properly. I guess
> you can fix your SIGSEGV by calling sqlite3_initialize() somewhere at
> the beginning of your application but I think inconsistency in static
> variables of SQLite can surface in some other errors along the way.
>
>
> Pavel
>
> On Sun, Apr 4, 2010 at 4:48 PM, Neville Dastur  wrote:
>
>> |Hi I am using Qt to connect to a Sqlite3 database (3.6.23-1). Source is
>> just included with my app, not a dll / lib.
>> System: Mac OSX 10.6 and Ubuntu 9.10 (GCC 4.4)
>> The method of getting a sqlite* is described in the Qt docs and I have
>> checked the memory addresses in gdb and the returned sqlite3* really is
>> valid. sqlite.magic also = OPEN.
>> Other ref:
>> |http://www.mail-archive.com/sqlite-users@sqlite.org/msg25807.html
>> |
>> The issues is I get a SIGSEGV on the sqlite_exec call
>>
>> QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
>> db.setDatabaseName("test.db");
>> if ( db.open() ) {
>>
>>// If uncommented this works fine
>>//db.exec("CREATE TABLE normalMethod1 ( id INT(64), value
>> CHAR(32) )");
>>
>> // This is taken from the Qt Docs
>> QVariant v = db.driver()->handle();
>> if (v.isValid()&&  qstrcmp(v.typeName(), "sqlite3*")==0) {
>> // v.data() returns a pointer to the handle
>> sqlite3 *handle = *static_cast(v.data());
>> if (handle != 0) { // check that it is not NULL
>>
>> qint64 id = sqlite3_last_insert_rowid((sqlite3
>> *)handle); //<-- This works fine
>>
>> char* localError=0;
>> const char* sql = "CREATE TABLE tblOne (id int(32), name
>> char(16))";
>> int rc = sqlite3_exec(handle, sql, 0, 0,);
>> //<-- This causes SIGSEGV
>> if (rc != SQLITE_OK) {
>> qFatal("Sqlite3 failed with %i", rc);
>> }
>> }
>> }
>> } |
>>
>> The gdb output and stack backtrace is:
>>
>> Core was generated by `./pcs_debug'.
>> Program terminated with signal 11, Segmentation fault.
>> [New process 31016]
>> [New process 31017]
>> #0  0x in ?? ()
>> (gdb) run ./pcs_debug
>> Starting program: /./pcs_debug ./pcs_debug
>> [Thread debugging using libthread_db enabled]
>> [New Thread 0xb7749970 (LWP 31147)]
>> Xlib:  extension "RANDR" missing on display ":0.0".
>>
>> ** (:31147): CRITICAL **: atk_object_set_name: assertion `name
>> != NULL' failed
>> [New Thread 0xb7516b70 (LWP 31152)]
>> [Thread 0xb7516b70 (LWP 31152) exited]
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 0xb7749970 (LWP 31147)]
>> 0x in ?? ()
>> (gdb) backtrace
>> #0  0x in ?? ()
>> #1  0x08058f8e in sqlite3_mutex_enter (p=0x90d98c0) at
>> 3rdparty/sqlite3/sqlite3.c:14549
>> #2  0x080bc6d5 in sqlite3_exec (db=0x90d3ab0, zSql=0x80f4f34 "CREATE
>> TABLE tblOne (id int(32), name char(16))", xCallback=0, pArg=0x0,
>> pzErrMsg=0xbfda0388) at 3rdparty/sqlite3/sqlite3.c:76821
>> #3  0x0804f732 in main (argc=2, argv=0xbfda04b4) at src/main.cpp:58
>>
>> I have to say that I'm stuck and think this is an issue with threading
>> in sqlite3.
>>
>> Thanks
>>
>> Neville
>>
>> 

Re: [sqlite] SQLite on Windows 2003

[Simon Slavin]

On 6 Apr 2010, at 8:30pm, Burnett, Joe wrote:

> I have a SQLite application running just fine on Windows XP SP3. I am
> attempting to move the application to Windows 2003
> Enterprise Edition. However, the connection open method returns null.
> The connection string is good. The database file is there.
> I have the appropriate permissions on the file. The event viewer is no
> help.

Are you using the sqlite3_ library calls in a C program you've written, or are 
you using some sort of 3rd Party framework or library ?

If you try to open the same database using the sqlite3 command-line tool, does 
it work ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] SQLite on Windows 2003

[Burnett, Joe]

Hi All.,

I have a SQLite application running just fine on Windows XP SP3. I am
attempting to move the application to Windows 2003
Enterprise Edition. However, the connection open method returns null.
The connection string is good. The database file is there.
I have the appropriate permissions on the file. The event viewer is no
help.

Is there something I am missing on Windows 2003?

Thanks and advance,

Joseph Burnett | Principal Software Engineer | FIDELITY INSTITUTIONAL
TECHNOLOGY
2 Contra Way
Merrimack, NH 03054
603.791.5113
CELL: 603.289.0481

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite template files

[Simon Slavin]

On 6 Apr 2010, at 5:50pm, BareFeet wrote:

> If the logic is built into the SQL itself, then the other development 
> language is not an issue. I can open my SQLite database files in a GUI tool 
> written in Objective-C or the command line tool written in C, or a PHP front 
> end to a web page, etc.

In SQLite there's a certain amount of logic which can be carried out or 
enforced by TRIGGERs and FOREIGN KEYs.  I can imagine a competent hacker 
setting up a database with a lot of these and using the resulting database 
without any other programming language support, possibly even from the sqlite3 
command-line tool.  If you add a SQLite GUI database manager to that you might 
even have a usable environment for someone who is fairly techie but not a 
programmer.

So I just spent a couple of minutes imagining what that would be like.  I think 
it still won't result in anything a non-techie can use.  Error messages from 
FOREIGN KEY constraints, for sure, wouldn't be much of an explanation to a 
non-techie user as to what they did wrong.  ("No you can't save this list of 
ingredients because I don't know what 'linguini' is.")

So you'd have to do the whole thing with TRIGGERs, because they can have 
specified error text.  Which is not normal (mostly constraints are imposed by 
the programming around the SQL logic, done in whatever programming language the 
programmer uses).  Which means you'll want schema specially designed for your 
GUI environment.  Which is why you won't be using any existing SQLite template 
files.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite template files

[BareFeet]

> I believe that by 'template files' you mean some example databases for common 
> applications, for instance invoicing, a list of the albums you have, a recipe 
> database, etc..

Yes, that is correct.

> That's because what you seek does not exist.

Well, it does exist if people share their SQLite database files ;-)

> User interfaces are separate from the rdbms back end.


Well, yes, I guess that's true. But an SQLite database file is a standalone 
database file that can be opened by any SQLite browser/management tool, such as 
the sqlite3 command line tool or any of the GUI tools listed here:
http://www.tandb.com.au/sqlite/compare/

It is possible to design, say, a general invoicing database using the sqlite3 
command line tool or a SQLite GUI tool and then open it in a completely 
different GUI tool. If the database designer has imbedded all the necessary 
logic in the SQL itself (such as through triggers and views), then the database 
file is a portable repository for information.

I understand that in many cases, the SQLite database developed are 
intrinsically tied to and designed for the application framework in which they 
reside. However, even those SQLite database files can be opened and manipulated 
by a SQLite GUI tool or the sqlite3 command line tool. In fact, the developer 
of those probably highly specialized database files no doubt created the 
databases using a GUI or command line tool.

> The UI depends on the application (accounting, addressbook, spatial analyses)


In its basic form, the UI is just data viewing and entry into columns in rows. 
A generic GUI tool does that.

> the language used for development (python, ruby, perl, C, Common LISP, object 
> COBOL), and the intended user base.


If the logic is built into the SQL itself, then the other development language 
is not an issue. I can open my SQLite database files in a GUI tool written in 
Objective-C or the command line tool written in C, or a PHP front end to a web 
page, etc.

> You also need to write the middleware that validates data entry, translates 
> between the human-readable UI widgets and the desired dbms action (INSERT, 
> DELETE, UPDATE, SELECT) using embedded SQL (a set-based language, not a 
> procedural one).


That depends on the tool used. At the most basic level data validation should 
be enforced at the SQL level and the GUI tool should show any SQl generated 
errors. At a basic minimum, the GUI tool only has to facilitate basic SQL 
statement execution. If it has some nice GUI elements for update, insert etc, 
all the better.

Thanks for the input,
Tom
BareFeetWare

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Exposing sqlite3BtreeCommitPhaseOne

[Dan Kennedy]

> Currently if an interrupt
> arrives during a long commit I have no way to know whether the commit
> was successful, and thus whether to commit or roll back the filesystem
> journal.

Maybe you can read the database to see if the new data
is there in this scenario.

> If I could flush the dirty pages first, then the actual commit
> is short enough to be effectively instantaneous and there's little
> chance of an interrupt arriving at the same time.

Roughly the same chance of an interrupt arriving during the
COMMIT and the COMMIT still being successful, no?

> Would it be possible to expose this first phase via a C or SQL API?

It's not an unreasonable suggestion, but it seems like the kind
of thing that won't happen to me.

Dan.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3_reset nor sqlite3_clear_bindings are clearing the auxdata when using sqlite3_set_auxdata

[Igor Tandetnik]

Philip Van Hoof  wrote:
> When we use our custom SQLite function function_sparql_regex (lower in
> this E-mail) together with bound values for the argvs of the function,
> then sqlite3_reset nor sqlite3_clear_bindings are clearning the
> auxdata. 
> 
> This makes it impossible to pass the regex as a sqlite3_bind_text (the
> same regex would be used even if you'd pass a new regex string).

You are supposed to handle cache invalidation yourself in the function. Store 
the original regex string together with its compiled form, compare the incoming 
regex with the cached one to see whether you can reuse the cached value or need 
to recompile.

Igor Tandetnik

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Concurrency for in-memory scenarios

[Pavel Ivanov]

BTW, beware of this thread
http://www.mail-archive.com/sqlite-users@sqlite.org/msg50640.html too,
although it wasn't understood what was the actual problem there.


Pavel

On Tue, Apr 6, 2010 at 11:03 AM, Pavel Ivanov  wrote:
>> Can anyone confirm whether concurrent
>> access to an in-memory database is supported?
>
> No, SQLite doesn't support full concurrent access to any database. The
> only concurrency you can earn is having on-disk database without
> shared cache (so actually having several copies of the database in
> memory). Of course I don't consider option of concurrency from
> different processes.
>
>
> Pavel
>
> On Tue, Apr 6, 2010 at 9:43 AM, Kent Boogaart  wrote:
>>  Hi there,
>>
>>  I've recently been struggling with concurrency for an in-memory
>> SQLite scenario. Basically, I want to be able to perform concurrent
>> reads against an in-memory SQLite database, thus using multiple CPUs
>> to good effect.
>>
>>  I've tried everything I could think of and find in various blog posts
>> to get this working, but SQLite insisted on synchronizing the reads.
>> Switching from an in-memory DB to an on-disk DB immediately rectified
>> the issue.
>>
>>  What I'm wondering is whether SQLite flat out doesn't support
>> concurrent access to an in-memory database, or perhaps whether I'm
>> just doing something wrong. Can anyone confirm whether concurrent
>> access to an in-memory database is supported?
>>
>>  Thanks,
>>  Kent
>>
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Concurrency for in-memory scenarios

[Pavel Ivanov]

> Can anyone confirm whether concurrent
> access to an in-memory database is supported?

No, SQLite doesn't support full concurrent access to any database. The
only concurrency you can earn is having on-disk database without
shared cache (so actually having several copies of the database in
memory). Of course I don't consider option of concurrency from
different processes.


Pavel

On Tue, Apr 6, 2010 at 9:43 AM, Kent Boogaart  wrote:
>  Hi there,
>
>  I've recently been struggling with concurrency for an in-memory
> SQLite scenario. Basically, I want to be able to perform concurrent
> reads against an in-memory SQLite database, thus using multiple CPUs
> to good effect.
>
>  I've tried everything I could think of and find in various blog posts
> to get this working, but SQLite insisted on synchronizing the reads.
> Switching from an in-memory DB to an on-disk DB immediately rectified
> the issue.
>
>  What I'm wondering is whether SQLite flat out doesn't support
> concurrent access to an in-memory database, or perhaps whether I'm
> just doing something wrong. Can anyone confirm whether concurrent
> access to an in-memory database is supported?
>
>  Thanks,
>  Kent
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Exposing sqlite3BtreeCommitPhaseOne

[Michael Tharp]

Hello,

I'm looking for a way to force sqlite to do its internal "phase one 
commit", which apparently flushes dirty pages to disk without committing 
the journal. This way I can minimize the time spent in the "phase two 
commit", which is the point at which the journal is unlinked and the 
changes become permanent.

I need this in a package manager to synchronize a sqlite database with a 
journal of changes made to a filesystem. Currently if an interrupt 
arrives during a long commit I have no way to know whether the commit 
was successful, and thus whether to commit or roll back the filesystem 
journal. If I could flush the dirty pages first, then the actual commit 
is short enough to be effectively instantaneous and there's little 
chance of an interrupt arriving at the same time.

Would it be possible to expose this first phase via a C or SQL API?

Thanks!

-- 
Michael Tharp
rPath, Inc.
mth...@rpath.com
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite template files

[Rich Shepard]

On Tue, 6 Apr 2010, BareFeet wrote:

> Thanks to those who replied on this topic, but no-one offered any
> repository of SQLite template files. Does that mean there aren't any
> available?

Tom,

   That's because what you seek does not exist. User interfaces are separate
from the rdbms back end. The UI depends on the application (accounting,
addressbook, spatial analyses), the language used for development (python,
ruby, perl, C, Common LISP, object COBOL), and the intended user base. You
also need to write the middleware that validates data entry, translates
between the human-readable UI widgets and the desired dbms action (INSERT,
DELETE, UPDATE, SELECT) using embedded SQL (a set-based language, not a
procedural one).

   Asking for a SQLite template is analoguos to deciding that you'll use an
International Harvester 6.9l diesel engine and asking for pre-existing
vehicle designs to wrap around it without specifying your intended use.

Rich
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite template files

[Simon Slavin]

On 6 Apr 2010, at 10:49am, BareFeet wrote:

> Thanks to those who replied on this topic, but no-one offered any repository 
> of SQLite template files. Does that mean there aren't any available?

I believe that by 'template files' you mean some example databases for common 
applications, for instance invoicing, a list of the albums you have, a recipe 
database, etc..

The answer is that I've never come across any such collection of schema, nor 
would I expect to.  SQLite itself is just an engine and needs a programming 
language around it to be of any use to someone.  And anyone who can write code 
to call SQLite is probably going to want to design their own schema to suit 
their own particular invoicing structure or music filing system.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] sqlite3_reset nor sqlite3_clear_bindings are clearing the auxdata when using sqlite3_set_auxdata

[Philip Van Hoof]

Hi there,

When we use our custom SQLite function function_sparql_regex (lower in
this E-mail) together with bound values for the argvs of the function,
then sqlite3_reset nor sqlite3_clear_bindings are clearning the auxdata.

This makes it impossible to pass the regex as a sqlite3_bind_text (the
same regex would be used even if you'd pass a new regex string).

The sqlite3_set_auxdata's destroy function ptr is called when you'd
sqlite3_finalize the stmt each time, of course. But then it's not
possible to cache our statements as efficiently as we are doing right
now.

Here's an example that reproduces it:

static sqlite *db = NULL;
static const *filename = "test.db";
static sqlite3_stmt *stmt = NULL;

static void
open_db (void)
{
  sqlite3_open (filename, ) != SQLITE_OK);
  sqlite3_create_function (db, "SparqlRegex", 3, SQLITE_ANY,
   priv, _sparql_regex, 
   NULL, NULL);
}

static void
exec_regex (char *regex, char *mod)
{
  if (!stmt) {
sqlite3_prepare_v2 (db, "SELECT field FROM Table"
"WHERE ... AND ... "
"SparqlRegex (field, ?, ?)",
   -1, , NULL);
  } else {
sqlite3_reset (stmt);
sqlite3_clear_bindings (stmt);
  }
  sqlite3_bind_text (stmt, 0, regex, -1, SQLITE_TRANSIENT);
  sqlite3_bind_text (stmt, 1, mod, -1, SQLITE_TRANSIENT);
  ...
  sqlite3_step () ...
  ...
}

static void
app (void)
{
  open_db ();
  exec_regex (".*", "i");
  exec_regex ("foo", "i");
}

-

static void
function_sparql_regex (sqlite3_context *context,
   int  argc,
   sqlite3_value   *argv[])
{
  gboolean ret;
  const gchar *text, *pattern, *flags;
  GRegexCompileFlags regex_flags;
  GRegex *regex;

  if (argc != 3) {
sqlite3_result_error (context, “Invalid argument count”, -1);
return;
  }

  regex = sqlite3_get_auxdata (context, 1);
  text = sqlite3_value_text (argv[0]);
  flags = sqlite3_value_text (argv[2]);
  if (regex == NULL) {
gchar *err_str;
GError *error = NULL;
pattern = sqlite3_value_text (argv[1]);
regex_flags = 0;
while (*flags) {
  switch (*flags) {
  case ’s’: regex_flags |= G_REGEX_DOTALL; break;
  case ‘m’: regex_flags |= G_REGEX_MULTILINE; break;
  case ‘i’: regex_flags |= G_REGEX_CASELESS; break;
  case ‘x’: regex_flags |= G_REGEX_EXTENDED; break;
  default:
err_str = g_strdup_printf (”Invalid SPARQL regex flag ‘%c’”, *flags);
sqlite3_result_error (context, err_str, -1);
g_free (err_str);
return;
  }
  flags++;
}
regex = g_regex_new (pattern, regex_flags, 0, );
if (error) {
  sqlite3_result_error (context, error->message, error->code);
  g_clear_error ();
  return;
}
sqlite3_set_auxdata (context, 1, regex, (void (*) (void*)) g_regex_unref);
  }
  ret = g_regex_match (regex, text, 0, NULL);
  sqlite3_result_int (context, ret);
  return;
}

-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Concurrency for in-memory scenarios

[Kent Boogaart]

 Hi there, 
 
 I've recently been struggling with concurrency for an in-memory 
SQLite scenario. Basically, I want to be able to perform concurrent 
reads against an in-memory SQLite database, thus using multiple CPUs 
to good effect. 
 
 I've tried everything I could think of and find in various blog posts 
to get this working, but SQLite insisted on synchronizing the reads. 
Switching from an in-memory DB to an on-disk DB immediately rectified 
the issue. 
 
 What I'm wondering is whether SQLite flat out doesn't support 
concurrent access to an in-memory database, or perhaps whether I'm 
just doing something wrong. Can anyone confirm whether concurrent 
access to an in-memory database is supported? 
 
 Thanks, 
 Kent 
  
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite template files

[BareFeet]

Thanks to those who replied on this topic, but no-one offered any repository of 
SQLite template files. Does that mean there aren't any available?

Thanks,
Tom
BareFeet


From: BareFeet 
Date: 27 March 2010 5:58:32 PM AEDT
To: General Discussion of SQLite Database 
Subject: [sqlite] SQLite template files
Hi all,

I'm on the verge of releasing a new GUI app for SQLite. I would like to include 
some SQLite template files other than my own. These templates will act as a 
starting point for the user to enter data and/or modify the schema as needed.

One feature of this software is its ability to facilitate data entry in views 
(ie not just tables). So I am especially interested in SQLite files that make 
use of views, triggers, relationships etc. Some simple, some advanced.

Can you direct me to were I might find some useful templates?

Thanks,
Tom
BareFeet

--
Comparison of SQLite GUI tools:
http://www.tandb.com.au/sqlite/compare/?ml

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Android database corruption

[D. Richard Hipp]

On Apr 6, 2010, at 9:15 AM, cliff 2 wrote:

>
>
> cliff 2 wrote:
>>
>>
>> Thanks very much for the quick response! Unfortunately, the android
>> SQLiteDatabase class deletes the database when it hits this, so it  
>> doesn't
>> give me a chance to do that. However I do have a channel to the  
>> android db
>> developers, so I will forward this onto them, and hopefully they  
>> can do
>> something with it.
>>
>> Thanks again for your help.
>>
>>
>>
>> D. Richard Hipp wrote:
>>>
>>>
>>> ...

 Not sure at this point what kind of work-arounds are available to  
 you.
>>>
>>>
>>> To amplify what Shane said, the corruption is probably just in the
>>> page cache, not in the database file itself.  So if you are able to
>>> close and reopen the database connection when you run into problems,
>>> that might provide a work-around (assuming we are guessing correctly
>>> at the root cause of your problem.)
>>>
>>> D. Richard Hipp
>>> d...@hwaci.com
>>>
>>>
>>>
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users@sqlite.org
>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>>
>>
>>
>
> We changed the db in question to only open in readonly mode when  
> doing a
> select rather than using a writable connection all the time. This has
> greatly reduced the frequency of this error occurring, though it still
> occurs. Given that the closing and opening of the db when this  
> happens isn't
> possible. Is there anything else you can think of that will further  
> reduce
> the frequency of this occurring? Is there an operation that we can  
> avoid for
> example?

Statically link your application against SQLite 3.6.23 instead of  
using the SQLite 3.5.9 that is found on Android.  The bug you are  
hitting was fixed in SQLite 3.6.2.


>
> Even if the android guys can fix their end, there will still be a  
> large
> number of devices deployed with the current version of android and  
> sqlite,
> therefore anything we can do to minimize this would be useful.
> -- 
> View this message in context: 
> http://old.nabble.com/Android-database-corruption-tp28044218p28151650.html
> Sent from the SQLite mailing list archive at Nabble.com.
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

D. Richard Hipp
d...@hwaci.com



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Android database corruption

[cliff 2]

cliff 2 wrote:
> 
> 
> Thanks very much for the quick response! Unfortunately, the android
> SQLiteDatabase class deletes the database when it hits this, so it doesn't
> give me a chance to do that. However I do have a channel to the android db
> developers, so I will forward this onto them, and hopefully they can do
> something with it.
> 
> Thanks again for your help.
> 
> 
> 
> D. Richard Hipp wrote:
>> 
>> 
>> ...
>>>
>>> Not sure at this point what kind of work-arounds are available to you.
>> 
>> 
>> To amplify what Shane said, the corruption is probably just in the  
>> page cache, not in the database file itself.  So if you are able to  
>> close and reopen the database connection when you run into problems,  
>> that might provide a work-around (assuming we are guessing correctly  
>> at the root cause of your problem.)
>> 
>> D. Richard Hipp
>> d...@hwaci.com
>> 
>> 
>> 
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>> 
>> 
> 
> 

We changed the db in question to only open in readonly mode when doing a
select rather than using a writable connection all the time. This has
greatly reduced the frequency of this error occurring, though it still
occurs. Given that the closing and opening of the db when this happens isn't
possible. Is there anything else you can think of that will further reduce
the frequency of this occurring? Is there an operation that we can avoid for
example? 

Even if the android guys can fix their end, there will still be a large
number of devices deployed with the current version of android and sqlite,
therefore anything we can do to minimize this would be useful.
-- 
View this message in context: 
http://old.nabble.com/Android-database-corruption-tp28044218p28151650.html
Sent from the SQLite mailing list archive at Nabble.com.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] paging with limit and offset

[Pavel Ivanov]

You can do this only by issuing 2 different queries - one with
count(*) and another selecting first 10 rows.

Pavel

On Tue, Apr 6, 2010 at 8:39 AM, David Hamm  wrote:
> Hi,
>
> I'd like to use a query like "select * from table limit 0,10" but I also need
> the number of rows that would have been returned without a limit/offset being
> in the query.
>
> I tried select count( *),* from table but that only returns 1 row.
>
> Thanks,
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] paging with limit and offset

[David Hamm]

Hi,

I'd like to use a query like "select * from table limit 0,10" but I also need 
the number of rows that would have been returned without a limit/offset being 
in the query.  

I tried select count( *),* from table but that only returns 1 row.

Thanks,
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Feature request: hash index

[Alexey Pechnikov]

Hello!

The problem is only with virtual tables. With ordinary tables "IN (...)" work 
fine:

sqlite> explain query plan select count(*) from data_content where rowid in (1);
0|0|TABLE data_content USING PRIMARY KEY
CPU Time: user 0.004000 sys 0.00

sqlite> select count(*) from data_content where rowid in (1);
1
CPU Time: user 0.00 sys 0.00
sqlite> 


Best regards, Alexey Pechnikov.
http://pechnikov.tel/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users