Re: [sqlite] sqlite 3.21.0 fts3rank test failure on big-endian linux

2018-01-24 Thread Petr Kubat

On 01/24/2018 03:45 PM, Richard Hipp wrote:

On 1/24/18, Petr Kubat  wrote:

I am hitting a failure in fts3rank test on big-endian architectures
(ppc64, s390x; logs in [1][2]) running Fedora rawhide:

Should now be fixed on trunk.
Thanks! I can verify that the fts3rank test case is now green on both 
ppc64 and s390x. For anyone else interested in the patch, here is the link:


https://www.sqlite.org/src/info/e4766cabdf64d8e9

Big-endian processors are getting harder to get ahold of.  I don't
have one that is still capable of running the TCL tests.  My old
circa-2000 iBook (PPC) can still compile and run the TH3 tests, but I
have not been able to get recent versions of TCL (required for the TCL
tests) to compile on that machine.

What do y'all do at RedHat for legacy testing hardware?  Find stuff on eBay?



Not sure actually since from my point of view, as I do not work 
primarily on ppc, the machines are just there. My guess is we are 
getting some servers from IBM to develop the ppc spins on when needed.

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Make some changes to the source code

2018-01-24 Thread Nick
Yup, I guess I understand you correctly about the mmap problem in the OS. I
have seen some threads about it before.
But I think wal+mmap is still a worthy consideration as db+mmap has already
been supported even though it is disabled by default. At least I think I
could use it in my own application until I find the mmap problem in my
system.

The one thing that bothers me the most is that I have no way to check my
code, as there is a testvfs in sqlite test. So could you please review my
train of thought about my wal+mmap? (Map file when opening it and do not
truncate the file)

By the way, is there a possibly way to submit patch to sqlite? 

Thank you Simon.



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Make some changes to the source code

2018-01-24 Thread Simon Slavin


On 25 Jan 2018, at 1:34am, Nick  wrote:

> Do you mean the corruption problems you mentioned will happen in db+mmap?

When memory-mapping was used for all file access of SQLite, some users reported 
that they occasionally got corrupted databases.  When memory mapping was turned 
off, the problem went away.  There was no error in the source code built into 
SQLite.  The fault was in the operating system’s implementation of memory 
mapping.  That’s all I remember.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Make some changes to the source code

2018-01-24 Thread Nick
I ran tests in my MacOS 10.12.6.



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Make some changes to the source code

2018-01-24 Thread Nick
I use sqlite in my Android application. And I tried to run sqlite test on my
MacOS PC. Some cases failed but I can not figure out it is indeed a
corruption.

Do you mean the corruption problems you mentioned will happen in db+mmap? I
guess it should happen in both wal+mmap and db+mmap if it exists. But I have
not found it until now even though I have heard about the mmap+OS problem
from the community.

And Simon, do you have any idea about the test_vfs problem? And is it OK to
change the code as 1.wal+mmap mentioned besides the mmap problem? 



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] SQLite DELUXE Re: sqlite command line tool NUL support

2018-01-24 Thread Richard Hipp
On 1/24/18, petern  wrote:
> Have you worked out an automated way for your changes to shadow and
> auto-merge from the official trunk?

https://www.sqlite.org/privatebranch.html

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] SQLite DELUXE Re: sqlite command line tool NUL support

2018-01-24 Thread J Decker
On Wed, Jan 24, 2018 at 1:30 PM, petern  wrote:

> Have you worked out an automated way for your changes to shadow and
> auto-merge from the official trunk?  That is, aside from collision edits,
> is it automated?  Longer term, your shadow distribution also needs
> new/merged test cases and a regression test run to re-qualify the merged
> changes at each cycle.
>
>
(yes, git pretty much manages a 'if merged with master compatibility')
Sometimes there may be modifications in master that affect the patch (so
far there may have been one or two)

(against my own fossil->github echo which can get me fossil head without
delay)
https://github.com/d3x0r/sqlite3/compare/escape-inline-nul-strings

then I port to this... making sure tests pass at least as well as expected
(usually)
(against mackyle repo which lags the official repository a little)
https://github.com/mackyle/sqlite/compare/master...d3x0r:escape-shell-inline-nuls

(fossil head composite)
https://github.com/d3x0r/sqlite3/compare/MySqlite
(against mackle; which would work great for a base of an official switch
(if never))
https://github.com/mackyle/sqlite/compare/master...d3x0r:MySqlite


> It seems to me there are many small usability changes which could be
> incorporated into a more comprehensive SQLite shadow release,  call it
> "SQLite DELUXE".  For example, here is another straightforward and useful
> change by Cezary that was never adopted despite having zero impact by
> conditional compilation switch:  http://sqlite.chncc.eu/
>
> The argument against improving generality of the official release because
> it slows down some phones will never go away.  Your changes may have to
> shadow the official release forever.
>
> Peter
>
> On Wed, Jan 24, 2018 at 11:46 AM, J Decker  wrote:
>
> > This is a picture.  This is a tortoise git log view of merges.
> >
> > https://drive.google.com/open?id=1RosGlprJRoutFsou2XDRlflxc8xWoHks
> >
> > On Wed, Jan 24, 2018 at 8:59 AM, J Decker  wrote:
> >
> > >
> > > So for output of a select in the shell ( unlike .dump ) is this.
> > >
> > > for(i=1; i > >   utf8_printf(p->out, ",%s", sqlite3_column_text(pSelect, i));
> > > }
> > >
> > > option 1) add text conversion in the column_text to escape NUL
> > characters.
> > > in TEXT... then everyone everywhere would inherit the change.
> > > 'asdf' is as valid as ''||'a'||'s'||'d'||'f'||''
> > >
> > > ( the postgres/sqlite way is to use  "\'||CHAR(0)||\'" (as a C string)
> if
> > > '\0' is required outside of a bind )
> > >
> > > 2) output literal characters regardless of terminal...
> > > //utf8_printf(p->out, "%s", z);
> > > raw_fwrite( z, 1, sqlite3_column_bytes(pSelect, 0), p->out);
> > >  for(i=1; i > >   //utf8_printf(p->out, ",%s", sqlite3_column_text(pSelect, i));
> > >   raw_fputc(p->out, ',');
> > >   raw_fwrite( sqlite3_column_text(pSelect, i), 1,
> > >   sqlite3_column_bytes(pSelect, 0), p->out);
> > > }
> > >
> > > 3)
> > >
> > > .dump uses
> > > output_quoted_string
> > > output_quoted_escaped_string
> > > (the latter of which will wrap "text\r\n" with
> > > replace(replace("text\\r\\n" ),'\\r',char(13)),'\\n',char(10))
> > > so it could additionally wrap that with
> > >
> > > the latter of which will wrap "u\0tf\r\n" with
> > > replace(replace(replace("u\\0tf\\r\\n" ),'\\r',char(13)),'\\n',char(1
> > > 0)),'\\0',char(0))
> > >
> > > instead of
> > > replace(replace("u'||CHAR(0)||'tf\\r\\n" ),'\\r',char(13)),'\\n',char(
> > 10))
> > >
> > >
> > >
> > >
> > > --
> > > * Change select * output to write full length text value
> > > * Change .dump output to include supporting char(0) substitution ( !
> > Shell
> > > Newlines )
> > > * Change other .dump output to inline subst '||CHAR(0)||'  (
> > > ShellHasFlag(p, SHFLG_Newlines) )
> > >
> > > https://drive.google.com/open?id=1UXaFBRoNypZBhkpfPh7TAwncJMbBT1cH
> > >
> > >
> > > Sample output (shell without newlines)
> > > "d\n and" has a char \x14
> > >
> > > (shell only; doesn't change test1.c)
> > >
> > > SQLite version 3.23.0 2018-01-24 01:58:49
> > > Enter ".help" for usage hints.
> > > sqlite> .dump test
> > > PRAGMA foreign_keys=OFF;
> > > BEGIN TRANSACTION;
> > > CREATE TABLE test (a);
> > > INSERT INTO test VALUES(0);
> > > INSERT INTO test VALUES(replace('TEST\0NUL\0','\0',char(0)));
> > > INSERT INTO test VALUES(replace('TEST\0\0NUL\0','\0',char(0)));
> > > INSERT INTO test VALUES(0);
> > > INSERT INTO test VALUES(0);
> > > INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0
> and
> > > all\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
> > > INSERT INTO test VALUES(replace(replace('char test\rand\n and
> > > all(null)','\r',char(13)),'\n',char(10)));
> > > INSERT INTO test VALUES(replace(replace('char test\rand\n and
> > > all(null)','\r',char(13)),'\n',char(10)));
> > > INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0
> 

[sqlite] SQLite DELUXE Re: sqlite command line tool NUL support

2018-01-24 Thread petern
Have you worked out an automated way for your changes to shadow and
auto-merge from the official trunk?  That is, aside from collision edits,
is it automated?  Longer term, your shadow distribution also needs
new/merged test cases and a regression test run to re-qualify the merged
changes at each cycle.

It seems to me there are many small usability changes which could be
incorporated into a more comprehensive SQLite shadow release,  call it
"SQLite DELUXE".  For example, here is another straightforward and useful
change by Cezary that was never adopted despite having zero impact by
conditional compilation switch:  http://sqlite.chncc.eu/

The argument against improving generality of the official release because
it slows down some phones will never go away.  Your changes may have to
shadow the official release forever.

Peter

On Wed, Jan 24, 2018 at 11:46 AM, J Decker  wrote:

> This is a picture.  This is a tortoise git log view of merges.
>
> https://drive.google.com/open?id=1RosGlprJRoutFsou2XDRlflxc8xWoHks
>
> On Wed, Jan 24, 2018 at 8:59 AM, J Decker  wrote:
>
> >
> > So for output of a select in the shell ( unlike .dump ) is this.
> >
> > for(i=1; i >   utf8_printf(p->out, ",%s", sqlite3_column_text(pSelect, i));
> > }
> >
> > option 1) add text conversion in the column_text to escape NUL
> characters.
> > in TEXT... then everyone everywhere would inherit the change.
> > 'asdf' is as valid as ''||'a'||'s'||'d'||'f'||''
> >
> > ( the postgres/sqlite way is to use  "\'||CHAR(0)||\'" (as a C string) if
> > '\0' is required outside of a bind )
> >
> > 2) output literal characters regardless of terminal...
> > //utf8_printf(p->out, "%s", z);
> > raw_fwrite( z, 1, sqlite3_column_bytes(pSelect, 0), p->out);
> >  for(i=1; i >   //utf8_printf(p->out, ",%s", sqlite3_column_text(pSelect, i));
> >   raw_fputc(p->out, ',');
> >   raw_fwrite( sqlite3_column_text(pSelect, i), 1,
> >   sqlite3_column_bytes(pSelect, 0), p->out);
> > }
> >
> > 3)
> >
> > .dump uses
> > output_quoted_string
> > output_quoted_escaped_string
> > (the latter of which will wrap "text\r\n" with
> > replace(replace("text\\r\\n" ),'\\r',char(13)),'\\n',char(10))
> > so it could additionally wrap that with
> >
> > the latter of which will wrap "u\0tf\r\n" with
> > replace(replace(replace("u\\0tf\\r\\n" ),'\\r',char(13)),'\\n',char(1
> > 0)),'\\0',char(0))
> >
> > instead of
> > replace(replace("u'||CHAR(0)||'tf\\r\\n" ),'\\r',char(13)),'\\n',char(
> 10))
> >
> >
> >
> >
> > --
> > * Change select * output to write full length text value
> > * Change .dump output to include supporting char(0) substitution ( !
> Shell
> > Newlines )
> > * Change other .dump output to inline subst '||CHAR(0)||'  (
> > ShellHasFlag(p, SHFLG_Newlines) )
> >
> > https://drive.google.com/open?id=1UXaFBRoNypZBhkpfPh7TAwncJMbBT1cH
> >
> >
> > Sample output (shell without newlines)
> > "d\n and" has a char \x14
> >
> > (shell only; doesn't change test1.c)
> >
> > SQLite version 3.23.0 2018-01-24 01:58:49
> > Enter ".help" for usage hints.
> > sqlite> .dump test
> > PRAGMA foreign_keys=OFF;
> > BEGIN TRANSACTION;
> > CREATE TABLE test (a);
> > INSERT INTO test VALUES(0);
> > INSERT INTO test VALUES(replace('TEST\0NUL\0','\0',char(0)));
> > INSERT INTO test VALUES(replace('TEST\0\0NUL\0','\0',char(0)));
> > INSERT INTO test VALUES(0);
> > INSERT INTO test VALUES(0);
> > INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and
> > all\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
> > INSERT INTO test VALUES(replace(replace('char test\rand\n and
> > all(null)','\r',char(13)),'\n',char(10)));
> > INSERT INTO test VALUES(replace(replace('char test\rand\n and
> > all(null)','\r',char(13)),'\n',char(10)));
> > INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and
> > all 23\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
> > INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and
> > all merged content\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
> > COMMIT;
> > sqlite> select * from test
> >...> ;
> > 0
> > TEST NUL
> > TEST  NUL
> > 0
> > 0
> > andr test
> >   and all
> > andr test
> > and all
> > andr test
> > and all
> > andr test
> >   and all 23
> > andr test
> >   and all merged content
> > sqlite>
> >
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] .DUMP displays floats differently from SELECT

2018-01-24 Thread Cezary H. Noweta

Hello,

On 2018-01-22 13:00, Iulian Onofrei wrote:


I have a "REAL" column with float values having up to 2 decimals, and using
".dump" with the latest version incorrectly converts them like this:

"0.0" -> "0.0"
"0.05" -> "0.050002775"

> [...]

Did you consider an using of INTEGERs as fixed-point numbers? Nearly the 
same arithmetic with one additional division by 100 after multiplication 
and one additional multiplication by 100 (of dividend) before division.



I rely on dumps to track changes to some databases, so this breaks it
completely, and I currently had to pin sqlite to an older version which
isn't desired.


IMHO, a better and more concise way is to rem out the following line:

p->mode = p->cMode = MODE_Insert;

in ``shell.c'' and to recompile it. Two slashes and the problem has gone.

-- best regards

Cezary H. Noweta
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sqlite command line tool NUL support

2018-01-24 Thread J Decker
This is a picture.  This is a tortoise git log view of merges.

https://drive.google.com/open?id=1RosGlprJRoutFsou2XDRlflxc8xWoHks

On Wed, Jan 24, 2018 at 8:59 AM, J Decker  wrote:

>
> So for output of a select in the shell ( unlike .dump ) is this.
>
> for(i=1; i   utf8_printf(p->out, ",%s", sqlite3_column_text(pSelect, i));
> }
>
> option 1) add text conversion in the column_text to escape NUL characters.
> in TEXT... then everyone everywhere would inherit the change.
> 'asdf' is as valid as ''||'a'||'s'||'d'||'f'||''
>
> ( the postgres/sqlite way is to use  "\'||CHAR(0)||\'" (as a C string) if
> '\0' is required outside of a bind )
>
> 2) output literal characters regardless of terminal...
> //utf8_printf(p->out, "%s", z);
> raw_fwrite( z, 1, sqlite3_column_bytes(pSelect, 0), p->out);
>  for(i=1; i   //utf8_printf(p->out, ",%s", sqlite3_column_text(pSelect, i));
>   raw_fputc(p->out, ',');
>   raw_fwrite( sqlite3_column_text(pSelect, i), 1,
>   sqlite3_column_bytes(pSelect, 0), p->out);
> }
>
> 3)
>
> .dump uses
> output_quoted_string
> output_quoted_escaped_string
> (the latter of which will wrap "text\r\n" with
> replace(replace("text\\r\\n" ),'\\r',char(13)),'\\n',char(10))
> so it could additionally wrap that with
>
> the latter of which will wrap "u\0tf\r\n" with
> replace(replace(replace("u\\0tf\\r\\n" ),'\\r',char(13)),'\\n',char(1
> 0)),'\\0',char(0))
>
> instead of
> replace(replace("u'||CHAR(0)||'tf\\r\\n" ),'\\r',char(13)),'\\n',char(10))
>
>
>
>
> --
> * Change select * output to write full length text value
> * Change .dump output to include supporting char(0) substitution ( ! Shell
> Newlines )
> * Change other .dump output to inline subst '||CHAR(0)||'  (
> ShellHasFlag(p, SHFLG_Newlines) )
>
> https://drive.google.com/open?id=1UXaFBRoNypZBhkpfPh7TAwncJMbBT1cH
>
>
> Sample output (shell without newlines)
> "d\n and" has a char \x14
>
> (shell only; doesn't change test1.c)
>
> SQLite version 3.23.0 2018-01-24 01:58:49
> Enter ".help" for usage hints.
> sqlite> .dump test
> PRAGMA foreign_keys=OFF;
> BEGIN TRANSACTION;
> CREATE TABLE test (a);
> INSERT INTO test VALUES(0);
> INSERT INTO test VALUES(replace('TEST\0NUL\0','\0',char(0)));
> INSERT INTO test VALUES(replace('TEST\0\0NUL\0','\0',char(0)));
> INSERT INTO test VALUES(0);
> INSERT INTO test VALUES(0);
> INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and
> all\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
> INSERT INTO test VALUES(replace(replace('char test\rand\n and
> all(null)','\r',char(13)),'\n',char(10)));
> INSERT INTO test VALUES(replace(replace('char test\rand\n and
> all(null)','\r',char(13)),'\n',char(10)));
> INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and
> all 23\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
> INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and
> all merged content\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
> COMMIT;
> sqlite> select * from test
>...> ;
> 0
> TEST NUL
> TEST  NUL
> 0
> 0
> andr test
>   and all
> andr test
> and all
> andr test
> and all
> andr test
>   and all 23
> andr test
>   and all merged content
> sqlite>
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] sqlite command line tool NUL support

2018-01-24 Thread J Decker
So for output of a select in the shell ( unlike .dump ) is this.

for(i=1; iout, ",%s", sqlite3_column_text(pSelect, i));
}

option 1) add text conversion in the column_text to escape NUL characters.
in TEXT... then everyone everywhere would inherit the change.
'asdf' is as valid as ''||'a'||'s'||'d'||'f'||''

( the postgres/sqlite way is to use  "\'||CHAR(0)||\'" (as a C string) if
'\0' is required outside of a bind )

2) output literal characters regardless of terminal...
//utf8_printf(p->out, "%s", z);
raw_fwrite( z, 1, sqlite3_column_bytes(pSelect, 0), p->out);
 for(i=1; iout, ",%s", sqlite3_column_text(pSelect, i));
  raw_fputc(p->out, ',');
  raw_fwrite( sqlite3_column_text(pSelect, i), 1,
  sqlite3_column_bytes(pSelect, 0), p->out);
}

3)

.dump uses
output_quoted_string
output_quoted_escaped_string
(the latter of which will wrap "text\r\n" with
replace(replace("text\\r\\n" ),'\\r',char(13)),'\\n',char(10))
so it could additionally wrap that with

the latter of which will wrap "u\0tf\r\n" with
replace(replace(replace("u\\0tf\\r\\n" ),'\\r',char(13)),'\\n',char(
10)),'\\0',char(0))

instead of
replace(replace("u'||CHAR(0)||'tf\\r\\n" ),'\\r',char(13)),'\\n',char(10))




--
* Change select * output to write full length text value
* Change .dump output to include supporting char(0) substitution ( ! Shell
Newlines )
* Change other .dump output to inline subst '||CHAR(0)||'  (
ShellHasFlag(p, SHFLG_Newlines) )

https://drive.google.com/open?id=1UXaFBRoNypZBhkpfPh7TAwncJMbBT1cH


Sample output (shell without newlines)
"d\n and" has a char \x14

(shell only; doesn't change test1.c)

SQLite version 3.23.0 2018-01-24 01:58:49
Enter ".help" for usage hints.
sqlite> .dump test
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test (a);
INSERT INTO test VALUES(0);
INSERT INTO test VALUES(replace('TEST\0NUL\0','\0',char(0)));
INSERT INTO test VALUES(replace('TEST\0\0NUL\0','\0',char(0)));
INSERT INTO test VALUES(0);
INSERT INTO test VALUES(0);
INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and
all\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
INSERT INTO test VALUES(replace(replace('char test\rand\n and
all(null)','\r',char(13)),'\n',char(10)));
INSERT INTO test VALUES(replace(replace('char test\rand\n and
all(null)','\r',char(13)),'\n',char(10)));
INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and all
23\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
INSERT INTO test VALUES(replace(replace(replace('char test\rand\n\0 and all
merged content\0','\r',char(13)),'\n',char(10)),'\0',char(0)));
COMMIT;
sqlite> select * from test
   ...> ;
0
TEST NUL
TEST  NUL
0
0
andr test
  and all
andr test
and all
andr test
and all
andr test
  and all 23
andr test
  and all merged content
sqlite>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sqlite3_btreeinfo causes sql logic error

2018-01-24 Thread x
Thanks Richard, working now.

From: Richard Hipp
Sent: 24 January 2018 14:08
To: SQLite mailing list
Subject: Re: [sqlite] sqlite3_btreeinfo causes sql logic error

On 1/24/18, x  wrote:
> Sorry Richard, it was my app that was triggering the exception in response
> to the error. The queries prepare OK but, for any select containing the
> aforementioned non-text cols, sqlite3_step returns 1 and
> (wchar_t*)sqlite3_errmsg16(DB) returns “SQL logic error”.
>
>
>
> I’m using the amalgamation in c++ builder 10.2, 64 bit clang compiler on
> windows 10

Please add -DSQLITE_ENABLE_DBPAGE_VTAB to your build.

--
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sqlite 3.21.0 fts3rank test failure on big-endian linux

2018-01-24 Thread Richard Hipp
On 1/24/18, Petr Kubat  wrote:
>
> I am hitting a failure in fts3rank test on big-endian architectures
> (ppc64, s390x; logs in [1][2]) running Fedora rawhide:

Should now be fixed on trunk.

Big-endian processors are getting harder to get ahold of.  I don't
have one that is still capable of running the TCL tests.  My old
circa-2000 iBook (PPC) can still compile and run the TH3 tests, but I
have not been able to get recent versions of TCL (required for the TCL
tests) to compile on that machine.

What do y'all do at RedHat for legacy testing hardware?  Find stuff on eBay?

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] sqlite 3.21.0 fts3rank test failure on big-endian linux

2018-01-24 Thread Petr Kubat

Hi all,

I am hitting a failure in fts3rank test on big-endian architectures 
(ppc64, s390x; logs in [1][2]) running Fedora rawhide:


Time: fts3query.test 515 ms
! fts3expr5-1.5 expected: [1 {invalid matchinfo blob passed to function rank()}]
! fts3expr5-1.5 got:  [1 {wrong number of arguments to function rank()}]
Time: fts3rank.test 12 ms

This seems to have been introduced in version 3.21.0 (but is still 
present in 3.22.0). Checking the source of the failing test case it 
looks like an endianess issue due to the blob:


do_catchsql_test 1.5 {
  SELECT * FROM t1 ORDER BY rank(x'01000100') DESC, rowid
} {1 {invalid matchinfo blob passed to function rank()}}

Seems similar to a different issue I hit before with fts3conf.test 
(fixed in [3]).


Petr

[1] https://kojipkgs.fedoraproject.org//work/tasks/2189/24412189/build.log
[2] https://kojipkgs.fedoraproject.org//work/tasks/2191/24412191/build.log
[3] https://www.sqlite.org/src/info/87ccdf9cbb928455

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sqlite3_btreeinfo causes sql logic error

2018-01-24 Thread Richard Hipp
On 1/24/18, x  wrote:
> Sorry Richard, it was my app that was triggering the exception in response
> to the error. The queries prepare OK but, for any select containing the
> aforementioned non-text cols, sqlite3_step returns 1 and
> (wchar_t*)sqlite3_errmsg16(DB) returns “SQL logic error”.
>
>
>
> I’m using the amalgamation in c++ builder 10.2, 64 bit clang compiler on
> windows 10

Please add -DSQLITE_ENABLE_DBPAGE_VTAB to your build.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sqlite3_btreeinfo causes sql logic error

2018-01-24 Thread x
Sorry Richard, it was my app that was triggering the exception in response to 
the error. The queries prepare OK but, for any select containing the 
aforementioned non-text cols, sqlite3_step returns 1 and 
(wchar_t*)sqlite3_errmsg16(DB) returns “SQL logic error”.



I’m using the amalgamation in c++ builder 10.2, 64 bit clang compiler on 
windows 10




From: sqlite-users  on behalf of 
Richard Hipp 
Sent: Wednesday, January 24, 2018 1:16:22 PM
To: SQLite mailing list
Subject: Re: [sqlite] sqlite3_btreeinfo causes sql logic error

On 1/24/18, x  wrote:
> Works OK for
>
> SELECT name FROM sqlite_btreeinfo WHERE type='table'
>
> But crashes with
>
> SELECT name, nEntry FROM sqlite_btreeinfo WHERE type='table'

Do it crash, or does it return an error?  There is a difference.

--
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sqlite3_btreeinfo causes sql logic error

2018-01-24 Thread Richard Hipp
On 1/24/18, x  wrote:
> Works OK for
>
> SELECT name FROM sqlite_btreeinfo WHERE type='table'
>
> But crashes with
>
> SELECT name, nEntry FROM sqlite_btreeinfo WHERE type='table'

Do it crash, or does it return an error?  There is a difference.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Make some changes to the source code

2018-01-24 Thread Simon Slavin
On 24 Jan 2018, at 11:35am, Nick  wrote:

> As I found
> there will be a little improvement if I support MMAP to wal file.

Which OS are you using ?  The developers of SQLite have previously had 
corruption problems with memory-mapping under some versions of Windows and 
macOS.  Corruption occurs rarely, and it may appear that your system works fine 
during testing.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sqlite 3.21.0 bug? SELECT CAST ('9223372036854775807 ' AS NUMERIC);

2018-01-24 Thread Petr Kubat

Thanks for the patch! Tests on i686 are green when included.

There are still some failures for other architectures but I will make 
separate threads for these once I investigate.


Petr

On 01/23/2018 06:35 PM, Richard Hipp wrote:

On 1/23/18, Petr Kubat  wrote:

Still present in 3.22.0:

! e_expr-32.2.5 expected: [integer 9223372036854775807]
! e_expr-32.2.5 got:  [real 9.22337203685478e+18]

Is there any more information I can provide to get this looked at?

Ralf provided the information I needed, which was a method of
reproducing the problem.  It is now fixed on trunk.  See
https://www.sqlite.org/src/info/1b02731962c21bb0 for the patch.



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] Make some changes to the source code

2018-01-24 Thread Nick
I am trying to make some changes to the source code of sqlite. As I found
there will be a little improvement if I support MMAP to wal file. I guess
the optimization is micro but it is useful in my test and this is a good way
for me to study the code of sqlite :D

1.wal+mmap
I could use unixMapfile() to map the wal file while MMAP may cause SIGBUS if
the mapped file is truncated. This could happen when reseting the wal file,
in another word, if journal_size_limit is reached or
SQLITE_CHECKPOINT_TRUNCATE is called. But I guess it works if these two APIs
will always not be called in my application.
So, I want to create file holes to get a 4M wal-file in sqlite3WalOpen(),
and always set journal_size_limit to 4M. Then mmap will be supported by
simply calling unixMapfile(4M) in sqlite3WalOpen(). After that, memcpy()
instead of read() will be used when read the first 4M of wal file.
I am wondering if it is all right in my Android applications?

2.Further more. I know mmap is supported when fetching db file:
To map file:
In getPageMMap(), sqlite3OsFetch() MMAPs the whole db file, and return the
mapped page through *pData. Then pagerAcquireMapPage will obtain a page
reference PgHdr based on the pData.
(A small question here, why pData is needed? As xRead() will always use
memcpy instead of read() after unixMapfile(-1) is called.)

sqlite3OsFileControlHint is called to remap the db file when the db grows as
a result of a checkpoint.

To avoid SIGBUS:
Process will catch the CHANGE of other processes by comparing
pWal->hdr.iChange and the corresponding number in wal-index. Whenever a
read, write or checkpoint operation happens, unixUnmapfile() will be called
if there is a CHANGE.

3.Thus another way of wal+mmap:
I want to use pWal->hdr.unused to catch the CHANGE when other process
truncate the wal file(journal_size_limit or SQLITE_CHECKPOINT_TRUNCATE).
Then I will check the hdr.unused to call unixMapfile(-1) before whenever
sqlite3OsRead(pWal->pWalFd) is called.
Is there a better timing to remap the file? Just like
sqlite3WalBeginReadTransaction and walcheckpoint in db+mmap;

I run sqlite test to check my code, but I find pVfs->szOsFile is 16 when
test_vfs.c is called, which means pRet->pWalFd is no longer a unixFile
struct. At this time, sqlite3OsOpen() binds to tvfsOpen() instead of
unixOpen(). So I cannot use unixMapfile() and the test that uses test_vfs.c
will not pass. So could you give me some advices to pass the test?

Thank you.




--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] sqlite3_btreeinfo causes sql logic error

2018-01-24 Thread curmudgeon
Seems to be if you include any non-text column (apart from rootpage) from
sqlite_btreeinfo in the select it causes an sql logic error.



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] sqlite3_btreeinfo causes sql logic error

2018-01-24 Thread x
Works OK for

SELECT name FROM sqlite_btreeinfo WHERE type='table'

But crashes with

SELECT name, nEntry FROM sqlite_btreeinfo WHERE type='table'

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users