Re: [sqlite] Bug report

2019-11-21 Thread Simon Slavin
CVE will not record this bug if it doesn't affect a /released/ version of any product. One hopes that none of the products which incorporate SQLite would incorporate a version of SQLite which never received a release number. In other words, the reporters told the developer team before the bug

Re: [sqlite] Bug report

2019-11-21 Thread Kees Nuyt
On Thu, 21 Nov 2019 21:02:57 +, Jose Isaias Cabrera wrote: >Kees Nuyt, on Thursday, November 21, 2019 03:48 PM, wrote... [...] >> >> I see no CVE entered by the OP, but maybe I missed something. > > Yes, you are right. After pasting it, I went through the top 5 > and none of these

Re: [sqlite] Request: Combining skip-scan with 'max' optimization

2019-11-21 Thread Andy Bennett
Hi, I hadn't seen this thread when I posted my recent thread on optimising MAX aggregates but I suspect this could help my case as well. At the moment I'm trying to limit the amount of data that the aggregate query has to visit in order to keep latency low but this optimisation would give

Re: [sqlite] What is the C language standard to which sqlite conforms ?

2019-11-21 Thread Dennis Clarke
On 11/21/19 5:15 PM, Dan Kennedy wrote: On 22/11/62 00:06, Jens Alfke wrote: On Nov 21, 2019, at 7:01 AM, Richard Hipp wrote: The memset() just forces the bug to the surface in builds where the ckmalloc()/ckfree() routines of TCL are using caching that prevents valgrind/ASAN from seeing

[sqlite] Request: Combining skip-scan with 'max' optimization

2019-11-21 Thread Jens Alfke
I'm following up on my "Optimizing `SELECT a, max(b) GROUP BY a`" thread from a few weeks ago, rephrasing it as a clearer enhancement request. ACTUAL BEHAVIOR: A query of the form `SELECT a, max(b) GROUP BY a` runs slowly (O(n) with the number of table rows), even if there is an index on (a, b

Re: [sqlite] Bug report

2019-11-21 Thread Jose Isaias Cabrera
Kees Nuyt, on Thursday, November 21, 2019 03:48 PM, wrote... > > > Thanks, Jose. > > I see no CVE entered by the OP, but maybe I missed something. Yes, you are right. After pasting it, I went through the top 5 and none of these aren't/weren't the one. Apologies. I thought that by searching on

Re: [sqlite] Mixed ASC and DESC in single column

2019-11-21 Thread Hamish Allan
Wizardry! This is just what I was trying to conceive of. Thank you very much indeed. Best wishes, Hamish On Thu, 21 Nov 2019 at 17:17, Keith Medcalf wrote: > > with a (identifier, prefix, suffix) as ( > select identifier, >substr(info, 1, instr(info, '/') - 1), >substr(info,

Re: [sqlite] Bug report

2019-11-21 Thread Kees Nuyt
Thanks, Jose. I see no CVE entered by the OP, but maybe I missed something. A quick look to your list : > NameDescription > CVE-2019-9937 > In SQLite 3.27.2, interleaving reads and writes in a single transaction with > an fts5

[sqlite] SQLITE_USE_ALLOCA compile time option on FreeBSD

2019-11-21 Thread tom-sqlite
Hi, Looking through the recommended compile-time options at: https://sqlite.org/compile.html It recommends using SQLITE_USE_ALLOCA on systems that support it. On macOS and FreeBSD, the man page for alloca says something like this: BUGS The alloca() function is machine and compiler

Re: [sqlite] Bug report

2019-11-21 Thread Richard Hipp
On 11/19/19, Yongheng Chen wrote: > Hi, > > This is Yongheng Chen from Gatech and Rui Zhong from PSU. We found 7 crashes > for sqlite of the newest commit 3842e8f166e23a1ed6e6094105e7a23502d414da. > We have attached the samples that crash sqlite in the email. FYI, we have > also reported the

Re: [sqlite] Mixed ASC and DESC in single column

2019-11-21 Thread Keith Medcalf
with a (identifier, prefix, suffix) as ( select identifier, substr(info, 1, instr(info, '/') - 1), substr(info, instr(info, '/') + 1) from data where instr(info, '/') > 1 ), b (identifier, bar, foo, baz) as ( select identifier, max(case prefix when 'bar' then suffix end),

Re: [sqlite] What is the C language standard to which sqlite conforms ?

2019-11-21 Thread Dan Kennedy
On 22/11/62 00:06, Jens Alfke wrote: On Nov 21, 2019, at 7:01 AM, Richard Hipp wrote: The memset() just forces the bug to the surface in builds where the ckmalloc()/ckfree() routines of TCL are using caching that prevents valgrind/ASAN from seeing the use-after-free. The memset() is not

Re: [sqlite] What is the C language standard to which sqlite conforms ?

2019-11-21 Thread Jens Alfke
> On Nov 21, 2019, at 7:01 AM, Richard Hipp wrote: > > The memset() just forces the bug to the surface in builds where the > ckmalloc()/ckfree() routines of TCL are using caching that prevents > valgrind/ASAN from seeing the use-after-free. The memset() is not > part of the bug fixx itself,

Re: [sqlite] Mixed ASC and DESC in single column

2019-11-21 Thread Jim Morris
On 11/21/2019 8:08 AM, Hamish Allan wrote: > Thank you very much Clemens, but I've realised I've asked the wrong > question. > > Is it possible to achieve the same if the identifiers are not unique? > > So for instance: > > CREATE TABLE IF NOT EXISTS Data (identifier TEXT, info TEXT); > > INSERT

Re: [sqlite] Bug report

2019-11-21 Thread Jose Isaias Cabrera
NameDescription CVE-2019-9937 In SQLite 3.27.2, interleaving reads and writes in a single transaction with an fts5 virtual table will lead to a NULL Pointer Dereference in fts5ChunkIterate in sqlite3.c. This is related to

Re: [sqlite] Mixed ASC and DESC in single column

2019-11-21 Thread Hamish Allan
Thank you very much Clemens, but I've realised I've asked the wrong question. Is it possible to achieve the same if the identifiers are not unique? So for instance: CREATE TABLE IF NOT EXISTS Data (identifier TEXT, info TEXT); INSERT INTO Data VALUES ("id1", "foo/2"); INSERT INTO Data VALUES

Re: [sqlite] What is the C language standard to which sqlite conforms ?

2019-11-21 Thread Richard Hipp
On 11/21/19, Warren Young wrote: > On Nov 21, 2019, at 3:54 AM, Richard Hipp wrote: >> >> The solution is here: https://www.sqlite.org/src/info/0d1055a5da8274a5 > > memset before free? Why does that help? The memset() just forces the bug to the surface in builds where the ckmalloc()/ckfree()

Re: [sqlite] What is the C language standard to which sqlite conforms ?

2019-11-21 Thread Warren Young
On Nov 21, 2019, at 3:54 AM, Richard Hipp wrote: > > The solution is here: https://www.sqlite.org/src/info/0d1055a5da8274a5 memset before free? Why does that help? ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Bug report

2019-11-21 Thread Kees Nuyt
On Tue, 19 Nov 2019 00:19:13 -0500, you wrote: > Hi, > > This is Yongheng Chen from Gatech and Rui Zhong from PSU. > We found 7 crashes for sqlite of the newest commit > 3842e8f166e23a1ed6e6094105e7a23502d414da. > We have attached the samples that crash sqlite in the email. The mailing list

Re: [sqlite] Mixed ASC and DESC in single column

2019-11-21 Thread Clemens Ladisch
Hamish Allan wrote: > I want to get the uuids in order as if `foo`, `bar` and `bar` were > different columns, e.g. if the desired order were "bar ASC, foo DESC, baz > ASC" SELECT * FROM (SELECT uuid FROM Data WHERE info LIKE 'bar/%' ORDER BY info ASC) UNION ALL SELECT * FROM (SELECT uuid FROM

Re: [sqlite] Optimising query with aggregate in subselect.

2019-11-21 Thread Andy Bennett
Hi, Thanks to everyone who helped with this! I'll try some stuff out and see if I can get things efficient, fast *and* simple. :-) "There's a small sidenote (that I'm too lazy too find right now) in the select docs that mentions that, in case of using min or max as aggregate, the

[sqlite] 回复: stack-overflow issue in fts4 module

2019-11-21 Thread 林性伟(林以)
Hi, Sorry to make you inconvenient. poc, test.sql: CREATE VIRTUAL TABLE t0 USING fts4(content=t0,0); SELECT count() FROM t0(0); Full asan bt: $ ./sqlite3 SQLite version 3.31.0 2019-11-19 21:22:16 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to

Re: [sqlite] [EXTERNAL] Symlink to update a database table

2019-11-21 Thread Hick Gunter
What is the use case? The statement you give will set the value of the "column" field of table "table" to the whole contents of file.txt in each and every row that matches SQLite does not have a symbolic link type. You can store the text of a symbolic link, but accessing the contents would

Re: [sqlite] [EXTERNAL] [Importing CSV] Empty colums != NULL?

2019-11-21 Thread Hick Gunter
CSV is unable to represent the NULL value. The best it can do is "empty string", which gets converted to - drumroll - an empty string (or the value 0 for numeric fields). The field names you are using suggest that you may have not sufficiently normalized your data, since you have duplicated

[sqlite] Symlink to update a database table

2019-11-21 Thread sofiestoc
Hello, Complete newbie here. I have a text file that contains contigs from an assembly. I was wondering if there is a way to symlink the file to table in sqlite by using bash command. I came across the function readfile but I would prefer to create a symbolic link to the pathway instead of

Re: [sqlite] [Importing CSV] Empty colums != NULL?

2019-11-21 Thread Winfried
For other newbies' benefit: Yes, indeed, importing empty columns doesn't set them to NULL. UPDATE Shops SET "contact:postcode" = NULL WHERE "contact:postcode" = ""; UPDATE Shops SET "addr:postcode" = NULL WHERE "addr:postcode" = ""; SELECT name, IFNULL("contact:postcode","addr:postcode") zipcode

Re: [sqlite] stack-overflow issue in fts4 module

2019-11-21 Thread Richard Hipp
On 11/20/19, 林性伟(林以) wrote: > Hi all, > > I found a stack overflow issue in fts4 module, which is in > `sqlite-snapshot-201911192122.tar.gz` version. Thank you for the bug report. However, your report is not helpful in finding and fixing the problem. If possible, please send the following

[sqlite] [Importing CSV] Empty colums != NULL?

2019-11-21 Thread Winfried
Hello, I imported data from a CSV file. Some of the columns are empty, so I expected them to be set to NULL, but it looks like it's not how SQLite handles them internally: = sqlite> .schema CREATE TABLE Shops( "id" TEXT, "name" TEXT, "contact:postcode" TEXT, "contact:city" TEXT,

[sqlite] Mixed ASC and DESC in single column

2019-11-21 Thread Hamish Allan
Hi, Given a table like the following: CREATE TABLE IF NOT EXISTS Data (uuid TEXT, info TEXT); with data like: C94464EB|foo/x1 8A740A4C|foo/x2 FBC49814|bar/x1 F9B0921F|bar/y1 1914F587|bar/y2 E51EC596|baz/a1 549298B6|baz/a2 822DC1A8|foo/x2/a1 46F2854F|foo/x1/a1 47FE9DB1|bar/z1 841716A3|bar/y1/b1

[sqlite] stack-overflow issue in fts4 module

2019-11-21 Thread 林性伟(林以)
Hi all, I found a stack overflow issue in fts4 module, which is in `sqlite-snapshot-201911192122.tar.gz` version. bt: Program received signal SIGSEGV, Segmentation fault. 0x76e5130e in _int_malloc (av=av@entry=0x771a8c40 , bytes=bytes@entry=1032) at malloc.c:3557 3557in

[sqlite] Bug report

2019-11-21 Thread Yongheng Chen
Hi, This is Yongheng Chen from Gatech and Rui Zhong from PSU. We found 7 crashes for sqlite of the newest commit 3842e8f166e23a1ed6e6094105e7a23502d414da. We have attached the samples that crash sqlite in the email. FYI, we have also reported the bugs for CVE at cve.mitre.org

Re: [sqlite] What is the C language standard to which sqlite conforms ?

2019-11-21 Thread Richard Hipp
The solution is here: https://www.sqlite.org/src/info/0d1055a5da8274a5 -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users