> On 04/03/2015 10:16 PM, Artem wrote:
>> Hi!
>>
>> The situation is like that. There?s a SQLite database with around 3 billion
>> records. Each record consists of a certain CHAR field and several other
>> additional fields with different types. The file size is approx. 340 gb. The
>> maximum content length in the doc field is 256 symbols, the content is in
>> Russian.
>>
>> I?m trying to create a full-text index, but it results in a Segmentation
>> Fault error. I?ve been trying to create it in different possible ways, both
>> under Windows (with SQLite Expert and my own .NET software, including one
>> with x64 architecture) and Linux (both Ubuntu and Centos). I?ve even
>> compiled sqlite from the sources, having included necessary flags for FTS3
>> and FTS4, but every time I get one and the same error.
> This does sound like a real problem, but one that might be difficult to
> track down.
> Are you able to get us a stack trace of the crash? Ideally one from a
> build with compiler options "-g -DSQLITE_DEBUG" set.
> Thanks,
> Dan.
Hi, Dan. Now I can't to do this because I haven't Linux on my PC.
But I tried to create FTS table again (now it was another database with
1 350 000 000 rows, smaller than before). And I got the same error
(out of memory) on function:
internal override SQLiteErrorCode Reset(SQLiteStatement stmt)
in file SQLite3.cs
It returns System.Data.SQLite.SQLiteErrorCode.NoMem.
I home it helps.
P.S. It is latest version of SQLite.Net compiled in Visual Studio
2012.
>>
>> I?ve tried two options:
>> - creating a contentless FTS4, when content is stored in a regular table,
>> and FTS-table contains only index (create virtual table docs_fts using
>> fts4(content='docs'... )
>> - creating a full-fledged FTS table from a regular one (insert into docs_fts
>> select doc... from docs;)
>>
>> SQLite is functioning for about 4 hours, after which Segmentation Fault
>> error occurs inevitably.
>> There?re no NULL fields in the database.
>>
>> I?ve worked with 3 different SQLite versions, including the latest one,
>> available on the website. I started trying to create the full-text index of
>> the base shortly after it was created and filled; no other activity, apart
>> from filling the base with data, was conveyed. It has only one docs table,
>> that for sure doesn?t contain any NULL values.
>> I also had an idea that Reindex and Vacuum might have done something wrong,
>> however Reindex happens instantly and Vacuum works rather slowly, but
>> successfully.
>>
>> In short, my actions are:
>>
>> CREATE TABLE docs ([id] INTEGER PRIMARY KEY AUTOINCREMENT, [doc] CHAR...
>> (other fields here)
>> (here?s the process of filling the base with the data, which are downloaded
>> by means of my own script from text files.)
>>
>> CREATE VIRTUAL TABLE docs_fts using fts4 (content='docs', doc... (other
>> fields here)
>> (here?s an attempt of INSERT INTO docs_fts(docs_fts) VALUES ('rebuild') to
>> rebuild contentless FTS index)
>>
>> or
>>
>> CREATE VIRTUAL TABLE docs_fts using fts4 (doc... (other fields here)
>> (here?s an attempt of INSERT INTO docs_fts SELECT doc... from docs;)
>>
>> or
>>
>> CREATE VIRTUAL TABLE docs_fts using fts3 (doc... (other fields here)
>> (here?s an attempt of INSERT INTO docs_fts SELECT doc... from docs;)
>>
>> For each attempt I?ve been making a new copy of the source file, because I
>> suspected that the base could have got broken after Segmentation Fault. I
>> even changed the ram-cards, in case if memory was the problem.
>>
>> But every time I get one and the same result - Segmentation Fault error.
>>
>> So, can you please pay your attention to this problem and fix it ASAP?
>>
>> I can send you a file if you need.
>>
>> Thank you.
>>
>> _______________________________________________
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> _______________________________________________
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
