Re: [sqlite] importing CSV with a random empty line at the end

[Joe Mistachkin]

Thanks for the report.  Fixed on trunk, here:

https://www.sqlite.org/src/info/fc918f7d33

--
Joe Mistachkin

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] importing CSV with a random empty line at the end

[gwenn]

Hello,
I've taken the time to investigate the problem: only the last
parameter is not correctly bound to NULL.

May I suggest a patch:

--- shell_.c 2014-06-15 14:22:39.0 +0200
+++ shell.c 2014-06-15 14:23:11.0 +0200
@@ -2553,7 +2553,7 @@
   "filling the rest with NULL\n",
   sCsv.zFile, startLine, nCol, i+1);
   i++;
-  while( i wrote:
> gwenn, thanks for this. I did not understand what you saw, and then I
> realized my shell exe was probably too old.
> I downloaded the new exe and this solves the problem just fine!
> thanks for your help.
>
> gert
>
>
> 2014-06-02 19:03 GMT+02:00 gwenn :
>
>> Hello,
>> I doesn't fail for me (it may depend on the constraints on the target
>> table) but the behaviour is unexpected:
>>
>> $ echo "1|test
>> > " > empty.csv
>> $ sqlite3
>> SQLite version 3.8.4.3 2014-04-03 16:53:12
>> sqlite> create table test(opt text, data text not null);
>> sqlite> .import empty.csv test
>> empty.csv:2: expected 2 columns but found 1 - filling the rest with NULL
>> sqlite> select * from test;
>> 1|test
>> |test
>>
>> As the bindings are not cleared, it is not a null value but the
>> previous bound value which is inserted.
>> Regards.
>>
>> On Mon, Jun 2, 2014 at 5:41 PM, Gert Van Assche  wrote:
>> > All,
>> >
>> > I received 100.000 UTF-8 files (average size 50kb)  "ready for import" in
>> > an SQLite db.
>> > 90% of them go fine, but some files have an empty line at the very end of
>> > the fine (so an extra EOL before the EOF).
>> >
>> > Of course, the import fails... Is there an easy way to get rid of that
>> > extra empty line before I import the file, or is there a way to ignore an
>> > empty line?
>> >
>> > thanks
>> >
>> > Gert
>> > ___
>> > sqlite-users mailing list
>> > sqlite-users@sqlite.org
>> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] importing CSV with a random empty line at the end

[Gert Van Assche]

gwenn, thanks for this. I did not understand what you saw, and then I
realized my shell exe was probably too old.
I downloaded the new exe and this solves the problem just fine!
thanks for your help.

gert


2014-06-02 19:03 GMT+02:00 gwenn :

> Hello,
> I doesn't fail for me (it may depend on the constraints on the target
> table) but the behaviour is unexpected:
>
> $ echo "1|test
> > " > empty.csv
> $ sqlite3
> SQLite version 3.8.4.3 2014-04-03 16:53:12
> sqlite> create table test(opt text, data text not null);
> sqlite> .import empty.csv test
> empty.csv:2: expected 2 columns but found 1 - filling the rest with NULL
> sqlite> select * from test;
> 1|test
> |test
>
> As the bindings are not cleared, it is not a null value but the
> previous bound value which is inserted.
> Regards.
>
> On Mon, Jun 2, 2014 at 5:41 PM, Gert Van Assche  wrote:
> > All,
> >
> > I received 100.000 UTF-8 files (average size 50kb)  "ready for import" in
> > an SQLite db.
> > 90% of them go fine, but some files have an empty line at the very end of
> > the fine (so an extra EOL before the EOF).
> >
> > Of course, the import fails... Is there an easy way to get rid of that
> > extra empty line before I import the file, or is there a way to ignore an
> > empty line?
> >
> > thanks
> >
> > Gert
> > ___
> > sqlite-users mailing list
> > sqlite-users@sqlite.org
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] importing CSV with a random empty line at the end

[gwenn]

Hello,
I doesn't fail for me (it may depend on the constraints on the target
table) but the behaviour is unexpected:

$ echo "1|test
> " > empty.csv
$ sqlite3
SQLite version 3.8.4.3 2014-04-03 16:53:12
sqlite> create table test(opt text, data text not null);
sqlite> .import empty.csv test
empty.csv:2: expected 2 columns but found 1 - filling the rest with NULL
sqlite> select * from test;
1|test
|test

As the bindings are not cleared, it is not a null value but the
previous bound value which is inserted.
Regards.

On Mon, Jun 2, 2014 at 5:41 PM, Gert Van Assche  wrote:
> All,
>
> I received 100.000 UTF-8 files (average size 50kb)  "ready for import" in
> an SQLite db.
> 90% of them go fine, but some files have an empty line at the very end of
> the fine (so an extra EOL before the EOF).
>
> Of course, the import fails... Is there an easy way to get rid of that
> extra empty line before I import the file, or is there a way to ignore an
> empty line?
>
> thanks
>
> Gert
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] importing CSV with a random empty line at the end

[Gert Van Assche]

All,

I received 100.000 UTF-8 files (average size 50kb)  "ready for import" in
an SQLite db.
90% of them go fine, but some files have an empty line at the very end of
the fine (so an extra EOL before the EOF).

Of course, the import fails... Is there an easy way to get rid of that
extra empty line before I import the file, or is there a way to ignore an
empty line?

thanks

Gert
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users