[sqlite] Creating an index seems to break table logic

2015-05-20 Thread Adam Podstawczyński
Thank you for responding Simon, Please see my last response to this thread where I copied more data from the console. I have to obfuscate the data somewhat unfortunately before public posting? but I preserved all structure and types as in original data. Thanks, -- adam > On 19 May 2015, at

[sqlite] Creating an index seems to break table logic

2015-05-19 Thread Adam Podstawczyński
Hi Dan, Thank you for responding. The integrity check result is OK: sqlite> pragma integrity_check; integrity_check -- ok Here is the session copied. I had to obfuscate some values though? I don?t think this should prevent investigating this.

[sqlite] Creating an index seems to break table logic

2015-05-19 Thread Adam Podstawczyński
Hi all, I have this schema: CREATE TABLE list_of_numbers ( nn TEXT, astart INT, aend INT, alength INT, usesflag TEXT, blength INT, coolflag NUM, alphaid ); Some example data: nn astart aendalength usesflag blength coolflag

[sqlite] Characters corrupt after importing a CSV file

2015-02-26 Thread Adam Podstawczyński
Hi Ryan, I got it from here: http://jonahellison.com/21640-translated-country-names-unicode-csv Also, to provide more input, I have now noticed that even if the column width is wider than the offending string, this issue still creates problems ? while nothing gets truncated, the position of

[sqlite] Characters corrupt after importing a CSV file

2015-02-26 Thread Adam Podstawczyński
Hi Simon, Yes, it seems so. Because column width is based on bytes, it may happen that e.g. a 2-byte character is split in half which produces a non-recognizable or control character (represented in the terminal as ???). The solution would be to check the column width based on text length but

[sqlite] Characters corrupt after importing a CSV file

2015-02-26 Thread Adam Podstawczyński
Thank you Ryan. I will provide the files if still necessary, but I have just discovered it has to do with column width. Please compare: .width 5 5 20 select * from countrynameslocalized where iso="AE"; AE gswVer?inigti Arabisch AE gu ??? AE he ? ? AE hi

[sqlite] Characters corrupt after importing a CSV file

2015-02-26 Thread Adam Podstawczyński
Hi all, Thanks for trying to help. Answering all suggestions and questions: 1. I don?t think this is a presentation issue: I exported the data which I had imported earlier, and the issue is there in the exported files. 2. Version information: Last login: Thu Feb 26 09:06:41 on ttys004 Hg:locs

[sqlite] Characters corrupt after importing a CSV file

2015-02-26 Thread Adam Podstawczyński
Hi all, I experienced an issue with character encoding in sqlite3 following an import from a CSV file. The issue is described here: http://stackoverflow.com/questions/28719413/sqlite3-corrupts-some-chars-after-importing?noredirect=1#comment45732717_28719413 In short: after importing a UTF-8