Pavel, I really appreciate you hanging with me on this subject. I can sometimes be a bit thick and your patience in dealing with my situation is greatly appreciated.
The data I am using is a very small set of names pulled from an InDesign document text frame. They look like this: John Darnell Patrick O'Reilley William Roberts Patricia Smith So you can see that all the characters in the data are ASCII characters. Nevertheless your point is well-taken that the data could be something other than UTF8, especially since InDesign uses a WideString class internally. I'm pretty sure I have taken the steps that need to be taken to transform the WideString objects to basic_string objects, but that is an assumption that needs to be tested, and I am working on a way to do that right now. Supposedly, basic_string deals with nothing other than UTF8 strings. That has been my assumption until now. I will let you know what I figure out. Soon. R, John > -----Original Message----- > From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] > On Behalf Of Pavel Ivanov > Sent: Friday, December 17, 2010 4:44 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Troubleshooting... > > > So you can see that when I add the hard-coded data, everything looks fine > > in the > results of the select statement, which leads me to believe that the problem > is not > confusion between UTF8 and UTF16 output. > > Your hardcoded data has only characters from ASCII set which don't > differ from the same in UTF-8. Besides SQLite API and command line > utility don't check your strings to be valid UTF-8. SQLite API puts > into database whatever you give to it, command line utility throws to > stdout whatever it finds in the database (so my guess could be wrong > and your input strings could be not in UTF-8 but in some other > encoding). It's totally developer's responsibility to make sure that > encoding put into database is the same as is expected when it's > retrieved from database. > > > I could not figure out how to pipe my info to a file > > One of possible solutions is to use .output command in the sqlite3 > utility (you can use .help command to see everything that is available > to you there) or to provide your sql statement as last argument when > you start sqlite3 utility and use standard redirection like this: > > sqlite3 database.db "select * from mytable" >output.txt > > > Pavel > > On Fri, Dec 17, 2010 at 5:23 PM, john darnell > <john.darn...@walsworth.com> wrote: > > I could not figure out how to pipe my info to a file (I guess I am still a > > very young > (at 57 years) newbie when it comes to SQLite) so I tried something else. In > my > code, I hardcoded the data instead of using variables. Here is a copy of one > of > the statement groups I am using to bind data to the insert statement: > > > > idx = -1; > > idx = sqlite3_bind_parameter_index(ResultStmt, ":disp"); > > sqlite3_bind_text(ResultStmt, idx, CurrentName -> > second.GetDisplayName().c_str(), -1, SQLITE_STATIC); > > //sqlite3_bind_text(ResultStmt, idx, "Smith, John Jacob > > Jingleheimer", -1, > SQLITE_STATIC); > > > > The hard-coded data statement is commented out but left in FYI. > > > > The CurrentName -> second.GetDisplayName().c_str() is one element of an STL > map. The GetDisplayName() function returns a basic_string, and the c_str() is > supposed to return a null-terminated c string style pointer to the data > captured in a > basic_string. > > > > > > Using "SELECT * from NAMES" in SQLite3.exe here is a glimpse of the data > output with a mixture of hard-coded and variable-governed records (52 - 57 are > hard-coded, above are records created using basic_string input): > > > > 52||Smith|John|Jacob|Jingleheimer||Smith, John Jacob Jingleheimer|dont use > anything|1|101|0 > > 53||Smith|John|Jacob|Jingleheimer||Smith, John Jacob Jingleheimer|dont use > anything|1|101|0 > > 54||Smith|John|Jacob|Jingleheimer||Smith, John Jacob Jingleheimer|dont use > anything|1|101|0 > > 55||Smith|John|Jacob|Jingleheimer||Smith, John Jacob Jingleheimer|dont use > anything|1|101|0 > > 56||Smith|John|Jacob|Jingleheimer||Smith, John Jacob Jingleheimer|dont use > anything|1|101|0 > > 57||Smith|John|Jacob|Jingleheimer||Smith, John Jacob Jingleheimer|dont use > anything|1|101|0 > > 58|||||||||0|1|0 > > 59|||||||ε■ε■ε■ε■ε■ε■ε■ε■ε||0|1|0 > > 60|||||||ε■ε■ε■ε■ε■ε■ε■ε■||0|1|0 > > 61|||||||ε■ε■ε■ε■ε■ε■ε■ε■ε■ε||0|1|0 > > > > So you can see that when I add the hard-coded data, everything looks fine > > in the > results of the select statement, which leads me to believe that the problem > is not > confusion between UTF8 and UTF16 output. > > > > Does SQLite3 have problems dealing with basic_strings? > > > > R, > > John > > > >> -----Original Message----- > >> From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- > boun...@sqlite.org] > >> On Behalf Of Pavel Ivanov > >> Sent: Friday, December 17, 2010 9:22 AM > >> To: General Discussion of SQLite Database > >> Subject: Re: [sqlite] Troubleshooting... > >> > >> > I am using SQLite3.exe to query the data with standard select statements > (select > >> * from table). Is there some setting within SQLite3 that I should be > manipulating to > >> provide me UTF8 output? > >> > >> I'm not strong in that but I believe your settings should be not in > >> SQLite3 but in Windows. Try to search the internet for Windows > >> terminal encoding. > >> Meanwhile to check that your database contains what you want try to > >> redirect all output from sqlite3.exe to some text file and then view > >> it in notepad or browser. They can understand UTF-8 encoding pretty > >> well. > >> > >> > >> Pavel > >> > >> On Thu, Dec 16, 2010 at 4:47 PM, john darnell > >> <john.darn...@walsworth.com> wrote: > >> > > >> > > >> >> -----Original Message----- > >> >> From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- > >> boun...@sqlite.org] > >> >> On Behalf Of Pavel Ivanov > >> >> Sent: Thursday, December 16, 2010 2:53 PM > >> >> To: General Discussion of SQLite Database > >> >> Subject: Re: [sqlite] Troubleshooting... > >> >> > >> >> > Once I bind the data to the Insert statement, how can I look at the > >> >> > final > >> statement > >> >> to see what I have done wrong when the statement does not work? > >> >> > >> >> There's no way to do that. You should print what you bind yourself. > >> >> For me it looks like you insert into database some UTF-8 string and > >> >> then try to look at it in the terminal working in some other encoding > >> >> and it thus unable to show your UTF-8 string correctly. > >> >> > >> >> > >> >> Pavel > >> > > >> > > >> > Thank you Pavel. I am using SQLite3.exe to query the data with standard > select > >> statements (select * from table). Is there some setting within SQLite3 > >> that I > should > >> be manipulating to provide me UTF8 output? > >> > > >> > The last four fields lead me to believe that my problem may be up front, > >> > on > input. > >> They accurately reflect the values that I am storing. The only difference > between > >> them and the earlier fields is that they are integer while the rest is > >> text. > >> > > >> > R, > >> > John > >> > > >> >> > >> >> On Thu, Dec 16, 2010 at 3:35 PM, john darnell > >> >> <john.darn...@walsworth.com> wrote: > >> >> > IAW the SQLite book I purchased, I have incorporated data binding > >> >> > into > my > >> >> “INSERT” statements, but neither of the two most important statements > >> >> are > >> >> working…or rather, the first adds a record to the table, but it is > >> >> mostly junk, > >> looking > >> >> like this: > >> >> > > >> >> > > >> >> > 4851||x|x|x||2|3|1|10 > >> >> > > >> >> > >> > 4852||ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■|ε■ε■ε■ε■ε■ε■ε■ε > >> >> ■ε|ε■ε■ε■ε■ε■ε■ε■ε■ε■ε||2|3|1|10 > >> >> > 4853||x|ε■ε■ε■ε■ε■ε■ε■ε■ε|ε■ε■ε■ε■ε■ε■ε■ε■ε■ε||2|3|1|10 > >> >> > > >> >> > >> > 4854||ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■ε■|ε■ε■ε■ε■ε■ε■ε■ε > >> >> ■ε|ε■ε■ε■ε■ε■ε■ε■ε■ε■ε||2|3|1|10 > >> >> > > >> >> > And the second one publishes a self-generating statement that the > >> >> > Insert > >> >> statement did not work. > >> >> > > >> >> > Once I bind the data to the Insert statement, how can I look at the > >> >> > final > >> statement > >> >> to see what I have done wrong when the statement does not work? > >> >> > > >> >> > R, > >> >> > John A.M. Darnell > >> >> > Senior Programmer > >> >> > Walsworth Publishing Company > >> >> > Brookfield, MO > >> >> > John may also be reached at > >> >> johnamdarn...@gmail.com<mailto:johnamdarn...@gmail.com> > >> >> > > >> >> > Trivia SF question: In the movie, THE MATRIX, just before Neo and > Trinity > >> take > >> >> a harrowing ride up an elevator shaft holding on to an elevator cable, > >> >> Neo > >> mutters > >> >> a single phrase. What is that phrase? > >> >> > > >> >> > > >> >> > > >> >> > _______________________________________________ > >> >> > sqlite-users mailing list > >> >> > sqlite-users@sqlite.org > >> >> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > >> >> > > >> >> _______________________________________________ > >> >> sqlite-users mailing list > >> >> sqlite-users@sqlite.org > >> >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > >> > _______________________________________________ > >> > sqlite-users mailing list > >> > sqlite-users@sqlite.org > >> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > >> > > >> _______________________________________________ > >> sqlite-users mailing list > >> sqlite-users@sqlite.org > >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > > > _______________________________________________ > > sqlite-users mailing list > > sqlite-users@sqlite.org > > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
