Oh, and the implicit question in there is: what should I be doing with the planet database? Should I be using a different version of spgsql? Should I be repairing the database by re-encoding the strings that I find?
Thanks, Robby On Tue, Jul 28, 2009 at 7:36 AM, Robby Findler<[email protected]> wrote: > On Tue, Jul 28, 2009 at 2:45 AM, Ryan Culpepper<[email protected]> wrote: >> Robby Findler wrote: >>> >>> Hi all (Ryan?): I've got a question about pgsql. From what I can tell, >>> string data is stored in the database in the latin-1 encoding >>> (sql-data.ss line 191), but is then retrieved from the database in the >>> utf-8 encoding (io.ss line 205). Am I getting that right? >>> >>> This doesn't mean much, but I changed planet's copy of io.ss to use >>> bytes->string/latin-1 instead of bytes->string/utf-8, and I was able >>> to avoid crashing (but the latin-1 encoding might not have any >>> unencodable octets, so that isn't really saying too too much). >> >> I don't know of any reason for latin-1 to show up in string code... maybe >> for "bytea" code. It might be a mistake from the port to 200, when string >> and bytes split. So the change sounds good. >> >> Were you able to get a crash before? Or data corruption? > > Yes, the current planet database contains the bytes #"Fl\341vio Cruz" > as the name of one of the planet contributors. This is not a valid > utf-8 encoding (and it does appear to be a latin-1 encoded string). > When it is retrieved, I get the following error: > > bytes->string/utf-8: string is not a well-formed UTF-8 encoding: > #"Fl\341vio Cruz" > > === context === > parse:DataRow > /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/p3.ss:87:5 > /local/svn/plt/collects/scheme/promise.ss:94:4: loop1 > /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:71:6: > recv method in base% > /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:469:6: > query1:data-loop method in ...rivate/connection.ss:371:4 > /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:402:10: > loop > /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:850:6: > -map method in ...rivate/connection.ss:759:4 > get-all-users > /local/svn/plt/collects/scheme/private/misc.ss:68:7 > >> If you send me some code to produce it, I'll add it to the test suite. > > Unfortunately, I only have a database that has what appears to be bad > data in it; I don't know how to make it happen with an empty database, > but I guess that if you could get the line I listed to run, you'd make > it happen. > >> Since the line numbers >> you list don't correspond to recent spgsql releases, it's possible this bug >> has already been fixed and planet is just using an old version. > > Oh, I was using version (2 3) of spgsql, sorry. > > Robby > _________________________________________________ For list-related administrative tasks: http://list.cs.brown.edu/mailman/listinfo/plt-dev
