> And its not just excel....I use OpenOffice Calc...and have the same > issues......CreatefromCSV works, but ImportTiddlers doesn´t...tried > exporting from Calc in various ways....
Ok.. I fixed several problems with CSV parsing: 1) Some spreadsheets are writing CSV text files with "\r\n" line endings (Windows format) while others are written with just "\n" line endings (Unix format). The "\r" isn't really needed, so I added code to ignore them. One problem fixed. 2) If a CSV value contains commas or quotes, then the whole value is enclosed in quotes, and any quotes it contains are doubled-up. For example: This is "quoted", with a comma becomes "This is ""quoted"", with a comma" The first row of the CSV contains the tiddler field names. By definition, these names can only be lower-case words that don't contain any commas or quotes, so they shouldn't have any quotes around them. However, it seems that the CSV written by OOCalc is putting quotes around *all* values, even if they don't need them. As a result, while the first row of the CSV data contained a field named: "title" (with the quotes), it didn't contain a field named: title (without the quotes), which is *required* in order to handle the row data as tiddler information. The simple fix was to strip off the surrounding quotes (if any) when reading the field names. 3) Related to the handling above, there was also an error in the way that the doubled-up encoded quotes were being converted back to just plain double quotes. As a result, any embedded quotes were being omitted from the text written into the tiddler. I had to do some re-writing of the parsing logic, but I think I've finally got it correct... at least it seems to be working properly now: the text in the tiddler preserves embedded quotes. Third bug... *squish*. 4) In OOCalc, depending upon your system/application character encoding settings, when you type double-quotes around a word in a cell it seems that, instead of the plain-old double-quote character, ASCII \x22, it inserts the matching left- and right- quotation marks, which use full 8-bit 'upper' ASCII character codes, \x93 and \x94. This *shouldn't* be a problem, as the upper ASCII characters should just be treated as tiddler text anyway. However, something about those fancy quote characters seems to be causing a fatal error when processing the file. A brief debugging session showed the problem occurs somewhere deep in the core's convertUTF8ToUnicode() handling, but I haven't figured out what is going wrong, much less how to fix it... but I'm working on it... Fortunately, I've found a workaround (in OOCalc, at least): when you save the CSV file, make sure to select the "[x] filter options" checkbox so that you will be asked about the kind of encoding you want to use for the CSV file. The default is: Windows-1252/WinLatin1 Change this to: ISO-8859-1 The resulting file will use the normal ASCII \x22 double-quote instead of the uppe ASCII quotes, and also seems to fix the file encoding so that TW doesn't throw an error when reading it. Except for this "encoding" issue, which has a temporary workaround, I think I've fixed the rest of the CSV parsing problems. Get the updates here: http://www.TiddlyTools.com/#ImportTiddlersPlugin (v4.5.5) http://www.TiddlyTools.com/#LoadTiddlersPlugin (v.3.7.7) http://www.TiddlyTools.com/#CreateTiddlersFromCSV (v1.3.0) enjoy, -e --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "TiddlyWiki" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/tiddlywiki?hl=en -~----------~----~----~----~------~----~------~--~---
