Hello, I just found this thread and I consider it very interesting.
John's porting of SQLite to z/OS is a very nice option, and as he mentioned before, the modifications to get to it were minor, if any. As far as I understand though, the result is a database which stores numbers as big-endian BUT strings in the local EBCDIC encoding of the hosting system. While this deviates from the statement that defines SQLite textual data as UTF-8 encoded, everything works fine (to the extent we tested it so far) if using a z/OS created DB on the same z/OS.
I believe this results from what is stated in http://www.sqlite.org/version3.html:
"SQLite is not particular about the text it receives and is more than happy to process text strings that are not normalized or even well-formed UTF-8 or UTF-16. Thus, programmers who want to store IS08859 data can do so using the UTF-8 interfaces. As long as no attempts are made to use a UTF-16 collating sequence or SQL function, the byte sequence of the text will not be modified in any way."
Of course the uncompatible text encoding makes the z/OS DB unusable on other platforms and vice-versa.
I personally agree with "k" that encoding text strings in the expected format is user's responsibility, and even on z/OS at least sqlite3_prepare() and sqlite3_bind_text() should pass their parameter in UTF-8 format where needed (I am sure there are other APIs who need the same approach). By the way this is easily doable using the iconv() family of services.
What I think though is that, even properly encoding API parameters in UTF-8 would not be enough to produce a portable SQLite DB.
To get there, in my opinion we should need to fix a set of text strings internally used by SQLite, that in z/OS are EBCDIC encoded, and other platforms expect to be UTF-8. One example is the SQLITE_FILE_HEADER, which John has fixed using a configure 'trick'. Another, I think, affects the sqlite_master table where in addition to the CREATE statement (which should be passed UTF-8 by the caller), there is a type colum, defined as text, which on z/OS is again EBCDIC encoded. Not sure this is the only one, but moving a very simple z/OS created DB on linux, and fixing the sqlite_master issue I was able to use it as-is.
I think that it would be nice to have z/OS creating a structurally healthy DB. This would allow portability for users willing to embrace the EBCDIC<->UTF-8 conversion for user data, without requiring additional effort for users not interested in portability.
While I am not fluent in SQLite internal structure, I am willing to help removing these flaws if directed / supported.
Thank you, mario _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
