Re: [sqlite] Need a wince test
On 12/21/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Uwe Sander <[EMAIL PROTECTED]> wrote: > Hi, > > Am Donnerstag, 21. Dezember 2006 04:03 schrieb [EMAIL PROTECTED]: > > My compiler does not define CP_OEM. Can somebody please tell me > > what the literal value of that macro is? > > I think, the name of the constant is CP_OEMCP, Yes. I figured that out about the same time you sent your email. CP_OEMCP is defined by my compiler. That fixed the problem. Thanks. I don't think Windows CE has any notion of CP_OEMCP. The OEM codepage was the old MS-DOS codepage (the one defined on config.sys) and exists on Windows only for compatibility with old code (and command line applications). A quick browsing over the MSDN site didn't made me sure that it really doesn't exist, but I didn't find any documentation for AreFileApisANSI() on Windows CE. Unfortunately can't test it because I don't have Windows anymore (and the gnu-wince toolchain is still too recent to be an authoritative source). Best regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Need a wince test
Uwe Sander <[EMAIL PROTECTED]> wrote: > Hi, > > Am Donnerstag, 21. Dezember 2006 04:03 schrieb [EMAIL PROTECTED]: > > My compiler does not define CP_OEM. Can somebody please tell me > > what the literal value of that macro is? > > I think, the name of the constant is CP_OEMCP, Yes. I figured that out about the same time you sent your email. CP_OEMCP is defined by my compiler. That fixed the problem. Thanks. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Need a wince test
Hi, Am Donnerstag, 21. Dezember 2006 04:03 schrieb [EMAIL PROTECTED]: > My compiler does not define CP_OEM. Can somebody please tell me > what the literal value of that macro is? I think, the name of the constant is CP_OEMCP, its value is 1. Don't have a CP_OEM on my disk. - Uwe - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Need a wince test
Brodie Thiesfield <[EMAIL PROTECTED]> wrote: > > To be safe, the unicodeToMbcs needs to determine which codepage to > convert to in the WideCharToMultiByte calls. This is done by calling the > AreFileApisANSI() http://snipurl.com/arefileapisansi like: > UINT codepage = AreFileApisANSI() ? CP_ACP : CP_OEM; My compiler does not define CP_OEM. Can somebody please tell me what the literal value of that macro is? -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Need a wince test
The utf8ToUnicode function uses MultiByteToWideChar(CP_UTF8) which it seems from recent comments isn't supported on all versions of Windows CE. This may need to be changed to use the internal UTF-8 to UTF-16 conversion routines. To be safe, the unicodeToMbcs needs to determine which codepage to convert to in the WideCharToMultiByte calls. This is done by calling the AreFileApisANSI() http://snipurl.com/arefileapisansi like: UINT codepage = AreFileApisANSI() ? CP_ACP : CP_OEM; Note also that many comments are wrong throughout the file. All WCHAR in Windows is UTF-16 not UTF-32 which is stated everywhere. You need casts for the void* to either char* or WCHAR* in the LoadLibraryA/W calls. Regards, Brodie [EMAIL PROTECTED] wrote: > win95/nt/ce users, please test check-in [3541] to see if extension > loading now works on wince and to make sure that nothing broke on > win95/nt. Tnx. > > http://www.sqlite.org/cvstrac/chngview?cn=3541 > > -- > D. Richard Hipp <[EMAIL PROTECTED]> > > > - > To unsubscribe, send email to [EMAIL PROTECTED] > - > > - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Need a wince test
win95/nt/ce users, please test check-in [3541] to see if extension loading now works on wince and to make sure that nothing broke on win95/nt. Tnx. http://www.sqlite.org/cvstrac/chngview?cn=3541 -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Need a wince test
Robert Simpson wrote:
CP_UTF8 doesn't work on most CE platforms and hence your proposed patch
doesn't work.
Then neither did drh's. Which then only leaves the option of
implementing it in os_win.c which I have been wanting to do all along.
> Robert wrote:
>> Brodie wrote:
>>> Robert wrote:
Here's what I propose ...
> #ifdef _UNICODE
> #define OSSTR wchar_t
> #else
> #define OSSTR char
>
> OSSTR *utf8toOS(char *utf8)
> void utf8toOSFree(OSSTR *apiString)
Although it is nice since it is simple, it is worse than the current
solution. Firstly, with the current method, only Windows 95 is broken.
Additionally, the current solution has the nice property of using the
unicode functions whenever available (i.e. when running on WinNT)
regardless of build type.
Your proposal takes a step backwards to providing Unicode support only
when compiled as a Unicode application. It is better to continue the
current method of calling W functions where possible and falling back
to
A only when necessary, just process the string sent into the A function
so that it is properly ACP/OEM as necessary.
I'm not sure what you're getting all worked up over. How is "providing
unicode support only when compiled as a unicode application" a Bad Thing?
Because as explained, the current sqlite DLL provides Unicode support to
Windows NT clients regardless of whether built as ansi or unicode.
Therefore a single DLL supports all clients on either Win95 or WinNT
with best available functionality. The Windows way of either/or is not
as flexible. You compile with as a unicode DLL and it isn't usable on
Win95 (unless you also use MSLU). You compile as multibyte and you lose
full unicode functionality on winNT. It's lose/lose compared to the
current method.
New patch to fix this. Attached to the original bug and included.
http://www.sqlite.org/cvstrac/tktview?tn=2023
--- ..\sqlite-source-3_3_8.orig\os.h2006-10-08 13:51:00.0 -0300
+++ os.h2006-12-20 21:30:39.612512000 -0300
@@ -133,4 +133,5 @@
#define sqlite3OsFree sqlite3GenericFree
#define sqlite3OsAllocationSize sqlite3GenericAllocationSize
+HANDLE sqlite3WinLoadLibrary(const char*);
#endif
#if OS_OS2
--- ..\sqlite-source-3_3_8.orig\os_win.c2006-10-08 13:51:00.0
-0300
+++ os_win.c2006-12-20 21:32:25.46472 -0300
@@ -1556,3 +1556,23 @@
return pTsd;
}
+
+/*
+** Return TRUE if the named file exists.
+*/
+HANDLE sqlite3WinLoadLibrary(const char *zFilename){
+ HANDLE h = NULL;
+ WCHAR *zWide = utf8ToUnicode(zFilename);
+ if( zWide ){
+h = LoadLibraryW(zWide);
+sqliteFree(zWide);
+ }else{
+#if OS_WINCE
+return NULL;
+#else
+h = LoadLibraryA(zFilename);
+#endif
+ }
+ return h;
+}
+
#endif /* OS_WIN */
--- ..\sqlite-source-3_3_8.orig\loadext.c 2006-10-08 13:51:00.0
-0300
+++ loadext.c 2006-12-20 21:28:19.420926400 -0300
@@ -224,5 +224,5 @@
# include
# define SQLITE_LIBRARY_TYPE HANDLE
-# define SQLITE_OPEN_LIBRARY(A) LoadLibrary(A)
+# define SQLITE_OPEN_LIBRARY(A) sqlite3WinLoadLibrary(A)
# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
# define SQLITE_CLOSE_LIBRARY(A) FreeLibrary(A)
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
RE: [sqlite] Need a wince test
> -Original Message-
> From: Brodie Thiesfield [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, December 20, 2006 4:43 PM
> To: sqlite-users@sqlite.org
> Subject: Re: [sqlite] Need a wince test
>
> Robert Simpson wrote:
> >> From: Brodie Thiesfield [mailto:[EMAIL PROTECTED]
> >> Robert, you are missing the point. Because of the way this is being
> >> defined, there is a need to check for _UNICODE. If you don't then a
> >> build with _UNICODE defined will fail. If it was implemented like
> the
> >> rest of the functions in os_win.c then it wouldn't be necessary.
> >
> > I can go either way on that. There is no need to check for _UNICODE
> if you
> > change the defines ... to this:
> >
> > # ifdef _WIN32_WCE
> > //snip
> > # else
> > # define SQLITE_OPEN_LIBRARY(A) LoadLibraryA(A) // <-- changed to
> A
> > # define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
> > # endif
>
> That way you are ensuring that non-ASCII strings won't work on any
> platform other than WINCE. It's the worse solution so far. Why do you
> have such a problem accepting the _UNICODE define? I feel like I am
> arguing with a brick wall.
I don't have a problem with the _UNICODE define, I have a problem with the
differences in support between CE and the desktop.
> The following patch is a version of drh's that will compile and work on
> in the cases that you and I want (e.g. Unicode build). See my original
> reply to drh as to why it is better than his (although see below for
> why
> it is still bad).
>
> # ifdef _UNICODE // hey, look, we catch _WIN_WCE here too!
> static HANDLE loadLibraryUtf8(const char *z){
>WCHAR zWide[MAX_PATH];
>DWORD dwLen = MultiByteToWideChar(CP_UTF8,0,z,-1,zWide,MAX_PATH);
>if (dwLen == 0 || dwLen > MAX_PATH) return NULL;
>return LoadLibraryW(zWide);
> }
> # define SQLITE_OPEN_LIBRARY(A) loadLibraryUtf8(A)
> # else
> # define SQLITE_OPEN_LIBRARY(A) LoadLibraryA(A)
> # endif
> # define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
>
> If _UNICODE is defined it works fine. When _UNICODE is not defined (the
> default) it has UTF-8/ANSI coding problems on *all* platforms (unlike
> os_win functions which have them only on Win9x).
CP_UTF8 doesn't work on most CE platforms and hence your proposed patch
doesn't work.
> This is why at a minimum it needs to be included in os_win.c and
> implemented like all other functions there. As per my original patch on
> the bug report. Can we just implement it there, export it from there
> and
> use it in loadext.c without touching the os.h header that drh seems so
> allergic to?
>
[snip diagram]
>
> > Here's what I propose ...
> > #ifdef _UNICODE
> > #define OSSTR wchar_t
> > #else
> > #define OSSTR char
> >
> > OSSTR *utf8toOS(char *utf8)
> > void utf8toOSFree(OSSTR *apiString)
>
> Although it is nice since it is simple, it is worse than the current
> solution. Firstly, with the current method, only Windows 95 is broken.
> Additionally, the current solution has the nice property of using the
> unicode functions whenever available (i.e. when running on WinNT)
> regardless of build type.
>
> Your proposal takes a step backwards to providing Unicode support only
> when compiled as a Unicode application. It is better to continue the
> current method of calling W functions where possible and falling back
> to
> A only when necessary, just process the string sent into the A function
> so that it is properly ACP/OEM as necessary.
I'm not sure what you're getting all worked up over. How is "providing
unicode support only when compiled as a unicode application" a Bad Thing?
99.9% of all API calls that take a string have an A and W version defined.
The LoadLibrary() function maps to LoadLibraryW when _UNICODE is defined,
and LoadLibraryA() when _UNICODE is not defined. What's so wrong with this
system? Why is it so terrible to convert a utf8 string to MBCS and call an
A function when _UNICODE isn't defined, and to convert a utf8 string to
unicode and call the W function when _UNICODE *is* defined?
Robert
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Need a wince test
Robert Simpson wrote:
From: Brodie Thiesfield [mailto:[EMAIL PROTECTED]
Robert, you are missing the point. Because of the way this is being
defined, there is a need to check for _UNICODE. If you don't then a
build with _UNICODE defined will fail. If it was implemented like the
rest of the functions in os_win.c then it wouldn't be necessary.
I can go either way on that. There is no need to check for _UNICODE if you
change the defines ... to this:
>
> # ifdef _WIN32_WCE
> //snip
> # else
> # define SQLITE_OPEN_LIBRARY(A) LoadLibraryA(A) // <-- changed to A
> # define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
> # endif
That way you are ensuring that non-ASCII strings won't work on any
platform other than WINCE. It's the worse solution so far. Why do you
have such a problem accepting the _UNICODE define? I feel like I am
arguing with a brick wall.
The following patch is a version of drh's that will compile and work on
in the cases that you and I want (e.g. Unicode build). See my original
reply to drh as to why it is better than his (although see below for why
it is still bad).
# ifdef _UNICODE // hey, look, we catch _WIN_WCE here too!
static HANDLE loadLibraryUtf8(const char *z){
WCHAR zWide[MAX_PATH];
DWORD dwLen = MultiByteToWideChar(CP_UTF8,0,z,-1,zWide,MAX_PATH);
if (dwLen == 0 || dwLen > MAX_PATH) return NULL;
return LoadLibraryW(zWide);
}
# define SQLITE_OPEN_LIBRARY(A) loadLibraryUtf8(A)
# else
# define SQLITE_OPEN_LIBRARY(A) LoadLibraryA(A)
# endif
# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
If _UNICODE is defined it works fine. When _UNICODE is not defined (the
default) it has UTF-8/ANSI coding problems on *all* platforms (unlike
os_win functions which have them only on Win9x).
This is why at a minimum it needs to be included in os_win.c and
implemented like all other functions there. As per my original patch on
the bug report. Can we just implement it there, export it from there and
use it in loadext.c without touching the os.h header that drh seems so
allergic to?
This is so simple. I'll draw a diagram.
If you want...
-> full support on WinNT/WinCE when built in _UNICODE,
broken when using non-ASCII chars at all other times?
== Implement as above.
-> full support on WinNT/WinCE regardless of build type,
broken when using non-ASCII chars on Win9x?
== Implement like all other functions in os_win,
export from there, use in load_ext.c
-> full support on all platforms regardless of build type
== Implement like all other functions in os_win,
export from there, use in load_ext.c
+ fix all functions in os_win to work on Win9x
Here's what I propose ...
> #ifdef _UNICODE
> #define OSSTR wchar_t
> #else
> #define OSSTR char
>
> OSSTR *utf8toOS(char *utf8)
> void utf8toOSFree(OSSTR *apiString)
Although it is nice since it is simple, it is worse than the current
solution. Firstly, with the current method, only Windows 95 is broken.
Additionally, the current solution has the nice property of using the
unicode functions whenever available (i.e. when running on WinNT)
regardless of build type.
Your proposal takes a step backwards to providing Unicode support only
when compiled as a Unicode application. It is better to continue the
current method of calling W functions where possible and falling back to
A only when necessary, just process the string sent into the A function
so that it is properly ACP/OEM as necessary.
Brodie
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
RE: [sqlite] Need a wince test
> -Original Message-
> From: Brodie Thiesfield [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, December 20, 2006 12:17 PM
> To: sqlite-users@sqlite.org
> Subject: Re: [sqlite] Need a wince test
>
> Robert, does the patch I provided work as is on Windows CE or not?
>
> Robert Simpson wrote:
> > There's some flaws in your arguments, Brodie ...
> >
> > 1. There's no need to do this whole _UNICODE test, only the _WINCE
> test is
> > needed. All versions of Windows that support unicode also support
> the ANSI
> > versions of the API calls -- the only Windows platform that doesn't
> have any
> > ANSI support is Windows CE. A single one-character modification to
> DRH's
> > proposed patch is all that's needed for regular Windows desktop
> support.
>
> Robert, you are missing the point. Because of the way this is being
> defined, there is a need to check for _UNICODE. If you don't then a
> build with _UNICODE defined will fail. If it was implemented like the
> rest of the functions in os_win.c then it wouldn't be necessary.
I can go either way on that. There is no need to check for _UNICODE if you
change the defines from this:
# ifdef _WIN32_WCE
//snip
# else
# define SQLITE_OPEN_LIBRARY(A) LoadLibrary(A)
# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
# endif
To this:
# ifdef _WIN32_WCE
//snip
# else
# define SQLITE_OPEN_LIBRARY(A) LoadLibraryA(A) // <-- changed to A
# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
# endif
Here's what I propose ...
#ifdef _UNICODE
#define OSSTR wchar_t
#else
#define OSSTR char
OSSTR *utf8toOS(char *utf8)
{
// for windows, allocate and return an OS string of the right encoding and
type
// For *nix, return the utf8 string as-is
}
void utf8toOSFree(OSSTR *apiString)
{
// For windows, free the apiString
// For *nix, do nothing
}
Once those are declared in all the os's, then we can dispense with the
rigamarol of calling CreateFileW and CreateFileA, and LoadLibraryW and
LoadLibraryA, etc. Anytime you make an OS-level call, you call these two
functions to allocate and free an appropriately-converted string. On *nix
these will be noop functions.
There will still be one exception, but its minor. GetProcAddress() in
Windows takes an ANSI string for the function name always. However, since
you can't export a function with a name outside the normal ASCII range,
there's no need to convert the utf8 string to an OS-specific string.
Robert
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Need a wince test
[EMAIL PROTECTED] wrote: > Brodie Thiesfield <[EMAIL PROTECTED]> wrote: [...] > SQLite should never expect strings in any encoding other > than UTF-8 or UTF-16. (Note that ASCII is a proper subset > of UTF-8 so SQLite will also accept ASCII.) I do not know > what CP_ACP is, but if it is not a subset of UTF-8 then > SQLite should (if implemented as designed) malfunction if > you give it a CP_ACP string that has a different representation > than the equivalent UTF-8. > > If you find a case where SQLite is expecting some character > encoding other than UTF-8 or UTF-16, then you have found a bug. > Please create a ticket and suggest an appropriate patch. CP_ACP is ANSI code page. The character encoding usually used by the file API on Windows. It will be different depending on the Windows legacy locale. e.g. Shift-JIS for Japanese. EUC-KR for Korean. I have filed the following ticket for this bug: http://www.sqlite.org/cvstrac/tktview?tn=2121 > My intent is to let yall continue to debate a proper fix > for porting the extension loading mechanism to wince and > then once you reach consensus I will check in whatever that > consensus happens to be. The problem is not just Windows CE. The current problem of the Unicode build (and therefore *also* a Windows CE build) failing can be temporarily fixed with the patch that I supplied in the first reply to your email. The larger problem of Win9x clients not seeing a ANSI char* API instead of the specified UTF-8 char* API is a problem that needs to be fixed with larger changes to the win_os.c layer. The LoadLibrary call should also be implemented there. What is the problem with implementing the LoadLibrary call in the os_win.c layer? Regards, Brodie - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Need a wince test
Brodie Thiesfield <[EMAIL PROTECTED]> wrote: > > As far as I understand, sqlite changed sometime in the early 3.0 > versions such that char* strings (input and output) are always assumed > to be UTF-8. > [...] > > Yes. But this is a separate problem. Look at the code in os_win.c, e.g. > sqlite3WinFileExists. On Windows NT platforms, it will work fine only if > the caller supplies a ASCII or UTF-8 string (CP_ACP encoded strings will > fail). On Windows 9x platforms it will work fine only if the caller > supplies an ASCII or CP_ACP string (UTF-8 encoded strings will fail). > > Probably most users on Win9x are still passing in ANSI strings which is > why it is working. Change it to actually match the documentation and you > will probably see a few more failures. > The intent of SQLite is that *all* input strings should be either UTF-8 or UTF-16. Usually strings should be UTF-8. Functions that accept UTF-16 are clearly labeled with a "16" at the end of their name. SQLite should never expect strings in any encoding other than UTF-8 or UTF-16. (Note that ASCII is a proper subset of UTF-8 so SQLite will also accept ASCII.) I do not know what CP_ACP is, but if it is not a subset of UTF-8 then SQLite should (if implemented as designed) malfunction if you give it a CP_ACP string that has a different representation than the equivalent UTF-8. If you find a case where SQLite is expecting some character encoding other than UTF-8 or UTF-16, then you have found a bug. Please create a ticket and suggest an appropriate patch. Note that there only two real platforms in the world: win32 and unix. In the unix world everything is UTF-8 and so it gives no problems in this area. Character encoding confusion seems to be confined to the win32 world. I do not use win32. And I have no desire to learn to use it. So I am depending on the win32 user community to find and fix problems with win32 character encodings. My intent is to let yall continue to debate a proper fix for porting the extension loading mechanism to wince and then once you reach consensus I will check in whatever that consensus happens to be. Thank you for your assistance. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Need a wince test
Robert, does the patch I provided work as is on Windows CE or not? Robert Simpson wrote: There's some flaws in your arguments, Brodie ... 1. There's no need to do this whole _UNICODE test, only the _WINCE test is needed. All versions of Windows that support unicode also support the ANSI versions of the API calls -- the only Windows platform that doesn't have any ANSI support is Windows CE. A single one-character modification to DRH's proposed patch is all that's needed for regular Windows desktop support. Robert, you are missing the point. Because of the way this is being defined, there is a need to check for _UNICODE. If you don't then a build with _UNICODE defined will fail. If it was implemented like the rest of the functions in os_win.c then it wouldn't be necessary. 2. You're ignoring that the ANSI versions that already exist in loadext.c are potentially passing a utf8 char * to LoadLibraryA() right now and have been since loadext.c was created. If you really want to be technical about it, all the strings passed to LoadLibrary() and GetProcAddress() need to be converted from utf8 to MBCS for non-CE platforms. As far as I understand, sqlite changed sometime in the early 3.0 versions such that char* strings (input and output) are always assumed to be UTF-8. See: http://www.sqlite.org/cvstrac/tktview?tn=1695 http://www.sqlite.org/cvstrac/tktview?tn=1533 If these strings are passed directly into win32 ansi functions then that is a different bug and one that needs to be fixed. It is not part of this bug. This patch would also need to be fixed too. There are two possible solutions, and these are the same issues we've had in os_win ... 1. Bite the bullet and realize that Windows API calls are natively MBCS, not utf8, and convert any char * from utf8 to mbcs before passing them to a Windows API call. Given the current API documentation and what has been written in the bug referenced earlier, this seems the only possibility. DRH, your proposed patch almost works with _UNICODE defined. Just change it to this instead (my changes are marked with -->): No, it doesn't. It needs to have the define check changed to _UNICODE instead of _WIN32_WCE (which will still support CE). It also needs error checking of the conversion. It needs to call the correct functions. Ideally it should be implemented like the rest of the functions in os_win.c Unfortunately the larger issue still remains, that all ANSI api calls in Windows expect MBCS strings and not utf8 strings. Technically what you should do is have two functions: loadLibraryUtf16() for WINCE that converts the utf8 string to utf16/unicode and calls LoadLibraryW() loadLibraryUtf8() for the rest of Windows, which converts a utf8 string to mbcs and calls LoadLibraryA() Yes. But this is a separate problem. Look at the code in os_win.c, e.g. sqlite3WinFileExists. On Windows NT platforms, it will work fine only if the caller supplies a ASCII or UTF-8 string (CP_ACP encoded strings will fail). On Windows 9x platforms it will work fine only if the caller supplies an ASCII or CP_ACP string (UTF-8 encoded strings will fail). Probably most users on Win9x are still passing in ANSI strings which is why it is working. Change it to actually match the documentation and you will probably see a few more failures. Brodie - To unsubscribe, send email to [EMAIL PROTECTED] -
RE: [sqlite] Need a wince test
> -Original Message-
> From: Brodie Thiesfield [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, December 19, 2006 9:17 PM
> To: sqlite-users@sqlite.org
> Subject: Re: [sqlite] Need a wince test
>
> There are a few problems with your patch.
>
> +# ifdef _WIN32_WCE
> +static HANDLE loadLibraryUtf8(const char *z){
> + WCHAR zWide[MAX_PATH];
> + MultiByteToWideChar(CP_ACP,0,z,-1,zWide,MAX_PATH);
> + return LoadLibrary(zWide);
> +}
> +# define SQLITE_OPEN_LIBRARY(A) loadLibraryUtf8(A)
> +# define SQLITE_FIND_SYMBOL(A,B) GetProcAddressA(A,B)
> +# else
> +# define SQLITE_OPEN_LIBRARY(A) LoadLibrary(A)
> +# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
> +# endif
>
> The problem in question is not a Windows CE only one. It occurs with
> any
> client that builds in Unicode mode. Therefore you need to test for the
> _UNICODE define instead of _WINCE. Windows CE compilers will also set
> _UNICODE (I believe - Robert?).
>
> CP_ACP is not UTF-8. Use either CP_UTF8 or your own UTF-8 conversion
> functions from the OS library. Note also that MultiByteToWideChar may
> fail or return ERROR_NO_UNICODE_TRANSLATION (1113L) for UTF-8
> conversions.
>
> There is no GetProcAddressA. You need to use GetProcAddress.
>
> The patch will need to be something like the following. Which I have
> tested and builds with no errors or warnings in _UNICODE mode. Still
> need someone to test it in WINCE to be sure.
There's some flaws in your arguments, Brodie ...
1. There's no need to do this whole _UNICODE test, only the _WINCE test is
needed. All versions of Windows that support unicode also support the ANSI
versions of the API calls -- the only Windows platform that doesn't have any
ANSI support is Windows CE. A single one-character modification to DRH's
proposed patch is all that's needed for regular Windows desktop support.
2. You're ignoring that the ANSI versions that already exist in loadext.c
are potentially passing a utf8 char * to LoadLibraryA() right now and have
been since loadext.c was created. If you really want to be technical about
it, all the strings passed to LoadLibrary() and GetProcAddress() need to be
converted from utf8 to MBCS for non-CE platforms.
There are two possible solutions, and these are the same issues we've had in
os_win ...
1. Bite the bullet and realize that Windows API calls are natively MBCS,
not utf8, and convert any char * from utf8 to mbcs before passing them to a
Windows API call.
2. Just assume that certain functions for Windows are going to take an mbcs
string instead of a utf8 string, like sqlite3_open() and
sqlite3_load_extension()
DRH, your proposed patch almost works with _UNICODE defined. Just change it
to this instead (my changes are marked with -->):
--- loadext.c 2006/09/22 23:38:21 1.14
+++ loadext.c 2006/12/20 03:37:35 1.15
@@ -223,8 +223,18 @@
#if defined(_WIN32) || defined(WIN32) || defined(__MINGW32__) ||
defined(__BORLANDC__)
# include
# define SQLITE_LIBRARY_TYPE HANDLE
-# define SQLITE_OPEN_LIBRARY(A) LoadLibrary(A)
-# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
+# ifdef _WIN32_WCE
+static HANDLE loadLibraryUtf8(const char *z){
+ WCHAR zWide[MAX_PATH];
+ MultiByteToWideChar(CP_ACP,0,z,-1,zWide,MAX_PATH);
+ return LoadLibrary(zWide);
+}
+# define SQLITE_OPEN_LIBRARY(A) loadLibraryUtf8(A)
--> +# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
+# else
--> +# define SQLITE_OPEN_LIBRARY(A) LoadLibraryA(A)
+# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
+# endif
# define SQLITE_CLOSE_LIBRARY(A) FreeLibrary(A)
#endif /* windows */
Unfortunately the larger issue still remains, that all ANSI api calls in
Windows expect MBCS strings and not utf8 strings. Technically what you
should do is have two functions:
loadLibraryUtf16() for WINCE that converts the utf8 string to utf16/unicode
and calls LoadLibraryW()
loadLibraryUtf8() for the rest of Windows, which converts a utf8 string to
mbcs and calls LoadLibraryA()
Robert
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Need a wince test
On 12/20/06, Brodie Thiesfield <[EMAIL PROTECTED]> wrote:
+static HANDLE loadLibraryUtf8(const char *z){
+ WCHAR zWide[MAX_PATH];
+ DWORD dwLen = MultiByteToWideChar(CP_UTF8,0,z,-1,zWide,MAX_PATH);
+ if (dwLen == 0 || dwLen > MAX_PATH) return NULL;
+ return LoadLibraryW(zWide);
+}
I can't test the patch right now, but it's better to use the sqlite
internal UTF-8 to "WideChar" function to MultiByteToWideChar, because
older WinCE versions didn't support CP_UTF8 as parameter (although I
think it was up to the WinCE platform builders to have more "character
encodings" built-in).
Regards,
~Nuno Lucas
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Need a wince test
There are a few problems with your patch.
+# ifdef _WIN32_WCE
+static HANDLE loadLibraryUtf8(const char *z){
+ WCHAR zWide[MAX_PATH];
+ MultiByteToWideChar(CP_ACP,0,z,-1,zWide,MAX_PATH);
+ return LoadLibrary(zWide);
+}
+# define SQLITE_OPEN_LIBRARY(A) loadLibraryUtf8(A)
+# define SQLITE_FIND_SYMBOL(A,B) GetProcAddressA(A,B)
+# else
+# define SQLITE_OPEN_LIBRARY(A) LoadLibrary(A)
+# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
+# endif
The problem in question is not a Windows CE only one. It occurs with any
client that builds in Unicode mode. Therefore you need to test for the
_UNICODE define instead of _WINCE. Windows CE compilers will also set
_UNICODE (I believe - Robert?).
CP_ACP is not UTF-8. Use either CP_UTF8 or your own UTF-8 conversion
functions from the OS library. Note also that MultiByteToWideChar may
fail or return ERROR_NO_UNICODE_TRANSLATION (1113L) for UTF-8 conversions.
There is no GetProcAddressA. You need to use GetProcAddress.
The patch will need to be something like the following. Which I have
tested and builds with no errors or warnings in _UNICODE mode. Still
need someone to test it in WINCE to be sure.
# include
# define SQLITE_LIBRARY_TYPE HANDLE
-# define SQLITE_OPEN_LIBRARY(A) LoadLibrary(A)
+# ifdef _UNICODE
+static HANDLE loadLibraryUtf8(const char *z){
+ WCHAR zWide[MAX_PATH];
+ DWORD dwLen = MultiByteToWideChar(CP_UTF8,0,z,-1,zWide,MAX_PATH);
+ if (dwLen == 0 || dwLen > MAX_PATH) return NULL;
+ return LoadLibraryW(zWide);
+}
+# define SQLITE_OPEN_LIBRARY(A) loadLibraryUtf8(A)
+# else
+# define SQLITE_OPEN_LIBRARY(A) LoadLibrary(A)
+# endif
# define SQLITE_FIND_SYMBOL(A,B) GetProcAddress(A,B)
# define SQLITE_CLOSE_LIBRARY(A) FreeLibrary(A)
Regards,
Brodie
[EMAIL PROTECTED] wrote:
> Can somebody with access to wince please test patch [3537]
> for me and let me know if it works to fix ticket #2023.
>
> http://www.sqlite.org/cvstrac/chngview?cn=3537
> http://www.sqlite.org/cvstrac/tktview?tn=2023
>
> --
> D. Richard Hipp <[EMAIL PROTECTED]>
>
>
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
>
>
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
