Please open an issue in bugzilla. I'll add some debugging to mapserver in cvs and then we'll start from there.
See you tomorrow. Best regards, Umberto On 4/20/06, Benedikt Rothe <[EMAIL PROTECTED]> wrote: > > Hi all > > For experimental purpose I added a "UTF-8 -> ISO-8859-Conversion" > in two functions of mapscript_wrap.c > The functions are the JNI-Implementations of mapObj.getLayerByName and > layerObj.queryByAttributes. > > I testet with the QueryByAttribute-Program Umberto send and with a Shapefile > (no database). > I tested to find the layer named "ÜÄÖßäüö" and in this layer the > regular-expression > "Süden". > > On my Windows-machine the results seem to be correct for the Command-Line- > parameter-case and for the hardcoded-case. > > The implementation is experimental only. If strings are longer than 500 > chars, > unforeseeable things may happen. In practice one should think twice wether > to > use this kind of implementation or to use the build-in Java-encoders from > JNI level. > > But the really difficult question is how anything like this could be > reasonably incorporated > with swig. I have no idea! > > Umberto? What do you think? > > Norbert: To be honorable, I don't think it is promising to change the > Strings in the > higher Java-level. > > For some days code with example can be dowloaded from > http://www2.hydrotec.de/webdemos/be/umlaute.zip > (Compiled mapscript.dll is included. Other dll's can be used from normal > mapserver 4.8.2) > > Benedikt > > PS: I think of this as a bug in mapserver. Should a bug in bugzilla be > opened? > > listuser HH <[EMAIL PROTECTED]> schrieb am 19.04.2006 10:05:13: > > > Hi Benedikt, > > > > thanks for the interest. I have the encoding problem when I try to > > "getLayerByName()". I could get around the problem at this point but I > > think I will have the problem later again. So testing is easy - I just > > use a simple layer with the name "Regierungspräsidien". When I use this > > string to get the layer by name it doesn't work. To be sure I tried to > > check this string with the name of the layer which I get from > > map.getLayer(0) - which is equal to the other. > > > > My thought was to convert the string before using it in mapscript > > functions. Java brings two things to convert strings - perhaps there are > > more. > > > > 1. make a new String from the old in a special encoding - e.g. --> new > > String(oldString.getBytes(), "ISO8859-1") > > 2. convert chars to different encoding - e.g. [snip] --> > > CharToByteISO8859_1 conv4ISO = new > > CharToByteISO8859_1(); > > char[] cs = layerName.toCharArray(); > > conv4ISO.convert(charArray, 0, charArray.length, > > byteArray, 0, byteArray.length); > > new String(byteArray); > > > > Because I'm not sure about which encodings are used at wich stpes I > > tried some combinations but without luck. I think my code from eclipse > > is CP1252. In the eclipse editor properties I changed this to UTF8 and > > ISO which also didn't work. I tried to I'm not sure what happens when > > mapscript use JNI. Perhaps the string gets converted to UTF8. If this is > > right I see no chance for me to change the string in java because it > > gets converted even if it is already UTF8. I have to say that I'm not > > familiar with this encoding things. If someone has an advice I will > > going on testing. > > > > I think it will be a good thing to have "UTF-8 -> ISO-8859-Conversion" > > like you suggested. At the moment I can't do this because I have no > > possibility to compile the c code. > > > > Best regards, > > > > Norbert > > > > Benedikt Rothe wrote: > > > > >Umberto, Nicol, Norbert, Oliver > > > > > >Umberto wrote > > > > > > > > >>try to run the attached Java source. > > >> > > >> > > >... > > > > > > > > >>"Südliche Weinstraße" as the second it will work! > > >> > > >> > > > > > >In my copy of your mail the queryByAttribute.java-program is not > > >attached. Could somebody post program including testdata (or > > >download-url) ? > > > > > >I 'd like to study a running example, because I don't understand > > >how umlaut-conversion from Java to Mapserver-kernel can run > > >properly anyway and I'd like to understand it :-) > > >----------- > > >It seems Norbert found a kind of answer to his question > > > > > > > > > >>is there a way to do the converion in java dircetly? > > >> > > >> > > >He suggested > > > > > > > > > >>Try to convert the String before you set the expression in your code(-> > > >> > > >> > > >String( byte > > > > > > > > >>bytes[], String ) > > >> > > >> > > >Could you be more precisly? I do not understand what must be converted to > > >what. > > >How must this be applied to convert a Java-String to a proper > > >"Mapserver-String" (?) > > >----------- > > >Is somebody willing to try to add an "UTF-8 -> ISO-8859-Conversion" in > > >mapscript_wrap.c for testpurposes? (Even in the case it works, this would > > >not be a real solution because it bypasses swig.) > > > > > >Benedikt > > > > > > > > >UMN MapServer Users List <[email protected]> schrieb am > > >14.04.2006 15:23:51: > > > > > > > > > > > >>Olivier, > > >>I GOT IT! > > >> > > >>try to run the attached Java source. If you pass it two arguments the > > >>first being the path to the map file and the second the string to > > >>search for and you pass > > >>"Südliche Weinstraße" as the second it will work! > > >> > > >>So why does it fail when "Südliche Weinstraße" is inside the Java > > >>code? That is a problem that only happens when javac compiles the > > >>source: javac translates all characters to unicode and in doing that > > >>it gets the german characters wrong. > > >>To solve this give javac the following option: -source 1.4 > > >> > > >>For more see this link: > > > >>http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5046139 > > > >> > > >>On 4/13/06, Umberto Nicoletti <[EMAIL PROTECTED]> wrote: > > >> > > >> > > >>>This is probably not related only to java mapscript, so please read > > >>> > > >>> > > >on. > > > > > > > > >>So I was wrong...but I'll leave the proof to the reader ;-) > > >> > > >>Best regards, > > >>Umberto > > > >> > > >> > > >> > > >>>On 3/30/06, Oliver Wesp <[EMAIL PROTECTED]> wrote: > > >>> > > >>> > > >>>>Dear List, > > >>>> > > >>>>I' struggling with queryByAttributes on an attribute field with > > >>>> > > >>>> > > >german > > > > > > > > >>>>umlauts using java mapscript. > > >>>>The odd thing is that the same thing works fine with php mapscript > > >>>> > > >>>> > > >and > > > > > > > > >>>>when I use expressions in my mapfile. I'm using a shapefile as > > >>>> > > >>>> > > >>datasource. > > > >> > > >> > > >>>Could someone of the other mapserver developers shed some light on > > >>> > > >>> > > >>this issue? > > > >> > > >> > > >>>I have a clue to give: php mapscript is using a different regex > > >>>library and this explains why the match does not happen for Java > > >>>mapscript, while it does happen in php mapscript. If I am right also > > >>>the mapserver cgi should be affected and possibly all other mapscript > > >>>too. > > >>> > > >>>It would be very interesting if someone could report on similar > > >>>experiences with the cgi-bin version of mapserver. > > >>> > > >>>Thanks, > > >>>Umberto > > >>> > > >>> > > >>> > > >>>>Here is what I do: > > >>>> > > >>>>layer.queryByAttributes(map,"KREIS_NAME", "/Südliche Weinstraße/", > > > >>>>mapscriptConstants.MS_MULTIPLE); > > >>>>layer.open(); > > >>>>System.out.println( "Result Count: " +layer.getNumResults() ); > > >>>>layer.close(); > > >>>> > > >>>>The result is always null while replacing the qstring with something > > >>>>that doesn't contain special characters (e.g. > > >>>>'Mainz-Bingen') works fine. > > >>>> > > >>>>As noted above the following layer definition in a mapfile works > > >>>> > > >>>> > > >fine > > > > > > > > >>>>LAYER > > >>>> NAME kreis > > >>>> STATUS DEFAULT > > >>>> TYPE polygon > > >>>> DATA "/tmp/subset" > > >>>> TEMPLATE "kreis.html" > > >>>> CLASSITEM KREIS_NAME > > >>>> CLASS > > >>>> NAME Boundary > > >>>> COLOR 128 128 0 > > >>>> OUTLINECOLOR 0 0 0 > > >>>> EXPRESSION /Südliche Weinstraße/ > > > >>>> END > > >>>>END > > >>>> > > >>>> > > >>>>but this does not: > > >>>> > > >>>>layer.setClassitem("KREIS_NAME"); > > >>>>classObj cl = new classObj(layer); > > >>>>cl.setName("Classname"); > > >>>>cl.setExpression("/Südliche Weinstraße/"); > > > >>>> > > >>>>I use Mapserver 4.8.1 on W2k, Tomcat 5.0.28. > > >>>> > > >>>>I can provide some sample data, just in case someone likes to > > >>>> > > >>>> > > >reproduce. > > > > > > > > >>>>Any help is appreciated. > > >>>> > > >>>>best regards > > >>>>Oliver > > >>>>-- > > >>>>Dipl.-Geogr. Oliver Wesp > > >>>>Gesellschaft fuer geografische Datenverarbeitung > > >>>>Binger Strasse 49-51 > > >>>>D-55218 Ingelheim > > >>>>fon: +49 6132 714818 > > >>>>fax: +49 6132 714828 > > >>>>http: www.gdv.com > > >>>> > > >>>> > > >>>> > > > > > > > > > > > > > > > > > > > > > > > >>is there a way to do the converion in java dircetly? > > >> > > >> > > >He suggested > > > > > > > > > >>Try to convert the String before you set the expression in your code(-> > > >> > > >> > > >String( byte > > > > > > > > >>bytes[], String ) > > >> > > >> > > >Could you be more precisly? I do not understand what must be converted to > > >what. > > >How must this be applied to convert a Java-String to a proper > > >"Mapserver-String" (?) > > >----------- > > >Is somebody willing to try to add an "UTF-8 -> ISO-8859-Conversion" in > > >mapscript_wrap.c for testpurposes? (Even in the case it works, this would > > >not be a real solution because it bypasses swig.) > > > > > >Benedikt > > > > > > > > >UMN MapServer Users List <[email protected]> schrieb am > > >14.04.2006 15:23:51: > > > > > > > > > > > >>Olivier, > > >>I GOT IT! > > >> > > >>try to run the attached Java source. If you pass it two arguments the > > >>first being the path to the map file and the second the string to > > >>search for and you pass > > >>"Südliche Weinstraße" as the second it will work! > > >> > > >>So why does it fail when "Südliche Weinstraße" is inside the Java > > >>code? That is a problem that only happens when javac compiles the > > >>source: javac translates all characters to unicode and in doing that > > >>it gets the german characters wrong. > > >>To solve this give javac the following option: -source 1.4 > > >> > > >>For more see this link: > > > >>http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5046139 > > > >> > > >>On 4/13/06, Umberto Nicoletti <[EMAIL PROTECTED]> wrote: > > >> > > >> > > >>>This is probably not related only to java mapscript, so please read > > >>> > > >>> > > >on. > > > > > > > > >>So I was wrong...but I'll leave the proof to the reader ;-) > > >> > > >>Best regards, > > >>Umberto > > > >> > > >> > > >> > > >>>On 3/30/06, Oliver Wesp <[EMAIL PROTECTED]> wrote: > > >>> > > >>> > > >>>>Dear List, > > >>>> > > >>>>I' struggling with queryByAttributes on an attribute field with > > >>>> > > >>>> > > >german > > > > > > > > >>>>umlauts using java mapscript. > > >>>>The odd thing is that the same thing works fine with php mapscript > > >>>> > > >>>> > > >and > > > > > > > > >>>>when I use expressions in my mapfile. I'm using a shapefile as > > >>>> > > >>>> > > >>datasource. > > > >> > > >> > > >>>Could someone of the other mapserver developers shed some light on > > >>> > > >>> > > >>this issue? > > > >> > > >> > > >>>I have a clue to give: php mapscript is using a different regex > > >>>library and this explains why the match does not happen for Java > > >>>mapscript, while it does happen in php mapscript. If I am right also > > >>>the mapserver cgi should be affected and possibly all other mapscript > > >>>too. > > >>> > > >>>It would be very interesting if someone could report on similar > > >>>experiences with the cgi-bin version of mapserver. > > >>> > > >>>Thanks, > > >>>Umberto > > >>> > > >>> > > >>> > > >>>>Here is what I do: > > >>>> > > >>>>layer.queryByAttributes(map,"KREIS_NAME", "/Südliche Weinstraße/", > > > >>>>mapscriptConstants.MS_MULTIPLE); > > >>>>layer.open(); > > >>>>System.out.println( "Result Count: " +layer.getNumResults() ); > > >>>>layer.close(); > > >>>> > > >>>>The result is always null while replacing the qstring with something > > >>>>that doesn't contain special characters (e.g. > > >>>>'Mainz-Bingen') works fine. > > >>>> > > >>>>As noted above the following layer definition in a mapfile works > > >>>> > > >>>> > > >fine > > > > > > > > >>>>LAYER > > >>>> NAME kreis > > >>>> STATUS DEFAULT > > >>>> TYPE polygon > > >>>> DATA "/tmp/subset" > > >>>> TEMPLATE "kreis.html" > > >>>> CLASSITEM KREIS_NAME > > >>>> CLASS > > >>>> NAME Boundary > > >>>> COLOR 128 128 0 > > >>>> OUTLINECOLOR 0 0 0 > > >>>> EXPRESSION /Südliche Weinstraße/ > > > >>>> END > > >>>>END > > >>>> > > >>>> > > >>>>but this does not: > > >>>> > > >>>>layer.setClassitem("KREIS_NAME"); > > >>>>classObj cl = new classObj(layer); > > >>>>cl.setName("Classname"); > > >>>>cl.setExpression("/Südliche Weinstraße/"); > > > >>>> > > >>>>I use Mapserver 4.8.1 on W2k, Tomcat 5.0.28. > > >>>> > > >>>>I can provide some sample data, just in case someone likes to > > >>>> > > >>>> > > >reproduce. > > > > > > > > >>>>Any help is appreciated. > > >>>> > > >>>>best regards > > >>>>Oliver > > >>>>-- > > >>>>Dipl.-Geogr. Oliver Wesp > > >>>>Gesellschaft fuer geografische Datenverarbeitung > > >>>>Binger Strasse 49-51 > > >>>>D-55218 Ingelheim > > >>>>fon: +49 6132 714818 > > >>>>fax: +49 6132 714828 > > >>>>http: www.gdv.com > > >>>> > > >>>> > > >>>> > > > > > > > > > > > > > >
