Re: [Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
Hi Borys, I agree to specify the ENCODING option. LDID/87 does not have definite meaning (depending on the environment), so avoiding to write shapefile with the value would be nice. If the output encoding is UTF-8, QGIS/OGR will generate cpg file that contains UTF-8. This behavior must be ideal. In case of the other encoding, even if there is a problem with the content of a cpg file (e.g. use in another application. I am not much sure about cpg file), the user will be able to fix it easily. Similarly, how about specifying the ENCODING when creating new shapefile (in createEmptyDataSource())? And one longterm: - after iconv conversion tio UTF-8 the layer doesn't remember the source encoding, so it's impossible to preserve it e.g when saving from Table Manager. In QgsVectorFileWriter, how about always using SHAPE_ENCODING= ? It will solve the problem and never make any encoding conversion failure with current encoding listbox. I also added Shift-JIS to the Shapefile Encoding Fixer plugin. Please let me know if any of you needs (or just knows) another important encodings. Thank you for your work on the plugin. Regards, Minoru Akagi ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
Similarly, how about specifying the ENCODING when creating new shapefile (in createEmptyDataSource())? Done In QgsVectorFileWriter, how about always using SHAPE_ENCODING= ? It will solve the problem and never make any encoding conversion failure with current encoding listbox. You're right, done ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
Hi, Thanks to all for the feedback :) I'm really not very familar with all the OGR encoding stuff. I applied my pull request [1] to allow wider testing. Many thanks to Minoru Akagi for two other patches. One of them is applied[2], the second not, because my patch covers this isue too. Your patch [3] was safer, as it just reset LDID to 0 on save, but I believe we should go on and use the encoding declaration. A big difference I see in our approaches is I set ENCODING to encoding name, while you you use LDID/0. I guess the LDID/ prefix tells OGR to use LDID instead of CPG? I think if we should use it for encodings supported by LDID, but I guess it's not worth of the additional work, unless some other important software prefers LDID over CPG. I also added Shift-JIS to the Shapefile Encoding Fixer plugin. Please let me know if any of you needs (or just knows) another important encodings. Still there are two urgent problems: - the System encoding has to be replaced with the proper value - unit tests are still missing And one longterm: - after iconv conversion tio UTF-8 the layer doesn't remember the source encoding, so it's impossible to preserve it e.g when saving from Table Manager. Maybe someone wants to take one of the remaining issues? [1] https://github.com/qgis/Quantum- GIS/commit/c0551a68c250489955c9831f5714f187df087d83 [2] http://hub.qgis.org/issues/7647 [3] http://hub.qgis.org/attachments/4749/qgsogrprovider3.patch ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
I do care a lot as is it essential for Chinese user to have a software able to read and write utf8 and gb2312 encoding without any issue. It seems QGis Master already mostly solved the issues I met with QGis 1.8, but I willing to help to make QGis 2.0 better in this aspect. What is the exact format needed for the test files? On 04/12/2013 07:30 PM, Borys Jurgiel wrote: Hi All, As most of us know, QGIS 1.8 and 1.9 *in fact doesn't support Shapefiles* with encodings other than Latin-1. Assuming that all recent QGIS versions use GDAL 1.9, *the last usable version was 1.7.3* (at least for windows users in Poland). Thanks to the 'ignore shapefile encoding' workaround, QGIS built against GDAL 1.9 is at least able to open other encodings, however, the 'Save layer as' action still creates corrupted files unless you know how to trick the encoding selector as well as the layer creating options. From my point of view it's just an *incredible regression*, as QGIS just doesn't support Shapefiles in most languages for a year or so. If there is anybody else interested in bringing back support for at least UTF-8 and cp12xx in QGIS 2.0 plase join me ;)) and review the pull request: https://github.com/qgis/Quantum-GIS/pull/492#issuecomment-16282081 Maybe someone have a better solution? Maybe someone can provide the test files Jürgen suggested? I don't believe nobody cares whether QGIS 2.0 supports not Latin-1 Shapefiles or not. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
On 04/12/2013 01:30 PM, Borys Jurgiel wrote: Hi All, As most of us know, QGIS 1.8 and 1.9 *in fact doesn't support Shapefiles* with encodings other than Latin-1. Assuming that all recent QGIS versions use GDAL 1.9, *the last usable version was 1.7.3* (at least for windows users in Poland). Thanks to the 'ignore shapefile encoding' workaround, QGIS built against GDAL 1.9 is at least able to open other encodings, however, the 'Save layer as' action still creates corrupted files unless you know how to trick the encoding selector as well as the layer creating options. From my point of view it's just an *incredible regression*, as QGIS just doesn't support Shapefiles in most languages for a year or so. If there is anybody else interested in bringing back support for at least UTF-8 and cp12xx in QGIS 2.0 plase join me ;)) and review the pull request: Hi Borys, we use QGIS compiled with GDAL 1.7 on Debian Squeeze and Ubuntu Precise, so we are currently not affected with this regression. I agree, it is total show stopper us too (Slovak and Czech rep. - windows-1250). I will prepare some shapefile in windows-1250 and utf-8 for testing. For a reference, here [1] is a blog post describing this problem. -- 1 - http://ssrebelious.wordpress.com/2012/03/11/qgis-and-gdal1-9-encoding-issue-a-workaround/ -- Ivan Mincik ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Il 16/04/2013 06:21, Sanghee Shin ha scritto: Dear Borys, Most of all thanks for your efforts. Actually that encoding issue highly affected Korean QGIS users badly and reduced usability. I'm not a developer, however I can help you by testing and giving you test files. Also I'll share your efforts with Korean developers as well. Hi Sanghee, your cooperation would be greatly appreciated. Testing and providing samples would be very useful. All the best. - -- Paolo Cavallini - Faunalia www.faunalia.eu Full contact details at www.faunalia.eu/pc Nuovi corsi QGIS e PostGIS: http://www.faunalia.it/calendario -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlFtTjQACgkQ/NedwLUzIr506ACdG4EP+e7k/W24Z/Hh98a+zngE PZwAnjSOY5E7X2WpkIEPkt6bScJba2Ns =OVxL -END PGP SIGNATURE- ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
Dear Borys, Most of all thanks for your efforts. Actually that encoding issue highly affected Korean QGIS users badly and reduced usability. I'm not a developer, however I can help you by testing and giving you test files. Also I'll share your efforts with Korean developers as well. Thanks again. Sanghee 2013. 4. 12., 오후 8:30, Borys Jurgiel li...@borysjurgiel.pl 작성: Hi All, As most of us know, QGIS 1.8 and 1.9 *in fact doesn't support Shapefiles* with encodings other than Latin-1. Assuming that all recent QGIS versions use GDAL 1.9, *the last usable version was 1.7.3* (at least for windows users in Poland). Thanks to the 'ignore shapefile encoding' workaround, QGIS built against GDAL 1.9 is at least able to open other encodings, however, the 'Save layer as' action still creates corrupted files unless you know how to trick the encoding selector as well as the layer creating options. From my point of view it's just an *incredible regression*, as QGIS just doesn't support Shapefiles in most languages for a year or so. If there is anybody else interested in bringing back support for at least UTF-8 and cp12xx in QGIS 2.0 plase join me ;)) and review the pull request: https://github.com/qgis/Quantum-GIS/pull/492#issuecomment-16282081 Maybe someone have a better solution? Maybe someone can provide the test files Jürgen suggested? I don't believe nobody cares whether QGIS 2.0 supports not Latin-1 Shapefiles or not. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
Hi All, As most of us know, QGIS 1.8 and 1.9 *in fact doesn't support Shapefiles* with encodings other than Latin-1. Assuming that all recent QGIS versions use GDAL 1.9, *the last usable version was 1.7.3* (at least for windows users in Poland). Thanks to the 'ignore shapefile encoding' workaround, QGIS built against GDAL 1.9 is at least able to open other encodings, however, the 'Save layer as' action still creates corrupted files unless you know how to trick the encoding selector as well as the layer creating options. From my point of view it's just an *incredible regression*, as QGIS just doesn't support Shapefiles in most languages for a year or so. If there is anybody else interested in bringing back support for at least UTF-8 and cp12xx in QGIS 2.0 plase join me ;)) and review the pull request: https://github.com/qgis/Quantum-GIS/pull/492#issuecomment-16282081 Maybe someone have a better solution? Maybe someone can provide the test files Jürgen suggested? I don't believe nobody cares whether QGIS 2.0 supports not Latin-1 Shapefiles or not. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Is there anybody interested whether QGIS 2.0 will support non-Latin1 Shapefiles?
Hi Borys! Once I finish the case studies I join you and try to get some test files from unusual encodings werner On Fri, Apr 12, 2013 at 1:30 PM, Borys Jurgiel li...@borysjurgiel.plwrote: Hi All, As most of us know, QGIS 1.8 and 1.9 *in fact doesn't support Shapefiles* with encodings other than Latin-1. Assuming that all recent QGIS versions use GDAL 1.9, *the last usable version was 1.7.3* (at least for windows users in Poland). Thanks to the 'ignore shapefile encoding' workaround, QGIS built against GDAL 1.9 is at least able to open other encodings, however, the 'Save layer as' action still creates corrupted files unless you know how to trick the encoding selector as well as the layer creating options. From my point of view it's just an *incredible regression*, as QGIS just doesn't support Shapefiles in most languages for a year or so. If there is anybody else interested in bringing back support for at least UTF-8 and cp12xx in QGIS 2.0 plase join me ;)) and review the pull request: https://github.com/qgis/Quantum-GIS/pull/492#issuecomment-16282081 Maybe someone have a better solution? Maybe someone can provide the test files Jürgen suggested? I don't believe nobody cares whether QGIS 2.0 supports not Latin-1 Shapefiles or not. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
