Re: [xml] Serialization of documents without encoding
On Thu, Sep 27, 2018 at 02:22:55PM +0200, Nick Wellnhofer wrote: > On 27/09/2018 10:59, Roumen Petrov wrote: > > Let consider case as "file" mode. > > > Let consider case as "stream" code. > > I'm not only talking about xmllint but the serialization API (xmlSave*, > xmlNodeDump*) in general. > > > Now about above test samples . if content is stored in file xmllint > > works fine with encoding(=codeset=charset). > > > > $ cat test-noencoding.xml > > Käse > > No, it doesn't work fine: > > $ xmllint test-noencoding.xml > > Käse > > > (2) Next a-umlaut character is encoded in hexadecimal. Minor > > inconsistency between "stream" and "file" mode. > > As shown above, "file" mode can also produce unwanted numeric character > references. > > > (3) Problem is that in "scream" mode xmllint application ignores value > > of encode argument: > > $ echo 'Käse' | xmllint - --encode UTF-8 > > > > Käse > > Right, there is an inconsistency in xmllint. But that's not my point. > > > From my point of view (1) and (2) are minor non-important issues. Only > > (3) could be fixed with low priority. > > Unneeded numeric character references in UTF-8 output are not a minor issue. > If you're working with non-Latin scripts, it makes serialized XML files > unreadable for humans and blows up the file size. Not breaking a decade os programs who may be expecting that behaviour sounds far more important to me honnestly. Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veill...@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/ ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] Serialization of documents without encoding
On Tue, Sep 25, 2018 at 01:19:51PM +0200, Nick Wellnhofer wrote: > libxml2 serializes documents without an encoding declaration differently > than documents with an explicit UTF-8 encoding: > > $ echo 'Käse' |xmllint - > > Käse > > $ echo 'Käse' |xmllint - > > Käse > > Since the encoding should default to UTF-8, can anyone explain why this > decision was made? Because using the codepoint is part of the core XML spec, there is no way this can be screwed up when people are doing manipulations like cutting parts of an XML document, pasting it somewhere else where the context may be differemt. So if you don't explicitely ask for an encoding libxml2 will deliver the most resilient serialization possible and that means using codepoint, except where not possible (and then specifics about attributes serialization, etc ...) Please keep it that way, you have no idea what people may have done and unless this really fixes an issue I would be very reluctant to change this behaviour. thanks, Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veill...@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/ ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] Serialization of documents without encoding
Sorry I ddin't watch my xml folder for a while ... a bit busy On Sat, Oct 06, 2018 at 07:32:00PM +0300, Roumen Petrov wrote: > Hi Nick, > > Nick Wellnhofer wrote: > > On 25/09/2018 14:36, Nick Wellnhofer wrote: > > > The whole situation is a mess. I'd love to change the code so that > > > non-ASCII chars are always encoded as UTF-8, but I'm scared to break > > > things. > > Long time ago I did some test with html - > http://roumenpetrov.info/tests/charset/ . > > The case is quite similar - encoding could be defined externally in HTTP > header Except it usually doesn't work so tons of workarounds need to be applied. > Content-Type: text/html; charset=ISO8859-5 > ... > and in the same time in HTML header (internal) > ... > > > > > > > ... > If I remember well (10-15 ago) Internet Explorer prefer internal while other > browsers prefer external encoding. yup it was a mess. I heard horror stories from various parties implementing even XML support in browsers. > > I create similar test to check what is situation with xml > http://roumenpetrov.info/tests/charset/index-xml.html and dis some tests ( > ( browsers - Firefox, Opera, Chromium, Konqueror ). > > The test show that all(1) browsers could read xml in following case : > - HTTP header without charset, i.e. Content-Type: text/html; > - XML prolog with encoding, i.e. > > Without encoding in prolog only file in UTF-8 codeset could be read (no > surprise). > > Behavior of some browsers depend from file suffix . This is reason to test > to use .xml and .none suffixes. > > Mix between charset and encoding fail as expected exept in case > charset=iso8859-1 where some browsers show properly content. > > > Based on tests I think that switch to UTF-8 encoded content by default is > good to have encoding in prolog. It is less risky. > > > > This is the change I have in mind: > > > > https://github.com/nwellnhof/libxml2/commit/53551ec2f6a2ef03bfcfb6d73b6fd18dc70ba15d > > Ok to remove "Special escaping routines" but patch shows that in regression > tests prolog remains as "". > I'm not sure that such code modification is save. > That kind of things can backfire *very* easilly. What is the problem we are trying to solve. Some people are likely to expect the behaviour of going back to codepoint when no encoding is specified outside of the ascii range. Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veill...@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/ ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
