To: Logan Owen <logan.o...@gmail.com> I took the attached schema and ran it through gDS (2.11a) with etree from lxml 3.2.3:
generateDS.py -qf -o foo2.py -a foo: --external-encoding='utf-8' --export="etree" foo.xsd I then used that in the attached test program. The problems are: 1. The resulting document is missing the <?xml ?> directive. 2. The namespace declaration drops the schema declaration. 3. The namespace prefix for the elements are wrong (only the document should have a namespace prefix, everything else should be bare). 4. The generated XML uses XML entities instead of UTF-8 encoded data. 5. If you pass encoding='utf-8' to etree.tostring(), trying to write the resulting *string* raises a UnicodeDecodeError, complaining about the 'ascii' codec. The result should be *unicode*, not string. This is ultimately the same problem that the export functions have. Thus you get this (which doesn't validate): <ns0:transactions xmlns:ns0="http://example.com/foo/foo-schema";> <ns0:insert> <ns0:items> <ns0:Foo> <ns0:foo>هذا هو الاختبار.</ns0:foo> </ns0:Foo> <ns0:Bar> <ns0:bar>これはテストです。</ns0:bar> </ns0:Bar> <ns0:Baz> <ns0:baz>Это тест.</ns0:baz> </ns0:Baz> </ns0:items> </ns0:insert> </ns0:transactions> Instead of this, which is what I want: <?xml version="1.0" encoding="utf-8"?> <foo:transactions xmlns:xs="http://www.w3.org/2001/XMLSchema"; xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance"; xmlns:foo=" http://example.com/foo/foo-schema"; xsi:schemaLocation="http:// example.com/foo/foo-schema foo.xsd "> <insert> <items> <Foo> <foo>هذا هو الاختبار.</foo> </Foo> <Bar> <bar>これはテストです。</bar> </Bar> <Baz> <baz>Это тест.</baz> </Baz> </items> </insert> </foo:transactions> Problem #1 seems to be a common problem -- gDS and etree both do it, but they are both *wrong* when using an external encoding. Problems #2, and #3 are actually gDS bugs -- gDS isn't passing all of the necessary/correct information to etree. Problem #4 comes from the wrapper doing base64 encoding of strings before passing them to etree. I haven't run #5 to ground yet. Thus I concluded that it was easier to do what I did: hack up the generator to fix the unicode problem, and then post-process as I described to fix the namespace problems. - Craig - On Thu, Aug 29, 2013 at 10:50 PM, W. Craig Trader <craig.tra...@gmail.com>wrote: > Well, as previously noted, my actual schema is significantly more > complicated than the sample I posted, and I can't release it. I'll see if I > can reproduce the problem, but it won't be until after Labor Day. > > Thanks again. > > - Craig - > On Aug 29, 2013 10:32 PM, "Logan Owen" <logan.o...@gmail.com> wrote: > >> Craig, >> >> That is unfortunate. Just for my own edification, I'd be interested in >> the specific errors you had with the etree implementation, because I rely >> quite heavily on it and would like to address any potential problems. >> >> -- Logan >> >> >> On Thu, Aug 29, 2013 at 10:18 PM, W. Craig Trader <craig.tra...@gmail.com >> > wrote: >> >>> Logan ... >>> >>> I tried using etree, and had problems with that as well. I forget what >>> the exact problem was, but it may have had to do with namespaces and/or >>> schema validation. >>> >>> - Craig - >>> On Aug 29, 2013 9:45 PM, "Logan Owen" <logan.o...@gmail.com> wrote: >>> >>>> Craig, >>>> >>>> I'm not sure what your use case is, but as an alternative, you might >>>> try using the "etree" export. So generate the bindings like: >>>> >>>> generateDS.py -qf -o foo.py --external-encoding='utf-8' --export="write >>>> etree" foo.xsd >>>> >>>> >>>> Then use the attached mintest-2.py file to write out your file. As far >>>> as I can tell, it successfully writes out the XML. You will need lxml, >>>> which I believe most distributions should have. >>>> >>>> Hope this helps, >>>> Logan >>>> >>>> >>>> On Thu, Aug 29, 2013 at 8:58 PM, W. Craig Trader < >>>> craig.tra...@gmail.com> wrote: >>>> >>>>> I have an application that needs to write I18N data to XML, using >>>>> UTF-8 encoding, running on Python 2.7. See attached schema, and example >>>>> program. I generated the foo.py wrapper as follows: >>>>> >>>>> generateDS.py -qf -o foo.py --external-encoding='utf-8' >>>>> --export="write" foo.xsd >>>>> >>>>> When I run the test program, I get the following error: >>>>> >>>>> Traceback (most recent call last): >>>>> File "mintest.py", line 15, in <module> >>>>> a.export( out, 0 ) >>>>> File "/home/ctrader/src/gds-test/foo.py", line 648, in export >>>>> self.exportChildren(outfile, level + 1, namespace_, name_, >>>>> pretty_print=pretty_print) >>>>> File "/home/ctrader/src/gds-test/foo.py", line 668, in exportChildren >>>>> outfile.write('<%sfoo>%s</%sfoo>%s' % (namespace_, >>>>> self.gds_format_string(quote_xml(self.foo).encode(ExternalEncoding), >>>>> input_name='foo'), namespace_, eol_)) >>>>> File "/home/ctrader/foo/lib/python2.7/codecs.py", line 691, in write >>>>> return self.writer.write(data) >>>>> File "/home/ctrader/foo/lib/python2.7/codecs.py", line 351, in write >>>>> data, consumed = self.encode(object, self.errors) >>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xd9 in position >>>>> 5: ordinal not in range(128) >>>>> >>>>> After a long debugging session, I determined that the problem is that >>>>> the code that generateDS generates for the export methods uses regular >>>>> strings, not unicode strings, for its constants. When you concatenate >>>>> 'aaa' + X + 'ccc', it works fine, as long as X is a regular string, or a >>>>> unicode string containing only ASCII characters. When X isn't ASCII, you >>>>> get the exception I noted above. >>>>> >>>>> For my short-term problem, I went through the entire generateDS.py >>>>> program and converted every statement that emits code for the wrapper >>>>> classes to emit unicode constants instead of string constants, and got rid >>>>> of the calls to gds_format_string(), but that's not a good long-term fix. >>>>> >>>>> Can anyone suggest a better solution? >>>>> >>>>> Other problems observed: >>>>> >>>>> - gDS doesn't generate the <?xml ?> directive at the start of the >>>>> export. >>>>> - gDS does not deal well with complex namespaces in schemas. >>>>> >>>>> I worked around both of those by writing my data to a string, and then >>>>> massaging the string before writing the result to a file. Again, it works >>>>> in the short-term, but isn't a long-term solution. >>>>> >>>>> - Craig - >>>>> >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! >>>>> Discover the easy way to master current and previous Microsoft >>>>> technologies >>>>> and advance your career. Get an incredible 1,500+ hours of step-by-step >>>>> tutorial videos with LearnDevNow. Subscribe today and save! >>>>> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk >>>>> _______________________________________________ >>>>> generateds-users mailing list >>>>> generateds-users@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/generateds-users >>>>> >>>>> >>>> >>
foo.xsd
Description: Binary data
test.py
Description: Binary data
<ns0:transactions xmlns:ns0="http://example.com/foo/foo-schema";> <ns0:insert> <ns0:items> <ns0:Foo> <ns0:foo>هذا هو الاختبار.</ns0:foo> </ns0:Foo> <ns0:Bar> <ns0:bar>これはテストです。</ns0:bar> </ns0:Bar> <ns0:Baz> <ns0:baz>Это тест.</ns0:baz> </ns0:Baz> </ns0:items> </ns0:insert> </ns0:transactions>
<?xml version="1.0" encoding="utf-8"?> <foo:transactions xmlns:xs="http://www.w3.org/2001/XMLSchema"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xmlns:foo="http://example.com/foo/foo-schema"; xsi:schemaLocation="http://nmec.dodiis.mil/nexsys/entity-schema nx-entity-schema.xsd "> <insert> <items> <Foo> <foo>هذا هو الاختبار.</foo> </Foo> <Bar> <bar>これはテストです。</bar> </Bar> <Baz> <baz>Это тест.</baz> </Baz> </items> </insert> </foo:transactions>
------------------------------------------------------------------------------ Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk
_______________________________________________ generateds-users mailing list generateds-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/generateds-users
