Dave,
Our product is an appliance. We mostly run Java, we have some c/c++ code,
and a lot of bash, perl, and python scripts. The appliance runs on Linux.
About a year ago we released a REST API for our customers to use. We
generate XSDs from YANG files using pyang. Then we use JAXB to convert the
XSD files to Java. For use in our own product. I started creating python
scripts as a way to test our REST API. Eventually, the scripts became
examples for how to use our REST API in python and we ship them as part of
our samples.
I was able to run the unit tests and I did encounter many failures, though
most of them were cosmetic. Whitespace changes. Some of them had new
namespace definitions where previously they were missing. One of the
failures was because the test was expecting "ascii" as the external
encoding, but my system Ubuntu 12.04 has a default of UTF-8.
Now there are about 15 failures. They all fail with this:
File "test.py", line 439, in check_result
self.failUnlessEqual(err, len(err), 0)
AssertionError: '' != 0
There isn't an error, but stderr seems to be different on my system than
yours. Have you encountered this?
To answer some of your questions, I am not using --no-process-includes.
With our XSDs, almost all of them import another XSD. I needed to create
python classes for all the XSDs. I could see though that at some point I
may just want to generate the top-level XSD because the rest have already
been created. I'll look into adding that functionality.
I added the following options:
--one-file-per-xsd Create a python class for each XSD processed.
--output-directory="XXX" Used in conjunction with --one-file-per-xsd.
The directory where the XSDs will be created.
--module--suffix="xxx" To be used in conjunction with
--one-file-per-xsd.
Append XXX to the end of each file created.
It didn't occur to me to create an __init__.py file in generateDs. Funny
because I created a bash script to iterate over all the XSDs and execute
generateDS for each one. That script also creates an empty __init__.py
file. I'll look into moving this functionality into generateDS with the
import statements you mentioned. I can also create an __all__ in this file,
not sure if this would be redundant or not.
At this point I'm going to check in the code on my fork of generateDS. It's
at https://bitbucket.org/gsalsero/generateds-fork.
Let me know if you think it's in good enough shape for me to submit a pull
request.
Thanks for the help.
George
On Mon, Sep 23, 2013 at 6:04 PM, Dave Kuhlman <dkuhl...@davekuhlman.org>wrote:
> On Sun 22 Sep 2013 09:25:48 AM PDT, George David wrote:
>
> Hi Dave,
>>
>> I've had a lot of trouble with generateDS not creating valid XML for the
>> XSDs I use. The biggest problem I had was getting generateDS to create XML
>> with valid namespaces. After looking at the code and running the debugger,
>> I believe the problem stems from the fact that generateDS creates a single
>> root with all the included/imported XSDs appended to it. This is done in
>> process_includes.prep_schema_**doc().
>>
>>
> George,
>
> Right. I suspect that what you are finding is that those separate schemas
> use different namespaces, and if we combine them all we generate Python
> classes in a single namespace, and we have a mess.
>
>
> After many tests I decided the best way to fix this was to avoid this
>> altogether and process each imported XSD independently. As a result of
>> this
>> change it appeared necessary to have generateDS create a python module for
>> each XSD (or at least it was easier to do this).
>>
>
> Sounds like an approach that is hopeful, if you can work with all those
> generated files. And, if you have not done so already, you might put all
> those generated files into a single Python package (a directory containing
> a file named __init__.py) and in the __init__.py file, import each of your
> generated files, so that when someone imports the package/directoryitself,
> those generated modules will be available at the top level. I'm thinking
> about this because this subject (Python packages) is one that I covered in
> the beginning Python class I taught last week.
>
> I'm assuming that you are using the --no-process-includes command line
> option, right?
>
>
> Having one python module
>> per XSD is better for me anyways as this is how we do it for our java
>> code.
>>
>
> You're in a Java shop. Then, if you have not already done so, you should
> at least look at XMLBeans:
>
>
> https://en.wikipedia.org/wiki/**XMLBeans<https://en.wikipedia.org/wiki/XMLBeans>
> http://xmlbeans.apache.org/
>
> And, maybe the JAXB too:
>
>
> https://en.wikipedia.org/wiki/**Java_Architecture_for_XML_**Binding<https://en.wikipedia.org/wiki/Java_Architecture_for_XML_Binding>
>
> Roughly speaking, it does for Java what generateDS.py does for Python.
>
> And, although I have not tried it for several years, back when I did, the
> code and the jar files generated by one or perhaps both of the above were
> usable with Jython.
>
>
>
>> I now have a working version of this that works with the 100 or so XSD
>> files I use.
>>
>
> Wow. Super.
>
>
> I attempted to make this change without disrupting how
>> genereateDS works otherwise. In a previous email you mentioned there are
>> unit tests. Would you explain how to run these? Once I get those to pass
>> I'll check these in to my repo and submit a pull request.
>>
>>
> To run the unit tests, do this:
>
> $ cd generateDS/tests
> $ python test.py
>
> But, it's likely that many of the unit tests will fail. What most of them
> do is generate the Python code, and then do a (diff) comparison with the
> code generated by a previous version of generateDS.py. So, any change that
> you make which causes a difference in the code generated is likely to make
> some unit tests fail.
>
> When that happens, I run diff on the previous and current code generated,
> then do a visual inspection of the differences, and if I can convince
> myself that any differences are really what I want, then I copy the new
> generated file on top of (replace) the previous file. Since I use both the
> -o and the -s command line flags to generate superclasses and subclasses,
> and these output files have names like this (in the generateDS/tests
> directory):
>
> xxxx1_sup.py -- old superclass file
> xxxx2_sup.py -- new superclass file
> xxxx1_sub.py -- old subclass file
> xxxx2_sub.py -- new subclass file
>
> If you want to add a new unit test, in tests/test.py, take a look at
> function test_018_simpletypes_other, if you want to compare the generated
> Python bindings and take a look at test_014_xsi_type, if you want to
> compare the exported XML file.
>
> Hope this helps a bit. Let me know if I can answer any more questions.
>
> Dave
> --
> Dave Kuhlman
> http://www.davekuhlman.org
>
>
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
generateds-users mailing list
generateds-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/generateds-users
