On Mon, Oct 24, 2016 at 08:37:06PM +0200, "Peter “Peter, Berlin” Rosemann"
wrote:
> Hello Dave,
>
> could it be that this is the reason why its not possible to process fpml
> (from main onwards) when I use " --one-file-per-xsd “?
> I use generateDS same way as you do in your GitHub repository, one by one at
> right now.
Peter,
I don't know whether those two problems are related.
Just now, I downloaded the Fpml schemas and tried to generate code
from them.
Here is what I did:
$ wget http://www.fpml.org/spec/fpml-5-9-3-wd-3/xml/confirmation-5-9_xml.zip
$ unzip confirmation-5-9_xml.zip
$ cd confirmation
$ generateds/generateDS.py -o fpmllib.py fpml-main-5-9.xsd
That caused an exception in the code that I added for the fix for
Pietro's issue.
So, I've added a new command line option "--remove-duplicate-child-elements".
The default for that option is False, which means that the new code is skipped.
Then, I can run:
$ generateds/generateDS.py -o fpmllib.py fpml-main-5-9.xsd
Now, it successfully generated a module. I can import that module in
Python, so it's syntactically correct. But, I do not know whether
that module could parse and generate Fpml XML instance documents.
More on that later. There are instance XML files in that download.
Perhaps I can try some test.
A patch file with support for the new command line option is
attached.
Dave
>
> Regards
>
> Peter
>
> p.s.
>
> Love your tool! Saves so much time! Thank you!
>
>
> > On 23 Oct 2016, at 22:56, Dave Kuhlman <dkuhl...@davekuhlman.org> wrote:
> >
> > On Sun, Oct 23, 2016 at 05:30:55PM +0200, Pietro Saccardi wrote:
> >> Hello Dave,
> >>
> >> my name is Pietro. I recently started using generateDS--great job! it's
> >> really useful and works seamlessly out of the box.
> >
> > Pietro,
> >
> > Great.
> >
> >>
> >> While testing, I encountered a bug. I wanted to file it to sourceforce, but
> >> I am unable to create tickets. I am also not very used to the platform so I
> >> may have missed something.
> >
> > Not so great.
> >
> > Thanks for reporting this. I'm going to be away traveling for most
> > of this week. I've put this on my To-do list, and will look at it when
> > I get back.
> >
> > My recollection is that generateDS.py does not distinguish very
> > carefully between <xsd:sequence> and <xsd:choice>. So, this may be
> > a pretty deep problem. But, at the least, generateDS.py should not
> > generate the silly code that your example shows.
> >
> > More later.
> >
> > Dave
> >
> >>
> >> Anyway, I'll describe the bug.
> >>
> >> Consider such a schema:
> >>
> >> <xsd:element name="authors">
> >> <xsd:complexType>
> >> <xsd:sequence>
> >> <xsd:element ref="author"/>
> >> <xsd:choice maxOccurs="unbounded" minOccurs="0">
> >> <xsd:element ref="author"/>
> >> <xsd:element name="cooperation" type="xsd:string"/>
> >> </xsd:choice>
> >> </xsd:sequence>
> >> </xsd:complexType>
> >> </xsd:element>
> >>
> >> This will require the <authors> element to have a first child of type
> >> <author>, followed by zero or more instances of the elements <author> or
> >> <cooperation>.
> >>
> >> Such schema is not handled correctly in generateDS; it will generate
> >> ambiguity between the unique, initial author and the list of author
> >> objects. This is an excerpt of the buildChildren method of the authors
> >> class:
> >>
> >> def buildChildren(self, child_, node, nodeName_, fromsubclass_=False):
> >> if nodeName_ == 'author':
> >> obj_ = person.factory()
> >> obj_.build(child_)
> >> self.author = obj_
> >> obj_.original_tagname_ = 'author'
> >> elif nodeName_ == 'author':
> >> obj_ = person.factory()
> >> obj_.build(child_)
> >> self.author.append(obj_)
> >> obj_.original_tagname_ = 'author'
> >> elif nodeName_ == 'cooperation':
> >> cooperation_ = child_.text
> >> cooperation_ = self.gds_validate_string(cooperation_, node,
> >> 'cooperation')
> >> self.cooperation.append(cooperation_)
> >>
> >> As you can see, the second elif will never be reached and any further
> >> <author> will override the author property, losing the remaining elements
> >> in the list. Also, the exportChildren method will crash, because it will
> >> try to enumerate the author property of authors.
> >>
> >> I changed the structure of my XSD to work around this problem, but I think
> >> this should either be remarked in the documentation or fixed.
> >>
> >> I hope this can be of any use; thank you for your great work on generateDS!
> >>
> >> Best regards,
> >>
> >> Pietro Saccardi
> >
> > --
> >
> > Dave Kuhlman
> > http://www.davekuhlman.org
> >
> > ------------------------------------------------------------------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > _______________________________________________
> > generateds-users mailing list
> > generateds-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/generateds-users
>
--
Dave Kuhlman
http://www.davekuhlman.org
diff -r b7ccc0e3579a generateDS.py
--- a/generateDS.py Thu Oct 13 15:31:43 2016 -0700
+++ b/generateDS.py Wed Nov 02 14:53:10 2016 -0700
@@ -153,7 +153,6 @@
import os.path
import time
import getopt
-from pprint import pprint
if sys.version_info.major == 2:
import urllib2
@@ -276,6 +275,7 @@
OutputDirectory = None
ModuleSuffix = ""
PreserveCdataTags = False
+RemoveDuplicateChildElements = False
SchemaToPythonTypeMap = {}
@@ -918,7 +918,12 @@
self.expandGroupReferences_tree(visited)
self.collect_element_dict()
self.annotate_find_type()
- self.annotate_tree()
+ element_dict = {}
+ to_be_removed = []
+ self.annotate_tree(
+ element_dict=element_dict,
+ to_be_removed=to_be_removed)
+ self.remove_children(to_be_removed)
self.fix_dup_names()
self.coerce_attr_types()
self.checkMixedBases()
@@ -1153,7 +1158,19 @@
for child in self.children:
child.annotate_find_type()
- def annotate_tree(self):
+ def annotate_tree(self, element_dict, to_be_removed):
+ if RemoveDuplicateChildElements:
+ name = self.getName()
+ if name in element_dict:
+ # If we've already seen this element (name), make it a
+ # list and throw the previous one away.
+ self.maxOccurs = 2
+ to_be_removed.append(element_dict[name])
+ err_msg(
+ '\n*** warning. Removing child with duplicate '
+ 'name: "{}"\n\n'.format(name))
+ else:
+ element_dict[name] = self
# If there is a namespace, replace it with an underscore.
if self.base:
self.base = strip_namespace(self.base)
@@ -1193,7 +1210,6 @@
sys.exit(1)
self.minOccurs = minOccurs
self.maxOccurs = maxOccurs
-
# If it does not have a type, then make the type the same as the name.
if self.type == 'NoneType' and self.name:
self.type = self.name
@@ -1212,8 +1228,17 @@
parent.collapseWhiteSpace):
self.collapseWhiteSpace = 1
# Do it recursively for all descendents.
+ element_dict = {}
+ to_be_removed = []
for child in self.children:
- child.annotate_tree()
+ child.annotate_tree(
+ element_dict=element_dict,
+ to_be_removed=to_be_removed)
+ self.remove_children(to_be_removed)
+
+ def remove_children(self, to_be_removed):
+ for element in to_be_removed:
+ self.children.remove(element)
#
# For each name in the attributeGroupNameList for this element,
@@ -6882,7 +6907,8 @@
ExportWrite, ExportEtree, ExportLiteral, \
FixTypeNames, SingleFileOutput, OutputDirectory, \
ModuleSuffix, UseOldSimpleTypeValidators, \
- PreserveCdataTags, CleanupNameList
+ PreserveCdataTags, CleanupNameList, \
+ RemoveDuplicateChildElements
outputText = True
args = sys.argv[1:]
try:
@@ -6900,6 +6926,7 @@
'one-file-per-xsd', 'output-directory=',
'module-suffix=', 'use-old-simpletype-validators',
'preserve-cdata-tags', 'cleanup-name-list=',
+ 'remove-duplicate-child-elements',
])
except getopt.GetoptError:
usage()
@@ -7084,6 +7111,8 @@
PreserveCdataTags = True
elif option[0] == '--cleanup-name-list':
CleanupNameList = capture_cleanup_name_list(option[1])
+ elif option[0] == '--remove-duplicate-child-elements':
+ RemoveDuplicateChildElements = True
if showVersion:
print('generateDS.py version %s' % VERSION)
sys.exit(0)
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
generateds-users mailing list
generateds-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/generateds-users
