Bernd, I agree with you about that name change. The existing name is misleading and confusing. I've followed your suggestion and changed `namespace_` to `namespaceprefix_`.
Hope that change does not cause too many problems in your code. But, then, as we say, you asked for it. Thank you for motivating me to make this change. I needed a nudge. Please let me know if you find any errors introduced by this change. The changes to the source were a bit messy, but I'm pretty sure that I got them all. And, thank you for working with me on this. I appreciate the help. Dave On Mon, Aug 27, 2018 at 08:04:53AM +0000, Zimmermann, Bernd wrote: > Hello Dave, > > thank you very much for your explanations. > > First of all, my code is working correct for me, as I am only using name_ and > namespacedef_. > I just came over these issues, trying out all options ... but finally I see > with your additional explanations, > that it is all related to some kind of misunderstanding 😉 or mix up between > "namespace" and "(namespace)prefix". > > I suggest to change the parameter name namespace_ to for example nameprefix_ > or namespaceprefix_ ... > > because: > > XML Namespaces - The xmlns Attribute > When using prefixes in XML, a namespace for the prefix must be defined. > The namespace can be defined by an xmlns attribute in the start tag of an > element. > The namespace declaration has the following syntax. xmlns:prefix="URI". > > So: > > <abc:tagname xmlns:abc="http://www.somewhere.com/abc"; ... > \___/\_____/ \______________________________________/ > ^ ^ ^ > | | | > | | +------------- namespacedef_ > | | > | +------------------------ name_ > | > +------------------------------ namespacePREFIX_ or namePREFIX_ > > with: > > abc: = PREFIX > > Together with this explanation the code is fully correct with a little > change just in the variable name: > > outfile.write('<%s%s%s' % (namespacePREFIX_, name_, namespacedef_ and ' ' + > namespacedef_ or '', )) > > Using the variable name namespacePREFIX_ just makes it more readable what to > print out 😉 > > Best wishes, > Bernd > > -----Ursprüngliche Nachricht----- > Von: Dave Kuhlman <dkuhl...@davekuhlman.org> > Gesendet: Freitag, 24. August 2018 22:20 > An: Zimmermann, Bernd <bernd.zimmerm...@qsc.de> > Cc: generateds-users <generateds-users@lists.sourceforge.net> > Betreff: Re: [Generateds-users] validate_XXX_patterns generated wrong from > XSD and Problems with export and namspaces > > Bernd, > > I apologize for taking up so much of your time on this. Basically, I'm > saying that I see no reason to change this code and the code that it > generates. The existing code enables you to insert a namespace prefix before > that tag name and a namespace declaration after the tag name. If there is > something additional that you need to be able to do, you will have to explain > that to me. > > What follows is *too* many details. I'm trying to clarify and explain this > to myself. Perhaps it will help you, too. > > Let's use some ascii-art to make sure whether we are in agreement on what the > existing generated code does: > > <abc:tagname xmlns:abc="http://www.somewhere.com/abc"; ... > \___/\_____/ \______________________________________/ > ^ ^ ^ > | | | > | | +------------- namespacedef_ > | | > | +------------------------ name_ > | > +------------------------------ namespace_ > > So, `namespace_` goes in front of (to the left of) the tag name and > `namespacedef_` is inserted after the tag name. > > And, if there is an importable module named `generatedsnamespaces.py` and if > the dictionary named "GenerateDSNamespaceDefs" in that module contains the > tag name as a key, then the value of that key overrides the value of > `namespacedef_`. > > Keep in mind that `namespace_` and `namespacedef_` hold, if not empty, two > different kinds of string: (1) a namespace prefix with colon (`namespace_`) > and a namespace definition/declaration (`namespacedef_`). > > The question that I cannot answer is: Why does the above not work for you? > > And, some questions: > > 1. Are you saying that the generated code (unmodified) is doing the > wrong thing? > > I've created a test and it seems to do what I expect it to do. > Specifically, (1) `namespace_` goes in front of the tag name; > (2) `namespacedef_` goes after the tag name; (3) the value > associated with a given tag name in generatedsnamespaces.py, if > any, overrides the value of `namespacedef_` passed as a parameter. > > 2. Or, are you saying that when *you* write your own code that calls > the export method, it behaves incorrectly? > > 3. And, if #2 above is true, then is it possible that you are either > calling it incorrectly or that you could change your code to get > the result you need? > > > OUTPUT with testxml.export(output, 0, name_='retainedDataMessageX', > > namespace_='urks="http://uri.etsi.org/02657/v1.18.1#/RetainedData";' > > gives: > > <urks="http://uri.etsi.org/02657/v1.18.1#/RetainedData"retainedDataMes > > sageX> > > Should be written as: > > testxml.export(output, 0, name_='retainedDataMessageX', > namespacedef_='urks="http://uri.etsi.org/02657/v1.18.1#/RetainedData";' > > In other words, change "namespace_=" to "namespacedef_=". Is there some > reason why you cannot do this or do not want to do this? > > What are you trying to accomplish that cannot be done with the currently > generated code? > > It seems that I am still missing the point. > > Note that: > > - `namespace_` should have a value that looks something like this: "abc:". > > - `namespacedef_` should have a value that looks something like this: > > xmlns:bark="http://www.animals.com/bark"; > xmlns:bark="http://www.animals.com/bark > xmlns:woof="http://www.animals.com/woof"; > > > So I suggest to write out only 2 parameters here: > > > > outfile.write('<%s%s' % (name_, namespaceOut_'', )) > > > > an write a logic strategy before the write to set namespaceOut_ to: > > > > 1. namespace_ if given > > 2. namespaceDef_ if given and namespace_ is not given > > > > So I suggest a change here, for example: > > imported_ns_def_ = > > GenerateDSNamespaceDefs_.get('RetainedDataMessage') > > if imported_ns_def_ is not None: > > namespaceOut_ = imported_ns_def_ > > if namespace_ is not None: > > namespaceOut_ = namespace_ > > No. `namespace_` and `namespacedef_` hold two different *kinds* of > things: > > - `namespace_` should contain a namespace *prefix* followed by a > colon, for example: "abc:". > > - `namespacedef_` should contain a namespace definition, for > example: 'xmlns:aaa="http://www.aaa.com";'. > > - `imported_ns_def_`, if not None, will contain a namespace > definition. > > And, if we use your logic and suggested change, there is no way to insert a > namespace prefix in front of the tag name. > > Hope this helps. But, if there is something that I can help you accomplish, > please let me know. I'll need to know what you are trying to accomplish. > > Dave > > On Wed, Aug 22, 2018 at 02:04:35PM +0000, Zimmermann, Bernd wrote: > > Hello Dave, > > > > thank you for your hints with the explicit parameter, but this is not the > > real problem. > > The parameter_ is only useful if you want to change the order of the > > parameters. > > > > Let me show you a few more examples: > > > > OUTPUT with testxml.export(output, 0, name_='retainedDataMessageX', > > namespace_='urks="http://uri.etsi.org/02657/v1.18.1#/RetainedData";' > > gives: > > <urks="http://uri.etsi.org/02657/v1.18.1#/RetainedData"retainedDataMes > > sageX> > > > > (Note that in this example I give no namespacedef_ not a default value voa > > GenerateDSNamespaceDefs. > > > > As you can see: name_ is printed after the namespace_ Parameter because: > > > > outfile.write('<%s%s%s' % (namespace_, name_, namespacedef_ and > > ' ' + namespacedef_ or '', )) > > > > here the namespace_ is the first argument, but it should name_ > > > > So I suggest to write out only 2 parameters here: > > > > outfile.write('<%s%s' % (name_, namespaceOut_'', )) > > > > an write a logic strategy before the write to set namespaceOut_ to: > > > > 1. namespace_ if given > > 2. namespaceDef_ if given and namespace_ is not given > > > > So I suggest a change here, for example: > > imported_ns_def_ = > > GenerateDSNamespaceDefs_.get('RetainedDataMessage') > > if imported_ns_def_ is not None: > > namespaceOut_ = imported_ns_def_ > > if namespace_ is not None: > > namespaceOut_ = namespace_ > > > > In the last if perhaps one can test if namespace is '' too, if given via > > cmdline but might by ignored. > > > > Can you guess what I am suggesting? > > > > The correction of the documentation is useful, but is not the solver > > for the above problem 😉 - I already set it the right way ... > > > > GenerateDSNamespaceDefs = { > > "RetainedDataMessage": > > 'xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData";', > > } > > > > Best regards, > > Bernd > > > > -----Ursprüngliche Nachricht----- > > Von: Dave Kuhlman <dkuhl...@davekuhlman.org> > > Gesendet: Freitag, 10. August 2018 22:26 > > An: Zimmermann, Bernd <bernd.zimmerm...@qsc.de> > > Cc: generateds-users <generateds-users@lists.sourceforge.net> > > Betreff: Re: [Generateds-users] validate_XXX_patterns generated wrong > > from XSD and Problems with export and namspaces > > > > Bernd, > > > > Thank you for your notes on this issue. > > > > I'm still trying to think this one through. I'm still a bit confused about > > it. > > > > However, here is one item that may help. The `namespace_` parameter is > > mis-named I suppose; its value should be a namespace prefix and a colon. > > That value should be something like this: "abc:". > > > > If you were to pass something like "xyz:" as the value of the `namespace_` > > parameter, would that fix one of your issues? > > > > By the way: I believe that the documentation included near the top of a > > generated module, is not correct. It incorrectly reads: > > > > # A sample table is: > > # > > # # File: generatedsnamespaces.py > > # > > # GenerateDSNamespaceDefs = { > > # "ElementtypeA": "http://www.xxx.com/namespaceA";, > > # "ElementtypeB": "http://www.xxx.com/namespaceB";, > > # } > > > > whereas, it should be: > > > > # A sample table is: > > # > > # # File: generatedsnamespaces.py > > # > > # GenerateDSNamespaceDefs = { > > # "ElementtypeA": 'xmlns:nsa="http://www.xxx.com/namespaceA";', > > # "ElementtypeB": 'xmlns:nsb="http://www.xxx.com/namespaceB";', > > # } > > > > With respect to your issue 2: > > > > > 2.) > > > OUTPUT with testxml.export(output, 0, > > > 'xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData";') > > > results in: > > > <xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData"RetainedData > > > Me ssage xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData";> > > > </xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData"RetainedDat > > > aM > > > essage> > > > > > > This is not correct. > > > > I believe that the namespace definition that you are passing in as the 2nd > > parameter is getting attached to the namespace_ parameter, whereas you > > actually want to set the namespacedef_ parameter. You might consider using > > a keyword argument to make this explicit. > > Example: > > > > testxml.export( > > output, > > 0, > > > > namespacedef_='xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData";' > > ) > > > > While trying to straighten my head out on this, I wrote up the following > > notes on the parameters to the `export` methods. > > > > Arguments to the generated export method: > > > > - outfile -- A file like object open for writing. > > > > - level -- the indentation level. If the pretty_print argument is > > True, the (generated) function showIndent is used to prefix each > > exported line with 4 spaces for each level of indent. > > > > - namespace_ -- An empty string or an XML namespace prefix plus a > > colon, example "abc:". This value is printed immediately in front > > of the tag name. > > > > - name_ -- The element tag name. Note that the tag name can be > > overridden by the original_tagname_, which can be set by the class > > constructor. > > > > - namespacedef_ -- Zero or more namespace prefix definitions. > > Actually, its value can be any attribute-value pairs. > > Examples: > > > > '' > > 'xmlns:abc="http://www.abc.com"http://www.def.com"; > > 'xmlns:abc="http://www.abc.com xmlns:def="http://www.def.com"; > > > > or, because it is printed where the attributes occur, even: > > > > 'size="25" color="blue"' > > > > For more on namespacedef_, see: > > > > http://www.davekuhlman.org/generateDS.html#namespaces-inserting-namesp > > ace-definition-in-exported-documents > > > > - pretty_print -- If True, exported output is printed with > > indentation and newlines. If False, indentation and newlines are > > omitted, which produces a more compact representation. > > > > I've added the above notes to the generateDS document. See: > > http://www.davekuhlman.org/generateDS.html#method-export > > > > Sigh. XML namespaces make my head hurt. > > > > I'll study this and your notes a bit more. There must be *some* way we can > > make sense of this and get it to work correctly. > > > > Hope your weekend is (or was) a good one. > > > > Dave > > > > On Thu, Aug 09, 2018 at 10:14:57AM +0000, Zimmermann, Bernd wrote: > > > Hello Dave, > > > > > > thank you very much for the patch for the validate patterns, it > > > works > > > 😉 so it could go productive on your next release. > > > > > > Just a few additional remarks on the export issue: > > > > > > Code: def export(self, outfile, level, namespace_='', > > > name_='RetainedDataMessage', namespacedef_='', pretty_print=True): > > > There are three possible parameters: namespace_, name_, > > > namespacedef_ > > > > > > The line: outfile.write('<%s%s%s' % (namespace_, name_, > > > namespacedef_ and ' ' + namespacedef_ or '', )) will print: > > > <namespace_ , name_, namespacedef_> if all three values are set. > > > Note: namespacedef_ may be set via GenerateDSNamespaceDefs option. > > > > > > Here is just a first problem, because the namespace must be a second > > > value in the xml tag <OBJECT_NAME NAMESPACE> </OBJECT_NAME> in a > > > valid xml, not the first. > > > > > > TESTing: > > > 1.) > > > using: testxml.export(output, 0, '', 'retainedDataMessage') results > > > in: > > > <retainedDataMessage > > > xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData";> > > > ... > > > </retainedDataMessage> > > > > > > this ist Ok. > > > Note that GenerateDSNamespaceDefs is set in my test case. > > > Parameter namespace_ is empty. > > > Parameter name_ is set > > > Parameter namespacedef_ is not used but overwritten by: > > > imported_ns_def_ = GenerateDSNamespaceDefs_.get('RetainedDataMessage') > > > if imported_ns_def_ is not None: > > > namespacedef_ = imported_ns_def_ > > > > > > 2.) > > > OUTPUT with testxml.export(output, 0, > > > 'xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData";') > > > results in: > > > <xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData"RetainedData > > > Me ssage xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData";> > > > </xmlns="http://uri.etsi.org/02657/v1.18.1#/RetainedData"RetainedDat > > > aM > > > essage> > > > > > > This is not correct. > > > There is the problem as stated above that the namespace_ is set und > > > printed out at first in the opening tag before the name_ > > > > > > The logical problem is: The parameter namespace_ is set (the first > > > one) and there is an GenerateDSNamespaceDefs setting, so there is a > > > logic missing, to decide which parameter to use in the export if > > > there are 2 possibilities. My suggestion would be to always use > > > namespacedef_ it set and use the value via GenerateDSNamespaceDefs > > > setting only if namespacedef_ parameter is not used or set. > > > > > > So for me it looks like to adjust the lines with outfile.write('< > > > and outfile.write('> to match an valid XML form like: > > > <OBJECT_NAME NAMESPACE> > > > </OBJECT_NAME> > > > > > > Hope you guess what I mean. > > > > > > Best wishes, > > > Bernd > > > > > > -----Ursprüngliche Nachricht----- > > > Von: Dave Kuhlman [mailto:dkuhl...@davekuhlman.org] > > > Gesendet: Dienstag, 24. Juli 2018 23:03 > > > An: Zimmermann, Bernd <bernd.zimmerm...@qsc.de> > > > Cc: generateds-users <generateds-users@lists.sourceforge.net> > > > Betreff: Re: [Generateds-users] validate_XXX_patterns generated > > > wrong from XSD and Problems with export and namspaces > > > > > > Bernd, > > > > > > With respect to the validation of time and the > > > validate_GeneralizedTime_patterns_, I've attached a patch. With this > > > patch, I believe that generateDS.py produces the pattern that works for > > > you. These regular expressions are complicated enough, and I'm in the > > > dark about what they should and should not be checking. So, if this > > > patch works for you, let's go with it. We can make adjustments when > > > someone else reports a problem with it. > > > > > > Here is a little explanation, as far as I can understand it: This patch > > > removes the code that was replacing "|" with "$|^". That replacement was > > > intended to enable us to handle alternatives in regular expressions by > > > breaking them up into several concatenated expressions. *But*, maybe the > > > ability to handle those alternatives is (or should be) written into the > > > regular expression itself, *if* the author of the XML schema writes them > > > correctly. If so, the replacement that we are removing is not needed, > > > and this patch is the right thing to do. At least, that's what I'm > > > hoping. > > > > > > About your other issue, I'll take a look tomorrow. But, at first glance, > > > it looks like the values you are passing in to the export method are > > > being attached to the wrong arguments. You are passing positional > > > arguments. You could try using the keyword arguments: > > > > > > namespace_ > > > name_ > > > namespacedef_ > > > > > > You will have to look at one of the export methods to see how each of > > > those parameters is used. > > > > > > By the way, your suggestion in your latest email that a module generated > > > by generateDS.py be able to do an automatic validation of the exported > > > XML instance document seems like a good idea. Although adding something > > > like this line: > > > > > > xmllint --schema test.xsd test01.xml > > > > > > to a batch file does seem like an easy enough way to handle that. > > > So, I'll put that feature on a "do" list, but it might not be at the very > > > top of the list. > > > > > > More later. Thanks again for your help with these issues. > > > > > > Dave > > > > > > On Thu, Jul 19, 2018 at 12:19:52PM +0000, Zimmermann, Bernd wrote: > > > > Hello Dave, > > > > > > > > very great stuff you've done with generateDS, works like a charm but > > > > only after a resolving a few little things. > > > > > > > > First on, this might be a bug: > > > > > > > > XSD Pattern: > > > > > > > > <xsd:simpleType name="GeneralizedTime"> > > > > <xsd:restriction base="xsd:string"> > > > > <xsd:pattern > > > > value="\d{4}(0[1-9]|1[012])(0[1-9]|[12][0-9]|3[01])([01][0-9]|2[0-3])([0-5][0-9])([0-5][0-9])(.[0-9][0-9]*)?(Z|[\+|\-][0-9]{4})"/> > > > > </xsd:restriction> > > > > </xsd:simpleType> > > > > > > > > results in: > > > > > > > > > > > > validate_GeneralizedTime_patterns_ = > > > > [['^\\d{4}(0[1-9]$|^1[012])(0[1-9]$|^[12][0-9]$|^3[01])([01][0-9]$ > > > > |^ > > > > 2[ > > > > 0-3])([0-5][0-9])([0-5][0-9])(.[0-9][0-9]*)?(Z$|^[\\+$|^\\-][0-9]{ > > > > 4} > > > > )$ > > > > ']] > > > > > > > > which give the warning: > > > > > > > > > > > > /var/etsi/work/V1.18.1/Natparas2V18.py:7644: UserWarning: Value > > > > "b'20180719123801+0200'" does not match xsd pattern > > > > restrictions: > > > > [['^\\d{4}(0[1-9]$|^1[012])(0[1-9]$|^[12][0-9]$|^3[01])([01][0-9]$ > > > > |^ > > > > 2[ > > > > 0-3])([0-5][0-9])([0-5][0-9])(.[0-9][0-9]*)?(Z$|^[\\+$|^\\-][0-9]{ > > > > 4} )$ ']] warnings_.warn('Value "%s" does not match xsd pattern > > > > restrictions: %s' % (value.encode('utf-8'), > > > > self.validate_GeneralizedTime_patterns_, )) > > > > > > > > Adjusting to: > > > > > > > > > > > > validate_GeneralizedTime_patterns_ = > > > > [['^\\d{4}(0[1-9]|1[012])(0[1-9]|[12][0-9]|3[01])([01][0-9]|2[0-3] > > > > )( > > > > [0 -5][0-9])([0-5][0-9])(.[0-9][0-9]*)?(Z|[\\+|\\-][0-9]{4})$']] > > > > > > > > gives no warning. > > > > > > > > So it might be a problem with the separator | which results in a wrong > > > > ^|$ pattern? > > > > > > > > > > > > Second one, discovering a problem with: > > > > > > > > testxml.export(sys.stdout, 0, '', 'retainedDataMessage') > > > > at: > > > > outfile.write('<%s%s%s' % (namespace_, name_, namespacedef_ and ' ' > > > > + namespacedef_ or '', )) > > > > > > > > This works correct, printing out <retainedDataMessage> or with a > > > > generatednamespaces.py with GenerateDSNamespaceDefs the > > > > <retainedDataMessage NSGIVEN> > > > > > > > > But: > > > > > > > > testxml.export(sys.stdout, 0, 'NSDATA', 'retainedDataMessage') > > > > > > > > gives: <NSDATAretainedDataMessage> which is not correct, it should > > > > be <retainedDataMessage NSDATA>, so it's printed in the wrong order and > > > > a space is missing between the two arguments. > > > > > > > > Perhaps a: > > > > outfile.write('<%s%s%s' % (name_, namespace_ and ' ' + namespace_ > > > > or '', namespacedef_ and ' ' + namespacedef_ or '', )) > > > > > > > > could fix this. Note: the closing tag must be adjusted too. > > > > > > > > Perhaps the export could also be adjusted to overwrite the > > > > namespacedef_ if a namespace_ is given or vice versa ... > > > > > > > > I am using the latest version 2.29.17 installed via pip under > > > > Python > > > > 3.7 > > > > > > > > Hope you could fix it. > > > > > > > > Again, great work! > > > > > > > > Best regards, > > > > Bernd > > > > > > > > > > > ------------------------------------------------------------------ > > > > -- > > > > -- > > > > -------- Check out the vibrant tech community on one of the > > > > world's most engaging tech sites, Slashdot.org! > > > > http://sdm.link/slashdot > > > > > > > _______________________________________________ > > > > generateds-users mailing list > > > > generateds-users@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/generateds-users > > > > > > > > > > > > -- > > > > > > Dave Kuhlman > > > http://www.davekuhlman.org > > > > -- > > > > Dave Kuhlman > > http://www.davekuhlman.org > > -- > > Dave Kuhlman > http://www.davekuhlman.org -- Dave Kuhlman http://www.davekuhlman.org ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ generateds-users mailing list generateds-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/generateds-users
