Sanja,
In order to reproduce your non-ascii character error message, I
created a schema containing an attribute declaration with a name
containing a non-ascii character. Example:
<xs:attribute name="İstanbul" type="xs:string"
fixed="a fascinating city" use="required" />
Is that where your non-ascii characters were? Note that the İ in
İstanbul has a dot on top.
I believe that there already is a way to handle this with
generateDS.py, but it only seems to work with Python 3. You can try
using the --cleanup-name-list command line option. In your case it
might be:
./generateDS.py --cleanup-name-list="[('µ', 'm')]" -f -o tmp01sup.py
test05-01.xsd
As I said, it only seems to work with Python 3, not Python 2. I'll
look into fixing that tomorrow.
The module that you generate, by the way, should be usable with
either Python 2 or Python3.
Thanks for your kind words. And, I'm gratified that your group is
getting use out of generateds.
Dave
On Thu, Feb 22, 2018 at 12:01:14PM +0000, Sanja Abbott wrote:
> Hi Dave,
>
> please do not apologise, we really appreciate all your effort; our whole
> group uses your software and the fact you can be easily contacted and act
> promptly to our requests is really valuable to us.
>
> Thank you for sending me a new distribution file. It worked (bar for some
> characters, more bellow) and indeed has solved the previous problem and
> produces the requested fixed attribute correctly:
> def __init__(self, emdb_id=None, version='3.0.0.0', admin=None,
> crossreferences=None, sample=None, ...
>
> My last request is to enable the newest generateDS.py to translate non-ascii
> characters. My schema xsd file contains these:
> 1. µ ( u'\xb5') and
> 2. Å (u'\u212b')
>
> When I run my schema with these characters against generateDS.py I get,
> e.g., this error message:
>
> sanja-ml:generateDS-2.29.7 sanja$ python generateDS.py --root-element="emd"
> -f -o "emdb.py" --external-encoding='utf-8'
> ../workspaceSelenium/v3/DA_schema/emdb.xsd
> Traceback (most recent call last):
> File "generateDS.py", line 7438, in <module>
> main()
> File "generateDS.py", line 7426, in main
> superModule=superModule)
> File "generateDS.py", line 6914, in parseAndGenerate
> prefix, root, options, args, superModule)
> File "generateDS.py", line 6724, in generate
> generateFromTree(wrt, prefix, elements, processed)
> File "generateDS.py", line 6639, in generateFromTree
> generateClasses(wrt, prefix, element, 0, nameSpacesDef)
> File "generateDS.py", line 4994, in generateClasses
> generateCtor(wrt, prefix, element)
> File "generateDS.py", line 4133, in generateCtor
> wrt(' def __init__(self%s):\n' % s2)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u212b' in
> position 30: ordinal not in range(128)
>
> However, when I substitute 'µ' with 'm' and 'Å' with 'A' no errors were
> reported.
>
> Additionally, previous versions of generateDS could also translate non-ascii
> characters without a problem.
>
> I think the distribution file worked really well and I would give it a
> preference when adding new features and testing fixes in future.
>
> Thank you again.
> Best wishes,
> Sanja
>
> On 21/02/2018 21:13, Dave Kuhlman wrote:
> > Sanja,
> >
> > I apologize. I did not realize that another file
> > (process_includes.py), imported by generateDS.py, had also changed.
> > Actually, that new version of process_includes.py was at Bitbucket,
> > but there was no way for you to know that.
> >
> > So, I've created a new distribution file containing this fix. You
> > should be able to use it as follows:
> >
> > $ tar xf generateDS-2.29.7.tar.gz
> > $ cd generateDS-2.29.7
> > $ python generateDS.py -o tmp01sup.py ../test04-01.xsd
> > $ diff -u ../generateDS-2.29.5/tmp01sup.py tmp01sup.py
> >
> > By the way, there is no need to do an install in order to run the
> > above from within that directory.
> >
> > The last line above (diff -u ...) should produce the diff file that
> > I've attached (in a separate email).
> >
> > The above produces the change that we have been trying to achieve, I
> > believe. Please let me know about that.
> >
> > In a separate email, I've attached that new distribution file and
> > the diff file, too.
> >
> > Again, I apologize for wasting your cycles.
> >
> > I like to put off pushing a change to Bitbucket and pypi.python.org
> > until I have some confidence that we want to keep it. I need to
> > come up with a more reliable way to pass along a temporary change
> > for testing. Maybe this (producing a distribution file) is that
> > better way. Let me know what you think about that, too, please.
> >
> > Dave
> >
> > On Wed, Feb 21, 2018 at 03:47:41PM +0000, Sanja Abbott wrote:
> > > Hi Dave,
> > >
> > > Thank you for sending me generateDS.py (version 2.29.7), a schema for
> > > testing the fixed attributes (test04-01.xsd) and the output file
> > > (tmp07sup.py) you get by running this script.
> > >
> > > I have tried to replicate your results but I get this error:
> > >
> > > sanja-ml:generateDS-2.29.5 sanja$ python generateDS.py -o "test.py"
> > > ../../Desktop/test04-01.xsd
> > > Traceback (most recent call last):
> > > File "generateDS.py", line 7438, in <module>
> > > main()
> > > File "generateDS.py", line 7426, in main
> > > superModule=superModule)
> > > File "generateDS.py", line 6904, in parseAndGenerate
> > > no_redefine_groups=noRedefineGroups,
> > > TypeError: 'lxml.etree._ElementTree' object is not iterable
> > >
> > > To explain how I got there. I've downloaded your latest available package
> > > generateDS-2.29.5.tar.gz. Ran the following commands:
> > >
> > > tar xzvf generateDS-2.29.5.tar.gz
> > > cd generateDS-2.29.5
> > > python setup.py build
> > > python setup.py install
> > >
> > > Then I renamed generateDS.py to generateDS_original.py and copied
> > > generateDS.py version '2.29.7' into the generateDS-2.29.5 folder. You can
> > > see that the two generateDS scripts are in the same directory and they are
> > > different:
> > >
> > > sanja-ml:generateDS-2.29.5 sanja$ diff generateDS.py
> > > generateDS_original.py
> > > 232c232
> > > < VERSION = '2.29.7'
> > > ---
> > > > VERSION = '2.29.5'
> > > 264d263
> > > < SchemaNamespaceDict = {}
> > > 1434,1435c1433
> > > < default=None,
> > > < fixed=None):
> > > ---
> > > > default=None):
> > > 1440,1442d1437
> > > < # treat `fixed` the same as `default`.
> > > < if fixed is not None:
> > > < self.default = fixed
> > > 1702,1705d1696
> > > < if 'fixed' in attrs:
> > > < fixed = attrs['fixed']
> > > < else:
> > > < fixed = None
> > > 1708,1709c1699
> > > < attribute = XschemaAttribute(
> > > < name, data_type, use, default, fixed)
> > > ---
> > > > attribute = XschemaAttribute(name, data_type, use,
> > > default)
> > > 1713,1714c1703
> > > < attribute = XschemaAttribute(
> > > < name, data_type, use, default, fixed)
> > > ---
> > > > attribute = XschemaAttribute(name, data_type, use,
> > > default)
> > > 2824,2829d2812
> > > < ns_prefix = SchemaNamespaceDict.get(name)
> > > < if ns_prefix is not None and ns_prefix[0] is not None:
> > > < namespace = ns_prefix[0] + ':'
> > > < ns_def = 'xmlns:{}'.format(ns_prefix[0])
> > > < if ns_def not in nameSpacesDef:
> > > < nameSpacesDef += ' {}="{}"'.format(ns_def, ns_prefix[1])
> > > 4365c4348
> > > < pats2 = [u'^{}$'.format(replaceVbars(p1)) for p1 in pats1]
> > > ---
> > > > pats2 = ['^{}$'.format(replaceVbars(p1)) for p1 in pats1]
> > > 4561,4562d4543
> > > < # Check special case: simpletype that restricts
> > > xs:simpletype.
> > > < # Prevent infinite recursion.
> > > 6870,6871c6851
> > > < SchemaLxmlTree, ModuleSuffix, UseSourceFileAsModuleName, \
> > > < SchemaNamespaceDict
> > > ---
> > > > SchemaLxmlTree, ModuleSuffix, UseSourceFileAsModuleName
> > > 6898c6878
> > > < doc, SchemaNamespaceDict =
> > > process_includes.process_include_files(
> > > ---
> > > > doc = process_includes.process_include_files(
> > > 6935c6915
> > > < doc, SchemaNamespaceDict =
> > > process_includes.process_include_files(
> > > ---
> > > > doc = process_includes.process_include_files(
> > >
> > > If I ran generateDS_original.py on your schema (test04-01.xsd) and compare
> > > the result with the tmp07sup.py file you've provided I get:
> > >
> > > sanja-ml:generateDS-2.29.5 sanja$ python generateDS_original.py -o
> > > "test.py"
> > > ../../Desktop/test04-01.xsd
> > > sanja-ml:generateDS-2.29.5 sanja$ diff test.py ../../Desktop/tmp07sup.py
> > > 5,6c5,6
> > > < # Generated Wed Feb 21 15:32:42 2018 by generateDS.py version 2.29.5.
> > > < # Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 12:54:16) [GCC
> > > 4.2.1
> > > (Apple Inc. build 5666) (dot 3)]
> > > ---
> > > > # Generated Mon Feb 19 14:13:30 2018 by generateDS.py version 2.29.7.
> > > > # Python 3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0]
> > > 9c9,14
> > > < # ('-o', 'test.py')
> > > ---
> > > > # ('-f', '')
> > > > # ('-o', 'tmp07sup.py')
> > > > # ('-s', 'tmp07sub.py')
> > > > # ('--super', 'tmp07sup')
> > > > # ('--member-specs', 'dict')
> > > > # ('--export', 'write')
> > > 12c17
> > > < # ../../Desktop/test04-01.xsd
> > > ---
> > > > # test04-01.xsd
> > > 15c20
> > > < # generateDS_original.py -o "test.py" ../../Desktop/test04-01.xsd
> > > ---
> > > > # ./generateDS.py -f -o "tmp07sup.py" -s "tmp07sub.py"
> > > --super="tmp07sup" --member-specs="dict" --export="write" test04-01.xsd
> > > 18c23
> > > < # generateDS-2.29.5
> > > ---
> > > > # Test01
> > > 448c453
> > > < ExternalEncoding = 'ascii'
> > > ---
> > > > ExternalEncoding = 'utf-8'
> > > 718a724,728
> > > > member_data_items_ = {
> > > > 'garden_name': MemberSpec_('garden_name', 'xs:string', 0, 0,
> > > {'use': 'required'}),
> > > > 'version': MemberSpec_('version', 'xs:token', 0, 0, {'use':
> > > 'required'}),
> > > > 'vegetable': MemberSpec_('vegetable', 'xs:string', 1, 1,
> > > > {'name':
> > > 'vegetable', 'type': 'xs:string', 'minOccurs': '0', 'maxOccurs':
> > > 'unbounded'}, None),
> > > > }
> > > 721c731
> > > < def __init__(self, garden_name=None, version=None, vegetable=None):
> > > ---
> > > > def __init__(self, garden_name='dave\'s backyard',
> > > > version='3.0.0.0',
> > > vegetable=None):
> > > 778c788
> > > < if self.garden_name is not None and 'garden_name' not in
> > > already_processed:
> > > ---
> > > > if self.garden_name != "dave's backyard" and 'garden_name'
> > > > not in
> > > already_processed:
> > > 781c791
> > > < if self.version is not None and 'version' not in
> > > already_processed:
> > > ---
> > > > if self.version != "3.0.0.0" and 'version' not in
> > > already_processed:
> > > 924,925c934,935
> > > < sys.stdout.write('#from test import *\n\n')
> > > < sys.stdout.write('import test as model_\n\n')
> > > ---
> > > > sys.stdout.write('#from tmp07sup import *\n\n')
> > > > sys.stdout.write('import tmp07sup as model_\n\n')
> > > As expected, my test.py was created with the original generateDS script
> > > and
> > > it's different to yours, mine used version '2.29.5', yours the intended
> > > version '2.29.7'.
> > >
> > > However, if I repeat the same with your new generateDS.py version '2.29.7'
> > > script I get the following:
> > > sanja-ml:generateDS-2.29.5 sanja$ python generateDS.py -o "test.py"
> > > ../../Desktop/test04-01.xsd
> > > Traceback (most recent call last):
> > > File "generateDS.py", line 7438, in <module>
> > > main()
> > > File "generateDS.py", line 7426, in main
> > > superModule=superModule)
> > > File "generateDS.py", line 6904, in parseAndGenerate
> > > no_redefine_groups=noRedefineGroups,
> > > TypeError: 'lxml.etree._ElementTree' object is not iterable
> > >
> > > I'm not sure what I could do next. I would be very grateful if you could
> > > suggest what that might be.
> > >
> > > Thank you,
> > > Sanja
> > >
> > > On 19/02/2018 22:36, Dave Kuhlman wrote:
> > > > Sanja,
> > > >
> > > > If I understand you correctly, we are getting different results.
> > > >
> > > > I have the following in my test XML schema:
> > > >
> > > > <xs:attribute name="garden_name" type="xs:string"
> > > > fixed="dave's backyard" use="required" />
> > > > <xs:attribute name="version" type="xs:token" use="required"
> > > > fixed="3.0.0.0"/>
> > > >
> > > > And, the constructor that is generated by generateDS.py is the
> > > > following:
> > > >
> > > > def __init__(self, garden_name='dave\'s backyard',
> > > > version='3.0.0.0', vegetable=None):
> > > >
> > > > Am I right that this is what you want generated?
> > > >
> > > > If so, and if you are not getting that result, is it possible that
> > > > another version of generateDS.py on your machine is being used by
> > > > mistake.
> > > >
> > > > Or, maybe there is something that I don't understand correctly?
> > > >
> > > > I'll send the complete files (schema, generated module, and
> > > > generateDS.py) in a separate email so as not to put too much bulk in
> > > > the generateds-users email list.
> > > >
> > > > Let me know if there is something else I need to test.
> > > >
> > > > About, the generateds-users email list -- Yes, it would be good to
> > > > reply to or CC that list. For one thing, that gives me an archive
> > > > of comments, requests, fixes, etc. And, also, there may be other
> > > > users who are interested, have suggestions, etc.
> > > >
> > > > Thanks for help with this.
> > > >
> > > > Dave
> > > >
> > > > On Mon, Feb 19, 2018 at 12:30:20PM +0000, Sanja Abbott wrote:
> > > > > Dear Dave,
> > > > >
> > > > > thank you for sending me the fix for the issue of 'fixed values for
> > > > > attributes'. I've tested your solution and it seems it's not
> > > > > producing the
> > > > > result I expected.
> > > > >
> > > > > I'd like to have this attribute:
> > > > > <xs:attribute name="version" type="xs:token" use="required"
> > > > > fixed="3.0.0.0"/>
> > > > >
> > > > > For my purpose it's essential that the attribute is 'required'. If I
> > > > > remove
> > > > > use="required" and set default="3.0.0.0" (default can't be required):
> > > > > <xs:attribute name="version" type="xs:token" default="3.0.0.0"/>
> > > > >
> > > > > I get exactly what I need in the python script using generateDS.py:
> > > > >
> > > > > def __init__(self, emdb_id=None, version='3.0.0.0', admin=None,
> > > > > crossreferences=None, sample=None, ...
> > > > >
> > > > > However, the attribute I require creates:
> > > > > def __init__(self, emdb_id=None, version=None, admin=None,
> > > > > crossreferences=None, sample=None, ...
> > > > >
> > > > >
> > > > > Is there anything that can be done so that I can use the intended
> > > > > format for
> > > > > my attribute.
> > > > >
> > > > > Since it's my first time contacting you I wasn't sure if I should cc
> > > > > generateds-users as well. Please let me know.
> > > > >
> > > > > Thank you for your help,
> > > > > Sanja
> > > > >
> > > > >
> > > > > On 15/02/2018 21:20, Dave Kuhlman wrote:
> > > > > > Sanja,
> > > > > >
> > > > > > I've been thinking about this issue a bit more.
> > > > > >
> > > > > > From what I can learn, an attribute with a `fixed` value is the
> > > > > > same
> > > > > > as one with a `default` value except that a validating parser should
> > > > > > enforce the `fixed` restriction. That means that if the value of
> > > > > > that attribute is different from the value specified by `fixed` in
> > > > > > the schema, then the parser should report that the XML document does
> > > > > > not validate.
> > > > > >
> > > > > > Here is a bit of information, if you are interested:
> > > > > >
> > > > > > - https://www.w3schools.com/xml/el_attribute.asp
> > > > > >
> > > > > > - https://msdn.microsoft.com/en-us/library/ms256143(v=vs.110).aspx
> > > > > >
> > > > > > The parsers generated by generateDS.py are *not* validating parsers.
> > > > > > So, I believe that we're good by treating `fixed` attributes in the
> > > > > > same way as `default` attributes.
> > > > > >
> > > > > > With this change, you will be able to get that fixed value even
> > > > > > when the XML instance document omits it.
> > > > > >
> > > > > > If the user (you, for example) wants to validate an XML instance
> > > > > > document and wants to enforce the `fixed` restriction, then s/he can
> > > > > > run `xmllint` (or some other XML validation program) against the XML
> > > > > > document and the schema.
> > > > > >
> > > > > > I've attached a modified version of generateDS.py containing this
> > > > > > fix to a separate email.
> > > > > >
> > > > > > Let me know what you think and whether this solves the problem for
> > > > > > you.
> > > > > >
> > > > > > And, thank you for suggesting this enhancement.
> > > > > >
> > > > > > Dave
> > > > > >
> > > > > > On Fri, Feb 09, 2018 at 11:42:22AM +0000, Sanja Abbott wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > I wonder if you could implement 'fixed' values for attributes in
> > > > > > > your
> > > > > > > generateDS. My schema has an attribute called 'version':
> > > > > > >
> > > > > > > However, the python script that generateDS creates has no means
> > > > > > > to use this
> > > > > > > fixed value.
> > > > > > >
> > > > > > > Thank you,
> > > > > > > Sanja Abbott
> > > > > > >
> > > > > > > --
> > > > > > > _____________________________________________
> > > > > > >
> > > > > > > Sanja Abbott, PhD
> > > > > > > Bioinformatician and Software Infrastructure Manager
> > > > > > > Cellular Structures and 3D Bioimaging
> > > > > > > European Bioinformatics Institute
> > > > > > > Welcome Trust Genome Campus
> > > > > > > Hinxton
> > > > > > > Cambridge
> > > > > > > CB10 1SD
> > > > > > > UK
> > > > > > > http://www.ebi.ac.uk
> > > > > > > Tel: 01223 49 4300
> > > > > > >
> > > > > > > ------------------------------------------------------------------------------
> > > > > > > Check out the vibrant tech community on one of the world's most
> > > > > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > > > > > _______________________________________________
> > > > > > > generateds-users mailing list
> > > > > > > generateds-users@lists.sourceforge.net
> > > > > > > https://lists.sourceforge.net/lists/listinfo/generateds-users
> > > > > --
> > > > > _____________________________________________
> > > > >
> > > > > Sanja Abbott, PhD
> > > > > Bioinformatician and Software Infrastructure Manager
> > > > > Cellular Structures and 3D Bioimaging
> > > > > European Bioinformatics Institute
> > > > > Welcome Trust Genome Campus
> > > > > Hinxton
> > > > > Cambridge
> > > > > CB10 1SD
> > > > > UK
> > > > > http://www.ebi.ac.uk
> > > > > Tel: 01223 49 4300
> > > > >
> > > --
> > > _____________________________________________
> > >
> > > Sanja Abbott, PhD
> > > Bioinformatician and Software Infrastructure Manager
> > > Cellular Structures and 3D Bioimaging
> > > European Bioinformatics Institute
> > > Welcome Trust Genome Campus
> > > Hinxton
> > > Cambridge
> > > CB10 1SD
> > > UK
> > > http://www.ebi.ac.uk
> > > Tel: 01223 49 4300
> > >
>
> --
> _____________________________________________
>
> Sanja Abbott, PhD
> Bioinformatician and Software Infrastructure Manager
> Cellular Structures and 3D Bioimaging
> European Bioinformatics Institute
> Welcome Trust Genome Campus
> Hinxton
> Cambridge
> CB10 1SD
> UK
> http://www.ebi.ac.uk
> Tel: 01223 49 4300
>
--
Dave Kuhlman
http://www.davekuhlman.org
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
generateds-users mailing list
generateds-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/generateds-users
