[issue462710] Python 2.2a3/XML leaks memory

[admin]

Change by admin :


--
github: None -> 35197

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

First release of python-escpos-xml

[Cédric Krier]

I'm pleased to announce the initial release of python-escpos-xml [1].

escpos-xml is a library to parse XML defined receipt and print it on ESC/POS 
Printer using python-escpos [2] library.
The XML syntax is quite similar to HTML and it can be generated by template 
engines.


[1] https://pypi.python.org/pypi/python-escpos-xml/0.1.0
[2] https://pypi.python.org/pypi/python-escpos
-- 
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

python and xml

[Nibin V M]

Hi,

I am trying to use cPanel XML-API and every API call return data in XML
format. I would like to know how to manipulate the data here.

For eg: How can I read the CPU load data from the below output


 loadavgone0.00/onefive0.00/fivefifteen0.00/fifteen/loadavg!--
whostmgrd --

Thank you,
-- 
Regards

Nibin.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python and xml

[Stefan Behnel]

Nibin V M, 16.05.2012 16:16:
 I am trying to use cPanel XML-API and every API call return data in XML
 format. I would like to know how to manipulate the data here.
 
 For eg: How can I read the CPU load data from the below output
 
 
  
 loadavgone0.00/onefive0.00/fivefifteen0.00/fifteen/loadavg!--
 whostmgrd --

Here's some untested code to print the text values:

  import xml.etree.ElementTree as ET

  loadavg = ET.fromstring('loadavgone0.00/onefive0.00/five'
  'fifteen0.00/fifteen/loadavg')

  for interval_tag in ('one', 'five', 'fifteen'):
  print(loadavg.findtext(interval_tag))


Stefan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python and xml

[Nibin V M]

thank you Stefan. but the XML output is assigned to a variable; how to
process the variable with XML contents?

On Wed, May 16, 2012 at 7:53 PM, Stefan Behnel stefan...@behnel.de wrote:

 Nibin V M, 16.05.2012 16:16:
  I am trying to use cPanel XML-API and every API call return data in XML
  format. I would like to know how to manipulate the data here.
 
  For eg: How can I read the CPU load data from the below output
 
 
 
  
 loadavgone0.00/onefive0.00/fivefifteen0.00/fifteen/loadavg!--
  whostmgrd --

 Here's some untested code to print the text values:

  import xml.etree.ElementTree as ET

  loadavg = ET.fromstring('loadavgone0.00/onefive0.00/five'
  'fifteen0.00/fifteen/loadavg')

  for interval_tag in ('one', 'five', 'fifteen'):
  print(loadavg.findtext(interval_tag))


 Stefan

 --
 http://mail.python.org/mailman/listinfo/python-list




-- 
Regards

Nibin.

http://TechsWare.in
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python and xml

[Stefan Behnel]

Hi,

please don't top-post (I fixed the citation order below).


Nibin V M, 16.05.2012 16:30:
 On Wed, May 16, 2012 at 7:53 PM, Stefan Behnel wrote:
 Nibin V M, 16.05.2012 16:16:
 I am trying to use cPanel XML-API and every API call return data in XML
 format. I would like to know how to manipulate the data here.

 For eg: How can I read the CPU load data from the below output



  loadavgone0.00/onefive0.00/fivefifteen0.00/fifteen
  /loadavg!-- whostmgrd --

 Here's some untested code to print the text values:

  import xml.etree.ElementTree as ET

  loadavg = ET.fromstring('loadavgone0.00/onefive0.00/five'
  'fifteen0.00/fifteen/loadavg')

  for interval_tag in ('one', 'five', 'fifteen'):
  print(loadavg.findtext(interval_tag))

 thank you Stefan. but the XML output is assigned to a variable; how to
 process the variable with XML contents?

Not sure what you mean. Maybe something like this?

xml = 'loadavgone0.00/onefive0.00/five.../loadavg'
loadavg = ET.fromstring(xml)

Stefan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3 - xml - crlf handling problem

[durumdara]

Dear Stefan!


So: may I don't understand the things well, but I thought that parser
drop the nondata CRLF-s + other characters (not preserve them).

Then don't matters that I read the XML from a file, or I create it
from code, because all of them generating SAME RESULT.
But Python don't do that.
If I make xml from code, the code is without plus characters.
But Python preserves parsed CRLF characters somewhere, and they are
also flushing into the result.

Example:

original='''
?xml version=1.0 encoding=utf-8?
doc a=1
element a=1
AnyText
/element
/doc
'''

If I parse this, and write with toxml, the CRLF-s remaining in the
code, but if I create this document line by line, there is no CRLF,
the toxml write only lined xml.

This also meaning that if I use prettyxml call, to prettying the xml,
the file size is growing.

If there is a multiple processing queue - if two pythons communicating
in xml files, the size can growing every time.

Py1 - read the Py2's file, process it, and write to a result file
Py2 - read the Py1's result file, process it, and pass back to Py1
this can grow the file with each call, because pretty CRLF-s not
normalized out from the code.

original='''
?xml version=1.0 encoding=utf-8?
doc a=1
element a=1
AnyText
/element
/doc
'''

def main():
f = open('test.0.xml','w')
f.write(original.strip())
f.close()

for i in range(1, 10 + 1):
xo = parse('test.%d.xml' % (i - 1))
de = xo.documentElement
de.setAttribute('c', str(i))
t = de.getElementsByTagName('element')[0]
tn = t.childNodes[0]
print (dir(t))
print (tn)
print (tn.nodeValue)
tn.nodeValue = str(i) + '\t' + '\n'
#s = xo.toxml()
s = xo.toprettyxml()
f = open('test.%d.xml' % i,'w')
f.write(s)
f.close()

sys.exit()

And: because Python is not converting CRLF to 013; I cannot make
different from prettied source's CRLF (loaded from template file),
my own pretty's CRLF (my own topretty), and really contained CRLF
(for example a memo field's value).

My case is that the processor application (for whom I pass the XML
from Python) is sensitive to plus CRLF-s in text nodes, I must do
something these plus items to avoid external's program errors.

I got these templates and input files from prettied format (with
CRLFS), but I must eat them to make an XML that one lined if
possible.

I hope you understand my problem with it.

Thanks:
   dd
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3 - xml - crlf handling problem

[Stefan Behnel]

durumdara, 02.12.2011 09:13:

So: may I don't understand the things well, but I thought that parser
drop the nondata CRLF-s + other characters (not preserve them).


Well, it does that, at least on my side (which is not under Windows):

===
original='''
?xml version=1.0 encoding=utf-8?
doc a=1
element a=1
AnyText
/element
/doc
'''

from xml.dom.minidom import parse

def main():
f = open('test.0.xml', 'wb')
f.write(original.strip().replace('\n', '\r\n').encode('utf8'))
f.close()

xo = parse('test.0.xml')
de = xo.documentElement
print(repr(de.childNodes[0].nodeValue))
print(repr(de.childNodes[1].childNodes[0].nodeValue))

if __name__ == '__main__':
main()
===

This prints '\n' and '\nAnyText\n' on my side, so the 
whitespace normalisation in the parser properly did its work.




Then don't matters that I read the XML from a file, or I create it
from code, because all of them generating SAME RESULT.
But Python don't do that.
If I make xml from code, the code is without plus characters.


What do you mean by plus characters? It's not the + character that you 
are referring to, right? Do you mean additional characters? Such as the 
additional '\r'?




But Python preserves parsed CRLF characters somewhere, and they are
also flushing into the result.

Example:

original='''
?xml version=1.0 encoding=utf-8?
doc a=1
 element a=1
 AnyText
 /element
/doc
'''

If I parse this, and write with toxml, the CRLF-s remaining in the
code, but if I create this document line by line, there is no CRLF,
the toxml write only lined xml.

This also meaning that if I use prettyxml call, to prettying the xml,
the file size is growing.

If there is a multiple processing queue - if two pythons communicating
in xml files, the size can growing every time.

Py1 - read the Py2's file, process it, and write to a result file
Py2 - read the Py1's result file, process it, and pass back to Py1
this can grow the file with each call, because pretty CRLF-s not
normalized out from the code.

original='''
?xml version=1.0 encoding=utf-8?
doc a=1
 element a=1
 AnyText
 /element
/doc
'''

def main():
 f = open('test.0.xml','w')
 f.write(original.strip())
 f.close()

 for i in range(1, 10 + 1):
 xo = parse('test.%d.xml' % (i - 1))
 de = xo.documentElement
 de.setAttribute('c', str(i))
 t = de.getElementsByTagName('element')[0]
 tn = t.childNodes[0]
 print (dir(t))
 print (tn)
 print (tn.nodeValue)
 tn.nodeValue = str(i) + '\t' + '\n'
 #s = xo.toxml()
 s = xo.toprettyxml()
 f = open('test.%d.xml' % i,'w')
 f.write(s)
 f.close()

 sys.exit()

And: because Python is not converting CRLF to013; I cannot make
different from prettied source's CRLF (loaded from template file),
my own pretty's CRLF (my own topretty), and really contained CRLF
(for example a memo field's value).

My case is that the processor application (for whom I pass the XML
from Python) is sensitive to plus CRLF-s in text nodes, I must do
something these plus items to avoid external's program errors.

I got these templates and input files from prettied format (with
CRLFS), but I must eat them to make an XML that one lined if
possible.

I hope you understand my problem with it.


Still not quite, but never mind. May or may not be a problem in minidom or 
your code. For example, you shouldn't open the output file in text mode but 
in binary mode (i.e. wb) because you are writing bytes into it.


Here's what I tried with ElementTree, and it seems to do what your code 
above wants. The indent() function is taken from Fredrik's element lib page:


http://effbot.org/zone/element-lib.htm


original='''
?xml version=1.0 encoding=utf-8?
doc a=1
element a=1
AnyText
/element
/doc
'''

def indent(elem, level=0):
i = \n + level*  
if len(elem):
if not elem.text or not elem.text.strip():
elem.text = i +   
if not elem.tail or not elem.tail.strip():
elem.tail = i
for elem in elem:
indent(elem, level+1)
if not elem.tail or not elem.tail.strip():
elem.tail = i
else:
if level and (not elem.tail or not elem.tail.strip()):
elem.tail = i

def main():
f = open('test.0.xml','w', encoding='utf8')
f.write(original.strip())
f.close()

from xml.etree.cElementTree import parse

for i in range(10):
tree = parse('test.%d.xml' % i)
root = tree.getroot()
root.set('c', str(i+1))
t = root.find('.//element')
t.text = '%d\t\n' % (i+1)
indent(root)
tree.write('test.%d.xml' % (i+1), encoding='utf8')

if __name__ == '__main__':
main()


Hope that helps,

Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Python 3 - xml - crlf handling problem

[durumdara]

Hi!

As I see that XML parsing is wrong in Python.

I must use predefined XML files, parsing them, extending them, and
produce some result.

But as I see that in Windows this is working wrong.

When the predefined XMLs are formatted (prettied) with CRLFs, then
the parser keeps these plus LF characters (not handle the logic that
CR = LF = CRLF), and it is appearing in the new result too.

xo = parse('test_original.xml')
de = xo.documentElement
de.setAttribute('b', 2)
b = xo.toxml('utf-8')
f = open('test_original2.xml', 'wb')
f.write(b)
f.close()

And: if I used text elements, this can extend the information with
plus characters and make wrong xml...

I can use only myowngenerated, and not prettied xmls because of this
problem!

Is this normal?

Thanks for your read:
   dd
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3 - xml - crlf handling problem

[Stefan Behnel]

durumdara, 30.11.2011 13:08:

As I see that XML parsing is wrong in Python.


You didn't say what you are using for parsing, but from your example, it 
appears likely that you are using the xml.dom.minidom module.




I must use predefined XML files, parsing them, extending them, and
produce some result.

But as I see that in Windows this is working wrong.

When the predefined XMLs are formatted (prettied) with CRLFs, then
the parser keeps these plus LF characters (not handle the logic that
CR = LF = CRLF), and it is appearing in the new result too.


I assume that you are referring to XML's newline normalisation algorithm? 
That should normally be handled by the parser, which, in the case of 
minidom, is usually expat. And I seriously doubt that expat has a problem 
with something as basic as newline normalisation.


Did you verify that the newlines are really not being converted by the 
parser? From your example, I can only see that you are serialising the XML 
tree back into a file, which may or may not alter the line endings by 
itself. Instead, take a look at the text content in the tree right after 
parsing to see how line endings look at that level.




 xo = parse('test_original.xml')
 de = xo.documentElement
 de.setAttribute('b', 2)
 b = xo.toxml('utf-8')
 f = open('test_original2.xml', 'wb')
 f.write(b)
 f.close()


This doesn't do any pretty printing, though, in case that's what you were 
really after (which appears likely according to your comments).




And: if I used text elements, this can extend the information with
plus characters and make wrong xml...


Sorry, I don't understand this sentence.



I can use only myowngenerated, and not prettied xmls because of this
problem!


Then what is the actual problem? Do you get an error somewhere? And if so, 
could you paste the exact error message and describe what you do to produce 
it? The mere fact that the line endings use the normal platform specific 
representation doesn't seem like a problem to me.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Python script xml-rpc to C# xml-rpc script

[Ymtrader]

If anyone is fluent in python as well as C# using the cook computing 
xml-rpc.net library I could really use some help.  I have been trying to write 
a C# program to access upcdatabase.com without much luck. I have a working 
python script. Would anyone be willing to show me the C# equivalent to this 
python script might be. Thank you

#!/usr/bin/env python
#
#

import sys
from xmlrpclib import ServerProxy, Error

rpc_key = 'enter rpc_key here'

if __name__=='__main__':
if len(sys.argv) != 2:
print 'Usage: fetchupc.py upc'
exit
else:
s = ServerProxy('http://www.upcdatabase.com/xmlrpc')
params = { 'rpc_key': rpc_key, 'upc': sys.argv[1] }
print s.lookup(params)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python script xml-rpc to C# xml-rpc script

[Adam Tauno Williams]

On Tue, 2011-03-15 at 08:00 -0700, Ymtrader wrote:
 If anyone is fluent in python as well as C# using the cook computing
 xml-rpc.net library I could really use some help.  I have been trying
 to write a C# program to access upcdatabase.com without much luck. 

Yes,  I've used both.

  have a working python script. Would anyone be willing to show me the
 C# equivalent to this python script might be. Thank you

If you ask on an appropriate forum - yes.  [the cook computing forum or
the Mono list].  This is *not* a Python question.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python parsing XML file problem with SAX

[Aahz]

In article mailman.1895.1281422126.1673.python-l...@python.org,
Stefan Behnel  stefan...@behnel.de wrote:
Christian Heimes, 10.08.2010 01:39:
 Am 10.08.2010 01:20, schrieb Aahz:
 The docs say, Parses an XML section into an element tree incrementally.
 Sure sounds like it retains the entire parsed tree in RAM.  Not good.
 Again, how do you parse an XML file larger than your available memory
 using something other than SAX?

 The document at
 http://www.ibm.com/developerworks/xml/library/x-hiperfparse/ explains it
 one way.

 The iterparser approach is ingenious but it doesn't work for every XML
 format. Let's say you have a 10 GB XML file with one millionpart/
 tags. An iterparser doesn't load the entire document. Instead it
 iterates over the file and yields (for example) one million ElementTrees
 for eachpart/  tag and its children. You can get the nice API of
 ElementTree with the memory efficiency of a SAX parser if you obey
 Listing 4.

In the very common case that you are interested in all children of the root 
element, it's even enough to intercept on the specific tag name (lxml.etree 
has an option for that, but an 'if' block will do just fine in ET) and just 
.clear() the child element at the end of the loop body. That results in 
very fast and simple code, but will leave the tags in the tree while only 
removing their content and attributes. Usually works well enough for 
several ten thousand elements, especially when using cElementTree.

Thanks to both of you!
-- 
Aahz (a...@pythoncraft.com)   * http://www.pythoncraft.com/

...if I were on life-support, I'd rather have it run by a Gameboy than a
Windows box.  --Cliff Wells
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python parsing XML file problem with SAX

[Stefan Behnel]

Christian Heimes, 10.08.2010 01:39:

Am 10.08.2010 01:20, schrieb Aahz:

The docs say, Parses an XML section into an element tree incrementally.
Sure sounds like it retains the entire parsed tree in RAM.  Not good.
Again, how do you parse an XML file larger than your available memory
using something other than SAX?


The document at
http://www.ibm.com/developerworks/xml/library/x-hiperfparse/ explains it
one way.

The iterparser approach is ingenious but it doesn't work for every XML
format. Let's say you have a 10 GB XML file with one millionpart/
tags. An iterparser doesn't load the entire document. Instead it
iterates over the file and yields (for example) one million ElementTrees
for eachpart/  tag and its children. You can get the nice API of
ElementTree with the memory efficiency of a SAX parser if you obey
Listing 4.


In the very common case that you are interested in all children of the root 
element, it's even enough to intercept on the specific tag name (lxml.etree 
has an option for that, but an 'if' block will do just fine in ET) and just 
.clear() the child element at the end of the loop body. That results in 
very fast and simple code, but will leave the tags in the tree while only 
removing their content and attributes. Usually works well enough for 
several ten thousand elements, especially when using cElementTree.


As usual, a bit of benchmarking will uncover the right way to do it in your 
case. That's also a huge advantage over SAX: iterparse code is much easier 
to tune into a streamlined loop body when you need it.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python parsing XML file problem with SAX

[Aahz]

In article mailman.1250.1280314148.1673.python-l...@python.org,
Stefan Behnel  stefan...@behnel.de wrote:

First of all: don't use SAX. Use ElementTree's iterparse() function. That 
will shrink you code down to a simple loop in a few lines.

Unless I'm missing something, that only helps if the final tree fits into
memory.  What do you suggest other than SAX if your XML file may be
hundreds of megabytes?
-- 
Aahz (a...@pythoncraft.com)   * http://www.pythoncraft.com/

...if I were on life-support, I'd rather have it run by a Gameboy than a
Windows box.  --Cliff Wells
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python parsing XML file problem with SAX

[Stefan Behnel]

Aahz, 09.08.2010 18:52:

In articlemailman.1250.1280314148.1673.python-l...@python.org,
Stefan Behnel wrote:


First of all: don't use SAX. Use ElementTree's iterparse() function. That
will shrink you code down to a simple loop in a few lines.


Unless I'm missing something, that only helps if the final tree fits into
memory.  What do you suggest other than SAX if your XML file may be
hundreds of megabytes?


Well, what about using ElementTree's iterparse() function in that case? 
That's what it's good at, and its cElementTree version is extremely fast.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python parsing XML file problem with SAX

[Aahz]

In article mailman.1860.1281375095.1673.python-l...@python.org,
Stefan Behnel  stefan...@behnel.de wrote:
Aahz, 09.08.2010 18:52:
 In articlemailman.1250.1280314148.1673.python-l...@python.org,
 Stefan Behnel wrote:

 First of all: don't use SAX. Use ElementTree's iterparse() function. That
 will shrink you code down to a simple loop in a few lines.

 Unless I'm missing something, that only helps if the final tree fits into
 memory.  What do you suggest other than SAX if your XML file may be
 hundreds of megabytes?

Well, what about using ElementTree's iterparse() function in that case? 
That's what it's good at, and its cElementTree version is extremely fast.

The docs say, Parses an XML section into an element tree incrementally.
Sure sounds like it retains the entire parsed tree in RAM.  Not good.
Again, how do you parse an XML file larger than your available memory
using something other than SAX?
-- 
Aahz (a...@pythoncraft.com)   * http://www.pythoncraft.com/

...if I were on life-support, I'd rather have it run by a Gameboy than a
Windows box.  --Cliff Wells
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python parsing XML file problem with SAX

[Christian Heimes]

Am 10.08.2010 01:20, schrieb Aahz:
 The docs say, Parses an XML section into an element tree incrementally.
 Sure sounds like it retains the entire parsed tree in RAM.  Not good.
 Again, how do you parse an XML file larger than your available memory
 using something other than SAX?

The document at
http://www.ibm.com/developerworks/xml/library/x-hiperfparse/ explains it
one way.

The iterparser approach is ingenious but it doesn't work for every XML
format. Let's say you have a 10 GB XML file with one million part/
tags. An iterparser doesn't load the entire document. Instead it
iterates over the file and yields (for example) one million ElementTrees
for each part/ tag and its children. You can get the nice API of
ElementTree with the memory efficiency of a SAX parser if you obey
Listing 4.

Christian

-- 
http://mail.python.org/mailman/listinfo/python-list

Python parsing XML file problem with SAX

[jia li]

I have an XML file with hundreds of error elements.

What's strange is only one of there elements could not be parsed correctly:
error
checkerREVERSE_INULL/checker
functionDispose_ParameterList/function
unmangled_functionDispose_ParameterList/unmangled_function
statusUNINSPECTED/status
num146/num
home1/146MMSLib_LinkedList.c/home
/error

I printed the data in characters(self, data) and after parsing. The result
is one \r\n is inserted after 1/ and 146MMSLib_LinkedList.c for the
latter.

But if I make my XML file only this element left, it could parse correctly.

My script below:
class CoverityErrorHandler(ContentHandler):
def __init__(self):
self.is_num = False
self.num = 
self.is_func = False
self.function = 
self.is_file = False
self.filename = 
self.is_Report = False
self.report = 
self.is_Checker = False
self.checker = 
self.is_unmangled_func = False
self.unmangled_func= 
self.is_Status = False
self.Status = 
self.mapping = {}

def startElement(self, name, attributes):
if name == num:
self.is_num = True
elif name == unmangled_function:
self.is_unmangled_func = True
elif name == checker:
self.is_Checker = True
elif name == file:
self.is_file = True
elif name == home:
self.is_Report = True
elif name == function:
self.is_func = True
elif name == status:
self.is_Status = True

def characters(self, data):
if self.is_num:
self.num = data
elif self.is_func:
self.function = data
elif self.is_Checker:
self.checker = data
elif self.is_file:
self.filename = data
elif self.is_Report:
self.report = data;
print self.report;
elif self.is_unmangled_func:
self.unmangled_func = data
elif self.is_Status:
self.Status = data

def endElement(self, name):
if name == error:
self.mapping[self.num] = CoverityError(self.checker,
self.filename, self.function, self.report)
elif name == num:
self.is_num = False
elif name == unmangled_function:
self.is_unmangled_func = False
elif name == checker:
self.is_Checker = False
elif name == file:
self.is_file = False
elif name == home:
self.is_Report = False
elif name == function:
self.is_func = False
elif name == status:
self.is_Status = False


Please any expert help to have a look. Thanks!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python parsing XML file problem with SAX

[Stefan Behnel]

jia li, 28.07.2010 12:10:

I have an XML file with hundreds oferror  elements.

What's strange is only one of there elements could not be parsed correctly:
error
checkerREVERSE_INULL/checker
functionDispose_ParameterList/function
unmangled_functionDispose_ParameterList/unmangled_function
statusUNINSPECTED/status
num146/num
home1/146MMSLib_LinkedList.c/home
/error

I printed the data in characters(self, data) and after parsing. The result
is one \r\n is inserted after 1/ and 146MMSLib_LinkedList.c for the
latter.

But if I make my XML file only this element left, it could parse correctly.


First of all: don't use SAX. Use ElementTree's iterparse() function. That 
will shrink you code down to a simple loop in a few lines.


Then, the problem is likely that you are getting separate events for text 
nodes. The \r\n most likely only occurs due to your print statement, I 
doubt that it's really in the data returned from SAX. Again: using 
ElementTree instead of SAX will avoid this kind of problem.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: SEC proposes the use of Python and XML

[Chris Rebert]

On Sat, Apr 10, 2010 at 9:13 PM, Ted Larson Freeman
free...@alumni.stanford.org wrote:
 This week the SEC proposed new requirements for asset-backed
 securities that include the use of XML and Python:

 The asset-level information would be provided according to proposed
 standards and in a tagged data format using eXtensible Markup Language
 (XML). In addition, we are proposing to require, along with the
 prospectus filing, the filing of a computer program of the contractual
 cash flow provisions expressed as downloadable source code in Python,
 a commonly used open source interpretive programming language.

I guess we've now gone from executable pseudocode to executable
legal code :-)

Cheers,
Chris
--
Cue the Perl jokes!
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SEC proposes the use of Python and XML

[Patrick Maupin]

On Apr 10, 11:13 pm, Ted Larson Freeman free...@alumni.stanford.org
wrote:
 This week the SEC proposed new requirements for asset-backed
 securities that include the use of XML and Python:

 The asset-level information would be provided according to proposed
 standards and in a tagged data format using eXtensible Markup Language
 (XML). In addition, we are proposing to require, along with the
 prospectus filing, the filing of a computer program of the contractual
 cash flow provisions expressed as downloadable source code in Python,
 a commonly used open source interpretive programming language.

 See:http://www.sec.gov/news/press/2010/2010-54.htmhttp://kelloggfinance.wordpress.com/2010/04/08/the-sec-gets-one-right

 Ted

Now all we need is for the IRS to follow suit...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SEC proposes the use of Python and XML

[Cameron Simpson]

On 10Apr2010 23:05, Chris Rebert c...@rebertia.com wrote:
| On Sat, Apr 10, 2010 at 9:13 PM, Ted Larson Freeman
| free...@alumni.stanford.org wrote:
|  This week the SEC proposed new requirements for asset-backed
|  securities that include the use of XML and Python:
| 
|  The asset-level information would be provided according to proposed
|  standards and in a tagged data format using eXtensible Markup Language
|  (XML). In addition, we are proposing to require, along with the
|  prospectus filing, the filing of a computer program of the contractual
|  cash flow provisions expressed as downloadable source code in Python,
|  a commonly used open source interpretive programming language.
| 
| I guess we've now gone from executable pseudocode to executable
| legal code :-)

And not before time, either :-(

But unless they require it to be usable as unit tests (eg running as
validation of a live transaction stream, or in some audit mode) it may be
less useful than one might hope. Of course, if they're gone as far as
requiring python code, perhaps that's exactly what they hope to be able to
do.

Cheers,
-- 
Cameron Simpson c...@zip.com.au DoD#743
http://www.cskk.ezoshosting.com/cs/

One of the guys in my homebrew club is a mailman. He says that one of the
requirements to work for the post office is to be able to work the slide on a
9mm while laughing. - Phillip J. Birmingham
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SEC proposes the use of Python and XML

[Terry Reedy]

On 4/11/2010 12:13 AM, Ted Larson Freeman wrote:

This week the SEC proposed new requirements for asset-backed
securities that include the use of XML and Python:

The asset-level information would be provided according to proposed
standards and in a tagged data format using eXtensible Markup Language
(XML). In addition, we are proposing to require, along with the
prospectus filing, the filing of a computer program of the contractual
cash flow provisions expressed as downloadable source code in Python,
a commonly used open source interpretive programming language.


Thanks for posting this. More jobs for Python programmers. And perhaps 
better transparency in such investment vehicles.



See:
http://www.sec.gov/news/press/2010/2010-54.htm
http://kelloggfinance.wordpress.com/2010/04/08/the-sec-gets-one-right


To help anyone trying to verify this, the quote is a quote within the 
blog post which seems to attribute it to the SEC press release. However, 
it actually comes from


http://www.sec.gov/rules/proposed/2010/33-9117.pdf

the 'Additional Material' referenced within the press release.

Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: SEC proposes the use of Python and XML

[Patrick Maupin]

On Apr 11, 3:12 am, Cameron Simpson c...@zip.com.au wrote:
 On 10Apr2010 23:05, Chris Rebert c...@rebertia.com wrote:
 | On Sat, Apr 10, 2010 at 9:13 PM, Ted Larson Freeman| 
 free...@alumni.stanford.org wrote:

 |  This week the SEC proposed new requirements for asset-backed
 |  securities that include the use of XML and Python:
 | 
 |  The asset-level information would be provided according to proposed
 |  standards and in a tagged data format using eXtensible Markup Language
 |  (XML). In addition, we are proposing to require, along with the
 |  prospectus filing, the filing of a computer program of the contractual
 |  cash flow provisions expressed as downloadable source code in Python,
 |  a commonly used open source interpretive programming language.
 |
 | I guess we've now gone from executable pseudocode to executable
 | legal code :-)

 And not before time, either :-(

 But unless they require it to be usable as unit tests (eg running as
 validation of a live transaction stream, or in some audit mode) it may be
 less useful than one might hope. Of course, if they're gone as far as
 requiring python code, perhaps that's exactly what they hope to be able to
 do.

 Cheers,
 --
 Cameron Simpson c...@zip.com.au DoD#743http://www.cskk.ezoshosting.com/cs/

 One of the guys in my homebrew club is a mailman. He says that one of the
 requirements to work for the post office is to be able to work the slide on a
 9mm while laughing.     - Phillip J. Birmingham

I would think there are two major reasons for this:

1) less wiggle room for the lawyers to argue about the terms of the
contract.

2) it makes it easier for parties to the contract to assess their
risks under different scenarios by plugging all their contracts into a
single model.

I don't know the SEC needs to define standards for unit-tests, etc.
-- that might happen organically, because these are all big companies
with their own in-house programming staffs.  Getting the programmers
involved before a contract is signed is a great thing.

Once the lawyers are told the program defines the contract you can
bet both sides are going to check out the program pretty carefully,
and if a programmer says hey, when this thing happens in the economy,
we owe them 5 times the GDP of the planet, the contract probably
won't happen.

Regards,
Pat
-- 
http://mail.python.org/mailman/listinfo/python-list

SEC proposes the use of Python and XML

[Ted Larson Freeman]

This week the SEC proposed new requirements for asset-backed
securities that include the use of XML and Python:

The asset-level information would be provided according to proposed
standards and in a tagged data format using eXtensible Markup Language
(XML). In addition, we are proposing to require, along with the
prospectus filing, the filing of a computer program of the contractual
cash flow provisions expressed as downloadable source code in Python,
a commonly used open source interpretive programming language.

See:
http://www.sec.gov/news/press/2010/2010-54.htm
http://kelloggfinance.wordpress.com/2010/04/08/the-sec-gets-one-right

Ted
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Help

[ookrin]

On Apr 15, 9:43 am, Scott David Daniels scott.dani...@acm.org wrote:
 ookrin wrote:
   I am still learning. And it's not that I won't take the advice
  for using ElementTree, I just currently don't know anything about it.
  I just didn't want to say, I have no idea what you're talking about!
  to Scott cause I figured that would be rude, but I guess so is not
  saying anything, sorry. So I'll need to read up on it before I
  actually try to do anything with that approach.

  Seeing the errors - I changed the two classes to this:
    ... [some code that actually says a _little_ bit about his target] 

 If you had done a better job of asking your question, you'd have found a
 quick answer.  That is why everyone points people at smartquestions, not
 to pick on them.

 Presuming you have an xml structure like:
      txt = '''abc something def
             name name='steve' characterID='sid' Some contents. /name
             name name='stevey' characterID='sidy'Other contents./name
             /def/abc'''
      import xml.etree.ElementTree as ET
      structure = ET.fromstring(xmls)
      for node in structure.findall('def/name'):
          print '%s: %s / %s: %s' % (node.tag, node.attrib['name'],
                                node.attrib['characterID'], node.text)

 or a file named 'my.xml' with the same contents:
      import xml.etree.ElementTree as ET
      structure = ET.parse('my.xml')
      for node in structure.findall('def/name'):
          print '%s: %s / %s: %s' % (node.tag, node.attrib['name'],
                                node.attrib['characterID'], node.text)

 --Scott David Daniels
 scott.dani...@acm.org

Thanks very much for your patience with me and thanks for the help
everyone.
This is my end result:

self.api = apiConnection(self.userID,self.apiKey)
xmlData = self.api.returnXml()
if(xmlData != None):
struct = ET.fromstring(xmlData)
self.charIDs =[]
i=0
for node in struct.findall('result/rowset/row'):
char = node.attrib['name']
id = node.attrib['characterID']
temp = char, id
#append them to the variable for later
self.charIDs.append(temp)
i+=1
#Setup the select window and run it
#selects which character I want
self.selectCharUi = selectChar(self,self.charIDs)
self.selectCharUi.selectWindow.exec_()

#Get the name that was selected
self.selectedCharName = self.selectCharUi.returnValue
()
#Set the text widget with the name
self.lE_charID.setText(self.selectedCharName)
#enable the Ok button
self.pbOk.setEnabled(True)

It collects the names and IDs, sends them to a pyqt4 window with a
list widget to list the names, a name gets selected and then I get the
name that was selected. Though now that I think about it, maybe using
a dictionary would be better. Elsewhere I gather up the userID, apiID,
characterName and characterID and write it t a file. I hope it doesn't
look too horrible.

John - my programming experience was two semesters of c++ about 6
years ago. Everything else I've learned from the internet.  =/

I'll try to make my questions better next time.

Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Help

[John Machin]

On Apr 15, 2:25 pm, ookrin ook...@gmail.com wrote:


 Seeing the errors - I changed the two classes to this:

 class offlineLoad():
     def loadXmlFile(self):
         print Loading from File
         xf = open('CharacterID.xml','r')
         xml = xmlHandler()
         saxparser = make_parser()
         saxparser.setContentHandler(xml)

         saxparser.parse(xf)

 class xmlHandler(ContentHandler):
     def __init__(self):
         print ---
         self.idList = []

     def startElement(self, name, attrs):
         if name == row:
             charName = attrs.get(name, )
             charID = attrs.get(characterID, )
             self.buildlist(charName,charID)

     def buildlist(self,charName,charID):
         temp = charName,charID
         self.idList.append(temp)
         print 
         print self.idList

 I know calling the self.idList = [] in the init probably isn't
 correct, but I couldn't think of any other way currently so it doesn't
 get rebuilt every time the definition gets called. This works and at
 least I think it puts everything into an array for me.

AFAICT having self.idList = [] in the __init__ method is the only
sensible way to do what you want. Non-sensible ways: make it global to
the module, or local to a class method

You do have a problem with what you've got so far: you've done the
heist, you've stuffed the loot into a sack, but where's the getaway
car? IOW your loadXmlFile method needs a line like

   return xml.idList

Wouldn't loadXMLFile be better as a stand-alone function? A class
which is not a subclass of anything more meaty than object and has
only one method (which isn't __init__) smells strongly of excess
reliance on paradigms best left behind with the language from which
you acquired them :-)

Cheers,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Help

[Scott David Daniels]

ookrin wrote:

 I am still learning. And it's not that I won't take the advice
for using ElementTree, I just currently don't know anything about it.
I just didn't want to say, I have no idea what you're talking about!
to Scott cause I figured that would be rude, but I guess so is not
saying anything, sorry. So I'll need to read up on it before I
actually try to do anything with that approach.

Seeing the errors - I changed the two classes to this:
  ... [some code that actually says a _little_ bit about his target] 


If you had done a better job of asking your question, you'd have found a
quick answer.  That is why everyone points people at smartquestions, not
to pick on them.

Presuming you have an xml structure like:
txt = '''abc something def
   name name='steve' characterID='sid' Some contents. /name
   name name='stevey' characterID='sidy'Other contents./name
   /def/abc'''
import xml.etree.ElementTree as ET
structure = ET.fromstring(xmls)
for node in structure.findall('def/name'):
print '%s: %s / %s: %s' % (node.tag, node.attrib['name'],
  node.attrib['characterID'], node.text)

or a file named 'my.xml' with the same contents:
import xml.etree.ElementTree as ET
structure = ET.parse('my.xml')
for node in structure.findall('def/name'):
print '%s: %s / %s: %s' % (node.tag, node.attrib['name'],
  node.attrib['characterID'], node.text)


--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Help

[ookrin]

On Apr 12, 12:51 am, Diez B. Roggisch de...@nospam.web.de wrote:
 ookrin schrieb:



  I'm in the process of learning python and PyQt4. I had decided to make
  myself a simple app and soon discovered that I needed to crash into
  xml to use some of the data I was going to be getting off of the
  server.

  I picked up enough xml to use the sax parser to get the data out of
  the xml. I can get as far as printing it to the screen, but there is
  where I get stuck I can't seem to send the data to anywhere else,
  into another variable, to another function. The parser stops after the
  first line it finds.

  class offlineLoad():
      def loadXmlFile(self):
          print Loading from File
          xf = open('CharacterID.xml','r')
          xml = xmlHandler()
          saxparser = make_parser()
          print parser created
          saxparser.setContentHandler(xml)
          print parser runn
          try:
              saxparser.parse(xf)
          except:
              print Error Reading xml

 This is a very bad thing to do - there are only very few justified cases
 of catch-all for exceptions. This certainly isn't one, as it suppresses
 the real cause for what is happening.

 Which makes it harder for you  for us to diagnose the problem, because
 you don't show (and currently can't) the stacktrace to us.

  class xmlHandler(ContentHandler):
      def startElement(self, name, attrs):
          self.charList = []

 If this shall gather some state over the coures of time, this is the
 wrong place to initialize it, as it will get overwritten with each element.

          charName = []
          charID = []

 There is no need to prefill variables. And it's confusing you do it
 with a different type than what you assign to them below.



          if name == row:
              charName = attrs.get(name, )
              charID = attrs.get(characterID, )
              print charName, '\t', charID
              self.buildlist(self,charName,charID)
              #self.test()

      def test(self):
          print TeST
      def buildlist(self,charName,charID):
          print charName, '\t',charID

  so here... test will print, but buildlist makes the parser stop and
  print Error reading XML after the first row it finds.  Appending
  charName and charID to another variable makes it stop as well. I'm
  confused at this point

 Please show us the stacktrace you suppress. Then help might be possible.

 Diez

Sorry for the delay, easter weekend:

I know you don't need to prefill the variables with python, and they
weren't there originally, I was just getting frustrated and trying
various things to see if they worked. There is actually no stacktrace
error. It will just print me the error message that I have when I try
to send the variables outside of the if loop. (from the try - except
statement.) I can still run click the button and it'll do it again.

There are three rows that I have to get out of my xml all the same
except with different values:
 row name=CharName charID=IDNumber 

So it prints out the first of the three lines and then just says
Error Reading XML if I'm running the buildlist() and trying to send
the variables out. As long as I don't try to do anything, it works
fine.

I can get these out and print them to the cmd line. But I want to
collect these in a variable and send it back off to my gui in pyqt4 to
list them. which is where i'm getting stuck.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Help

[John Machin]

On Apr 15, 12:29 pm, ookrin ook...@gmail.com wrote:
 On Apr 12, 12:51 am, Diez B. Roggisch de...@nospam.web.de wrote:



  ookrin schrieb:

   I'm in the process of learning python and PyQt4. I had decided to make
   myself a simple app and soon discovered that I needed to crash into
   xml to use some of the data I was going to be getting off of the
   server.

   I picked up enough xml to use the sax parser to get the data out of
   the xml. I can get as far as printing it to the screen, but there is
   where I get stuck I can't seem to send the data to anywhere else,
   into another variable, to another function. The parser stops after the
   first line it finds.

   class offlineLoad():
       def loadXmlFile(self):
           print Loading from File
           xf = open('CharacterID.xml','r')
           xml = xmlHandler()
           saxparser = make_parser()
           print parser created
           saxparser.setContentHandler(xml)
           print parser runn
           try:
               saxparser.parse(xf)
           except:
               print Error Reading xml

  This is a very bad thing to do - there are only very few justified cases
  of catch-all for exceptions. This certainly isn't one, as it suppresses
  the real cause for what is happening.

  Which makes it harder for you  for us to diagnose the problem, because
  you don't show (and currently can't) the stacktrace to us.

  Please show us the stacktrace you suppress. Then help might be possible.

 There is actually no stacktrace
 error. It will just print me the error message that I have when I try
 to send the variables outside of the if loop. (from the try - except
 statement.)

That's because (as Diez has already pointed out, and told you to stop
doing) you are *suppressing* the error message and stack trace that
will help diagnose what is going wrong in your callback method.

Instead of this:
try:
saxparser.parse(xf)
except:
print Error Reading xml
do just this:
saxparser.parse(xf)
or if you really want to print something at the time, do this:
try:
saxparser.parse(xf)
except:
print something meaningful
raise # throw the exception back for normal handling

And just in case you're not taking the advice of Scott D. D. either,
let me tell you again: use ElementTree, it's easier (no callbacks to
complicate things) and the ratio of helpful-user-count to problem-
likelihood is likely to be much higher than with sax.

Cheers,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Help

[ookrin]

On Apr 14, 8:15 pm, John Machin sjmac...@lexicon.net wrote:
 On Apr 15, 12:29 pm, ookrin ook...@gmail.com wrote:



  On Apr 12, 12:51 am, Diez B. Roggisch de...@nospam.web.de wrote:

   ookrin schrieb:

I'm in the process of learning python and PyQt4. I had decided to make
myself a simple app and soon discovered that I needed to crash into
xml to use some of the data I was going to be getting off of the
server.

I picked up enough xml to use the sax parser to get the data out of
the xml. I can get as far as printing it to the screen, but there is
where I get stuck I can't seem to send the data to anywhere else,
into another variable, to another function. The parser stops after the
first line it finds.

class offlineLoad():
    def loadXmlFile(self):
        print Loading from File
        xf = open('CharacterID.xml','r')
        xml = xmlHandler()
        saxparser = make_parser()
        print parser created
        saxparser.setContentHandler(xml)
        print parser runn
        try:
            saxparser.parse(xf)
        except:
            print Error Reading xml

   This is a very bad thing to do - there are only very few justified cases
   of catch-all for exceptions. This certainly isn't one, as it suppresses
   the real cause for what is happening.

   Which makes it harder for you  for us to diagnose the problem, because
   you don't show (and currently can't) the stacktrace to us.

   Please show us the stacktrace you suppress. Then help might be possible.

  There is actually no stacktrace
  error. It will just print me the error message that I have when I try
  to send the variables outside of the if loop. (from the try - except
  statement.)

 That's because (as Diez has already pointed out, and told you to stop
 doing) you are *suppressing* the error message and stack trace that
 will help diagnose what is going wrong in your callback method.

 Instead of this:
         try:
             saxparser.parse(xf)
         except:
             print Error Reading xml
 do just this:
         saxparser.parse(xf)
 or if you really want to print something at the time, do this:
         try:
             saxparser.parse(xf)
         except:
             print something meaningful
             raise # throw the exception back for normal handling

 And just in case you're not taking the advice of Scott D. D. either,
 let me tell you again: use ElementTree, it's easier (no callbacks to
 complicate things) and the ratio of helpful-user-count to problem-
 likelihood is likely to be much higher than with sax.

 Cheers,
 John

Ok, I didn't know that was what I was actively doing by using the try-
except. I am still learning. And it's not that I won't take the advice
for using ElementTree, I just currently don't know anything about it.
I just didn't want to say, I have no idea what you're talking about!
to Scott cause I figured that would be rude, but I guess so is not
saying anything, sorry. So I'll need to read up on it before I
actually try to do anything with that approach.


Seeing the errors - I changed the two classes to this:

class offlineLoad():
def loadXmlFile(self):
print Loading from File
xf = open('CharacterID.xml','r')
xml = xmlHandler()
saxparser = make_parser()
saxparser.setContentHandler(xml)

saxparser.parse(xf)

class xmlHandler(ContentHandler):
def __init__(self):
print ---
self.idList = []

def startElement(self, name, attrs):
if name == row:
charName = attrs.get(name, )
charID = attrs.get(characterID, )
self.buildlist(charName,charID)

def buildlist(self,charName,charID):
temp = charName,charID
self.idList.append(temp)
print 
print self.idList

I know calling the self.idList = [] in the init probably isn't
correct, but I couldn't think of any other way currently so it doesn't
get rebuilt every time the definition gets called. This works and at
least I think it puts everything into an array for me. I'm going to
look at the element tree now, but I probably won't figure it out
tonight.

I'm using python 2.6

Thanks so far,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Help

[Scott David Daniels]

ookrin wrote:

I'm in the process of learning python and PyQt4. I had decided to make
myself a simple app and soon discovered that I needed to crash into
xml to use some of the data I was going to be getting off of the
server.

I picked up enough xml to use the sax parser to get the data out of
the xml. I can get as far as printing it to the screen, but there is
where I get stuck


Assuming you are using a somewhat current Python (2.5 or later, you
did not specify), the etree package is buit in.  If not, you can
download it for your python easily.  You can then (for example)
use something like:
from xml.etree import cElementTree as ET
...
parsed = ET.parse('CharacterID.xml')
for node in parsed.getiterator('osArchitecture'): print node.text

or if you have the data in a string,
root = ET.fromstring('thexmlstring/xml/the')
print root[0].tag, root[0].text

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Help

[Diez B. Roggisch]

ookrin schrieb:

I'm in the process of learning python and PyQt4. I had decided to make
myself a simple app and soon discovered that I needed to crash into
xml to use some of the data I was going to be getting off of the
server.

I picked up enough xml to use the sax parser to get the data out of
the xml. I can get as far as printing it to the screen, but there is
where I get stuck I can't seem to send the data to anywhere else,
into another variable, to another function. The parser stops after the
first line it finds.

class offlineLoad():
def loadXmlFile(self):
print Loading from File
xf = open('CharacterID.xml','r')
xml = xmlHandler()
saxparser = make_parser()
print parser created
saxparser.setContentHandler(xml)
print parser runn
try:
saxparser.parse(xf)
except:
print Error Reading xml


This is a very bad thing to do - there are only very few justified cases 
of catch-all for exceptions. This certainly isn't one, as it suppresses 
the real cause for what is happening.


Which makes it harder for you  for us to diagnose the problem, because 
you don't show (and currently can't) the stacktrace to us.



class xmlHandler(ContentHandler):
def startElement(self, name, attrs):
self.charList = []


If this shall gather some state over the coures of time, this is the 
wrong place to initialize it, as it will get overwritten with each element.



charName = []
charID = []


There is no need to prefill variables. And it's confusing you do it 
with a different type than what you assign to them below.



if name == row:
charName = attrs.get(name, )
charID = attrs.get(characterID, )
print charName, '\t', charID
self.buildlist(self,charName,charID)
#self.test()

def test(self):
print TeST
def buildlist(self,charName,charID):
print charName, '\t',charID

so here... test will print, but buildlist makes the parser stop and
print Error reading XML after the first row it finds.  Appending
charName and charID to another variable makes it stop as well. I'm
confused at this point


Please show us the stacktrace you suppress. Then help might be possible.

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Python and XML Help

[ookrin]

I'm in the process of learning python and PyQt4. I had decided to make
myself a simple app and soon discovered that I needed to crash into
xml to use some of the data I was going to be getting off of the
server.

I picked up enough xml to use the sax parser to get the data out of
the xml. I can get as far as printing it to the screen, but there is
where I get stuck I can't seem to send the data to anywhere else,
into another variable, to another function. The parser stops after the
first line it finds.

class offlineLoad():
def loadXmlFile(self):
print Loading from File
xf = open('CharacterID.xml','r')
xml = xmlHandler()
saxparser = make_parser()
print parser created
saxparser.setContentHandler(xml)
print parser runn
try:
saxparser.parse(xf)
except:
print Error Reading xml

class xmlHandler(ContentHandler):
def startElement(self, name, attrs):
self.charList = []
charName = []
charID = []
if name == row:
charName = attrs.get(name, )
charID = attrs.get(characterID, )
print charName, '\t', charID
self.buildlist(self,charName,charID)
#self.test()

def test(self):
print TeST
def buildlist(self,charName,charID):
print charName, '\t',charID

so here... test will print, but buildlist makes the parser stop and
print Error reading XML after the first row it finds.  Appending
charName and charID to another variable makes it stop as well. I'm
confused at this point
--
http://mail.python.org/mailman/listinfo/python-list

A question about python and xml

[Holden]

Hello everyone I heard this was a good community to go too for help
and advice. I want to make a web site that uses the python programming
language which I am VERY new at. This website would store simple data
such as names in a form. At first I wanted to use mysql to store the
data but I want to export the data using xml.

So say if a user logged in they would be able to store there basic
information in an xml file and download it whenever. Is that possible
using XML and Python. I am sorry if this post doesn't make much sense
but any advice would be greatly appreciated. You can also e-mail me if
you need further information or clarification.

Thanks

Holden
--
http://mail.python.org/mailman/listinfo/python-list

Re: A question about python and xml

[Stefan Behnel]

Holden wrote:
 I want to make a web site that uses the python programming
 language which I am VERY new at. This website would store simple data
 such as names in a form. At first I wanted to use mysql to store the
 data but I want to export the data using xml.
 
 So say if a user logged in they would be able to store there basic
 information in an xml file and download it whenever. Is that possible
 using XML and Python. I am sorry if this post doesn't make much sense
 but any advice would be greatly appreciated. You can also e-mail me if
 you need further information or clarification.

Generating XML from your data shouldn't be too hard once it's in a database.
The harder part is getting it in there through a web interface. I would look
at a dedicated web framework like Django first:

http://www.djangoproject.com/

Stefan
--
http://mail.python.org/mailman/listinfo/python-list

Re: A question about python and xml

[J. Cliff Dyer]

On Fri, 2008-05-09 at 08:39 -0700, Holden wrote:
 Hello everyone I heard this was a good community to go too for help
 and advice. I want to make a web site that uses the python programming
 language which I am VERY new at. This website would store simple data
 such as names in a form. At first I wanted to use mysql to store the
 data but I want to export the data using xml.
 
 So say if a user logged in they would be able to store there basic
 information in an xml file and download it whenever. Is that possible
 using XML and Python. I am sorry if this post doesn't make much sense
 but any advice would be greatly appreciated. You can also e-mail me if
 you need further information or clarification.
 
 Thanks
 
 Holden
 --
 http://mail.python.org/mailman/listinfo/python-list
 

A general question will only get you a general answer, but yes, it's
very possible.  

You'll want to look into the cgi module (in the standard library)for
basic, ground level server/browser communication.  You might also be
interested in using a web framework, for keeping your site design clean
and manageable.  I recommend Django from my own experience, but you
might also look into turbogears, pylons, webpy, or any of a number of
other well-regarded options.  The downside to using a framework is that
it's another set of things to learn, which, if you're just starting out
with python, might be a bit much to chew, but they can be very helpful
with maintenance down the line.

For XML processing, lxml.etree or ElementTree should suit your needs.
ElementTree is included in the standard library of recent versions of
python, though lxml is easily downloadable, and adds a couple bells and
whistles.  There are other options available, but those are
straightforward to use, and well designed.

Good luck with your site.

Cheers,
Cliff


--
http://mail.python.org/mailman/listinfo/python-list

Python and xml

[Doran, Harold]

I am a python neophyte who has used python to parse through text files using 
basic functions and no OOP experience. I have a need to process some xml files 
and from what I am now reading python is the exact tool I need to work through 
this issue.

However, as I read online docs and peruse which books to buy, I am quickly 
becoming overwhelmed with the topic and could use some guidance on how to best 
tackle my task. To frame the issue, here is what I would like to be able to do. 

I have a statistical program that outputs the data into an xml document. My 
goal is to use python to parse this document and output a .txt document or .csv 
document that organizes the results into human-readable rows and columns in a 
nicely formatted manner.

As I read online docs about xml and python, it seems the topic is very big. 
While I am more than happy to invest significant time learning this topic, it 
may be possible to narrow the scope of what I need to know and be able to do in 
order to complete my task.

Any suggestions on how I should proceed and/or fundamentals I need to learn? Is 
anyone aware of simple how to docs or other tutorials that demonstrate this 
kind of task?

Thanks,
Harold


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and xml

[Konstantin Veretennicov]

On Sat, Mar 29, 2008 at 3:57 PM, Doran, Harold [EMAIL PROTECTED] wrote:

  I am a python neophyte who has used python to parse through text files
 using basic functions and no OOP experience. I have a need to process some
 xml files and from what I am now reading python is the exact tool I need to
 work through this issue.

 However, as I read online docs and peruse which books to buy, I am quickly
 becoming overwhelmed with the topic and could use some guidance on how to
 best tackle my task.

 You can start with this basic example (requires Python 2.5):

spam.xml:
monty
  episode number=14Dinsdale (Face the Press)/episode
  episode number=15The Spanish Inquisition/episode
  episode number=16Deja vu/episode
  episode number=17The Buzz Aldrin Show/episode
  episode number=18Live From the Grill-O-Mat Snack Bar/episode
  episode number=19It's a Living/episode
  episode number=20The Attila the Hun Show/episode
  episode number=21Archaeology Today/episode
  episode number=22How to Recognize Different Parts of the
Body/episode
  episode number=23Scott of the Antarctic/episode
  episode number=24How Not to Be Seen/episode
  episode number=25Spam/episode
  episode number=26Royal Episode 13/episode
/monty

spam.py:
from xml.etree.ElementTree import ElementTree as ET

et = ET(file='spam.xml')
for episode in et.findall('episode'):
print episode.attrib['number'] + ':', '' + episode.text + ''


Use standard csv module if you want to produce csv ouput (
http://docs.python.org/lib/module-csv.html).

--
kv
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python - CGI - XML - XSD

[Diez B. Roggisch]

 Sorry, it was really late when i wrote this post. The file is an XSL
 file. It defines HTML depending on what appears in the XML document.

Then the content-type might be the culprit, yes. But testing so would have
been faster than waiting for answers here... 

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python - CGI - XML - XSD

[Diez B. Roggisch]

xkenneth wrote:

 Hi All,
 
Quick question. I've got an XML schema file (XSD) that I've
 written, that works fine when my data is present as an XML file.
 (Served out by apache2.) Now when I call python as a cgi script, and
 tell it print out all of the same XML, also served up by apache2, the
 XSD is not applied. Does this have to do with which content type i
 defined when printing the xml to stdout?

Who's applying the stylesheet? The browser, some application like XmlSpy or
what?

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python - CGI - XML - XSD

[xkenneth]

On Mar 12, 6:32 am, Diez B. Roggisch [EMAIL PROTECTED] wrote:
 xkenneth wrote:
  Hi All,

     Quick question. I've got an XML schema file (XSD) that I've
  written, that works fine when my data is present as an XML file.
  (Served out by apache2.) Now when I call python as a cgi script, and
  tell it print out all of the same XML, also served up by apache2, the
  XSD is not applied. Does this have to do with which content type i
  defined when printing the xml to stdout?

 Who's applying the stylesheet? The browser, some application like XmlSpy or
 what?

 Diez

The browser.

Regards,
Kenneth Miller
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python - CGI - XML - XSD

[Stefan Behnel]

xkenneth wrote:
 On Mar 12, 6:32 am, Diez B. Roggisch [EMAIL PROTECTED] wrote:
 xkenneth wrote:
 Hi All,
Quick question. I've got an XML schema file (XSD) that I've
 written, that works fine when my data is present as an XML file.
 (Served out by apache2.) Now when I call python as a cgi script, and
 tell it print out all of the same XML, also served up by apache2, the
 XSD is not applied. Does this have to do with which content type i
 defined when printing the xml to stdout?
 Who's applying the stylesheet? The browser, some application like XmlSpy or
 what?
 
 The browser.

Well, why should it validate your file? Browsers don't do that just for fun.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python - CGI - XML - XSD

[xkenneth]

On Mar 12, 11:58 am, Stefan Behnel [EMAIL PROTECTED] wrote:
 xkenneth wrote:
  On Mar 12, 6:32 am, Diez B. Roggisch [EMAIL PROTECTED] wrote:
  xkenneth wrote:
  Hi All,
     Quick question. I've got an XML schema file (XSD) that I've
  written, that works fine when my data is present as an XML file.
  (Served out by apache2.) Now when I callpythonas a cgi script, and
  tell it print out all of the same XML, also served up by apache2, the
 XSDis not applied. Does this have to do with which content type i
  defined when printing the xml to stdout?
  Who's applying the stylesheet? The browser, some application like XmlSpy or
  what?

  The browser.

 Well, why should it validate your file? Browsers don't do that just for fun.

 Stefan

Sorry, it was really late when i wrote this post. The file is an XSL
file. It defines HTML depending on what appears in the XML document.

Regards,
Kenneth Miller
-- 
http://mail.python.org/mailman/listinfo/python-list

Python - CGI - XML - XSD

[xkenneth]

Hi All,

   Quick question. I've got an XML schema file (XSD) that I've
written, that works fine when my data is present as an XML file.
(Served out by apache2.) Now when I call python as a cgi script, and
tell it print out all of the same XML, also served up by apache2, the
XSD is not applied. Does this have to do with which content type i
defined when printing the xml to stdout?

Regards,
Kenneth Miller
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python object - XML

[Stefan Behnel]

Samuel wrote:
 Say you have the following XML:
 
 item ref=1
   nameitem 1/name
 /item
 item ref=2
   nameitem 2/name
 /item
 group
   item ref=1 /
   item ref=2 /
   namemy group/name
 /group
 
 Is there an easy way (i.e. without writing a sax/dom parser) to load
 this into a (number of) Python object(s), manipulate the instance, and
 save the result back to XML?

Try lxml.objectify. It doesn't copy your data into other objects, but it gives
you all the freedom to design your own objects as an abstraction of the XML
data. See here:

http://codespeak.net/lxml/objectify.html

especially these sections:

http://codespeak.net/lxml/objectify.html#element-access-through-object-attributes
http://codespeak.net/lxml/objectify.html#python-data-types
http://codespeak.net/lxml/objectify.html#defining-additional-data-classes

It's part of lxml, which makes it plenty fast, highly flexible and gives you
all the XML features you might ever need. :)

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Python object - XML

[Samuel]

Hi,

Say you have the following XML:

item ref=1
  nameitem 1/name
/item
item ref=2
  nameitem 2/name
/item
group
  item ref=1 /
  item ref=2 /
  namemy group/name
/group

Is there an easy way (i.e. without writing a sax/dom parser) to load
this into a (number of) Python object(s), manipulate the instance, and
save the result back to XML?

-Samuel

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python object - XML

[Jarek Zgoda]

Samuel napisał(a):

 Say you have the following XML:
 
 item ref=1
   nameitem 1/name
 /item
 item ref=2
   nameitem 2/name
 /item
 group
   item ref=1 /
   item ref=2 /
   namemy group/name
 /group
 
 Is there an easy way (i.e. without writing a sax/dom parser) to load
 this into a (number of) Python object(s), manipulate the instance, and
 save the result back to XML?

Yea, use ElementTree and you'd get a bunch of nested lists of very
simple objects.

-- 
Jarek Zgoda
http://jpa.berlios.de/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python object - XML

[Samuel]

On Sep 3, 10:19 pm, Jarek Zgoda [EMAIL PROTECTED] wrote:
  Is there an easy way (i.e. without writing a sax/dom parser) to load
  this into a (number of) Python object(s), manipulate the instance, and
  save the result back to XML?

 Yea, use ElementTree and you'd get a bunch of nested lists of very
 simple objects.

Sorry for being unclear. By load this into a number of Python
objects I mean, filling already existing objects with data. In other
words:

class Item(object):
  def __init__(self, ref, name):
self.ref  = ref
self.name = name

class Group(object):
  def __init__(self, ref, name, item = []):
self.ref  = ref
self.name = name
self.item = item

mapper = Mapper()
objects = mapper.load('data.xml')
print objects['1']
Item object at 0x324235a

(Obviously, in my example the mapper could not always know where a
list is required, but an existing mapper will surely have a solution
implemented.)
I guess what I am looking for is pretty much an ORM that also works
with XML.

-Samuel

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python object - XML

[Laurent Pointal]

Samuel wrote:

 Hi,
 
 Say you have the following XML:
 
 item ref=1
   nameitem 1/name
 /item
 item ref=2
   nameitem 2/name
 /item
 group
   item ref=1 /
   item ref=2 /
   namemy group/name
 /group
 
 Is there an easy way (i.e. without writing a sax/dom parser) to load
 this into a (number of) Python object(s), manipulate the instance, and
 save the result back to XML?
 
 -Samuel

I dont know if this suit your needs:

http://uche.ogbuji.net/tech/4suite/amara/

Take a look at the Amara Bindery:

http://uche.ogbuji.net/tech/4suite/etc/amara-manual.html#bindery

A+

Laurent.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML?

[Leonard J. Reder]

Stefan,

This package is looking better I tried the validation example, but it 
did not work
with 1.3beta so I grabbed 1.2.  If objectify works well I think this is a
great pick for anyone using xml.  Please post to me and the python announce
when 1.3 is released.

thanks,
Len
Stefan Behnel wrote:
 Hi,
 
 Leonard J. Reder wrote:
 Stefan Behnel wrote:
 http://codespeak.net/lxml/objectify.html
 Looks like this stuff might do what I want.  Need to work through the
 example.

 Are you the author?
 
 Yup! :)
 
 
 Why is this better then using gnossis?
 
 lxml is faster and supports more XML standards. It's also more flexible in
 that it allows you to replace element classes at basically any granularity and
 thus add things to the Element API or even replace it as you see fit, without
 loosing the standard XML features. You can even do that at the C-level, in
 case you really need high-performance.
 
 See the extending lxml section in the side menu at
 http://codespeak.net/lxml/dev/
 especially:
 http://codespeak.net/lxml/dev/element_classes.html
 
 Stefan


-- 
===
Leonard J. Reder
Home office email : [EMAIL PROTECTED]
Lab email : [EMAIL PROTECTED]
Lab web page  : http://reder.jpl.nasa.gov
===
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML?

[Stefan Behnel]

Hi,

Leonard J. Reder wrote:
 Stefan Behnel wrote:
 http://codespeak.net/lxml/objectify.html
 
 Looks like this stuff might do what I want.  Need to work through the
 example.
 
 Are you the author?

Yup! :)


 Why is this better then using gnossis?

lxml is faster and supports more XML standards. It's also more flexible in
that it allows you to replace element classes at basically any granularity and
thus add things to the Element API or even replace it as you see fit, without
loosing the standard XML features. You can even do that at the C-level, in
case you really need high-performance.

See the extending lxml section in the side menu at
http://codespeak.net/lxml/dev/
especially:
http://codespeak.net/lxml/dev/element_classes.html

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML?

[Stefan Behnel]

Leonard J. Reder wrote:
 What I as really after though was other possible solutions
 like JAX in the Java world or Gnosis in the Python world.
 
 Something that can take a Relax NG Schema and compile
 it into a bunch of parser/generator objects for handling
 an XML format.

That's why I was asking why you need this. Validation is provided by RNG
itself, so all you would need is some kind of mapping that outputs and parses
it for you? What's your target format? Python objects?

http://codespeak.net/lxml/objectify.html

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML?

[Leonard J. Reder]

Stefan Behnel wrote:
 Leonard J. Reder wrote:
 What I as really after though was other possible solutions
 like JAX in the Java world or Gnosis in the Python world.

 Something that can take a Relax NG Schema and compile
 it into a bunch of parser/generator objects for handling
 an XML format.
 
 That's why I was asking why you need this. Validation is provided by RNG
 itself, so all you would need is some kind of mapping that outputs and parses
 it for you? What's your target format? Python objects?
 
 http://codespeak.net/lxml/objectify.html
 
 Stefan

My target format are Python objects.  What I want to do is
auto-covert the Relax NG Schema into Python API objects
that can be used to intelligently read in an XML file.

Our XML files specify interfaces to flight software
but in the future I have an interest in using XMI to do more
complete modeling of software.  Hopefully the OMG publishes
a stand XMI schema.

Looks like this stuff might do what I want.  Need to work through the 
example.

Are you the author?  Why is this better then using gnossis?

Thanks,

Len

-- 
===
Leonard J. Reder
Home office email : [EMAIL PROTECTED]
Lab email : [EMAIL PROTECTED]
Lab web page  : http://reder.jpl.nasa.gov
===
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML?

[Stefan Behnel]

Leonard J. Reder wrote:
 What is the best way to process a Relax NG Schema and auto generate XML
 Python parser/generator code?
 Any suggestions?

Not sure what you mean with XML Python parser/generator code, but regarding
RNG, you should definitely give lxml a try. It may already be what you were
looking for anyway.

http://codespeak.net/lxml

Have fun,
Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML?

[Leonard J. Reder]

Thanks for the response.  This package does look interesting.
What I as really after though was other possible solutions
like JAX in the Java world or Gnosis in the Python world.

Something that can take a Relax NG Schema and compile
it into a bunch of parser/generator objects for handling
an XML format.   Guess I am going to just have try out
Gnosis since it seems like the only game in town.

Len

Stefan Behnel wrote:
 Leonard J. Reder wrote:
 What is the best way to process a Relax NG Schema and auto generate XML
 Python parser/generator code?
 Any suggestions?
 
 Not sure what you mean with XML Python parser/generator code, but regarding
 RNG, you should definitely give lxml a try. It may already be what you were
 looking for anyway.
 
 http://codespeak.net/lxml
 
 Have fun,
 Stefan


-- 
===
Leonard J. Reder
Home office email : [EMAIL PROTECTED]
Lab email : [EMAIL PROTECTED]
Lab web page  : http://reder.jpl.nasa.gov
===
-- 
http://mail.python.org/mailman/listinfo/python-list

Python and XML?

[Leonard J. Reder]

Hello,

What is the best way to process a Relax NG Schema and auto generate XML 
Python parser/generator code?
Any suggestions?

Thanks for all replies,

Len
-- 
===
Leonard J. Reder
Home office email : [EMAIL PROTECTED]
Lab email : [EMAIL PROTECTED]
Lab web page  : http://reder.jpl.nasa.gov
===
-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: Python and XML talk in Omaha

[Mike Hostetler]

On Sept 5, I will be giving a presentation on Python and XML for the
Omaha Dynamic Users Group.  

The goal of the Dynamic Users Group is to learn from and about dynamic
languages, (Perl, Ruby, Smalltalk as well as Python). 

Most of the audience will not be Python users. However, my talk will
not be an introductory talk about Python.  Instead it will be a
whet-your-appetite talk with loads of examples.  I will focus on XML
processing with minidom and cElementtree.  I would appreciate any
other Pythonistas in the area to join the fun.

The talk will take place on the campus of Creighton University and
will go from 7pm-9pm.

Here's a link with more info:
http://blog.blainebuxton.com/2006/08/python-and-xml-in-omaha.html


-- 
Mike Hostetler  
[EMAIL PROTECTED]
http://mike.hostetlerhome.com
-- 
http://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations.html

Python and XML Schema

[Trond]

I have a need to, given a XML document with XML Schema defined, to get the 
type of each element in the XML file based on the XML Schema URI given. IE, 
the title element is of simple datatype string. Is there any existing 
Python libraries were I can parse the XML file, and for a given node ask for 
the datatype (which should be taken from XML Schema?)

Trond
 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and XML Schema

[Dennis Benzinger]

Trond wrote:
 I have a need to, given a XML document with XML Schema defined, to get the 
 type of each element in the XML file based on the XML Schema URI given. IE, 
 the title element is of simple datatype string. Is there any existing 
 Python libraries were I can parse the XML file, and for a given node ask for 
 the datatype (which should be taken from XML Schema?)
 [...]

Take a look at lxml. It's a pythonic binding for libxml2 and libxslt.
libxml2 http://xmlsoft.org/ implements XML Schema Part 2: Datatypes 
and partially XML Schema Part 1: Structures.
I don't know how complete the structures implementation is and if it can 
already do what you want. So you might ask the libxml2 guys how to do 
what you want with libxml2 and then use lxml to do it in Python.

Bye,
Dennis
-- 
http://mail.python.org/mailman/listinfo/python-list