Re: [Zope] external method strangeness
garry saddington wrote at 2006-10-18 18:13 +0100:
> ...
>import psycopg, string, re, subprocess
>def scholarpack_xml_transform(source):
>f=open('/opt/scholarpack/ancillary/source.xml','w')
>f.write(source)
>f.close
The line above looks buggy: missing "()".
> ...
>do=curs.execute(transformed1)
>do
The line above looks doubtfull: no effect
>conn.commit()
>conn.close
>
>I get a programming error thrown from the conn.commit() line and the
>method fails.
Almost surely, "postgres" does not like the command your have generated.
It is quite picky with binary data...
--
Dieter
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] external method strangeness
On Wed, 2006-10-18 at 15:07 -0400, Paul Winkler wrote:
> On Wed, Oct 18, 2006 at 06:13:38PM +0100, garry saddington wrote:
> > Can anyone explain what is happening here. I am using Saxon to transform
> > an xml file(source) using a stylesheet(style) the result is then written
> > to Postgres. The contents of source are uploaded in Zope and passed to
> > the external method as a string. When I run this external method:
> >
> > import psycopg, string, re, subprocess
> > def scholarpack_xml_transform(source):
> > f=open('/opt/scholarpack/ancillary/source.xml','w')
> > f.write(source)
>
> I have no idea what's causing your problem, but this raises
> red flags for me: if two people upload sources at the same time,
> you'll have problems here.
There is no chance of this as it is a one-off populate the db exercise.
> If you really really must read/write
> files on the filesystem, either use locks or use python's
> tempfile module.
>
> > f.close
> > source= '/opt/scholarpack/ancillary/source.xml'
> > style='/opt/scholarpack/ancillary/style.xml'
> > source1=source.replace(''','`')
> > source2=source1.replace('NLPGaddress','BS7666address')
> > p=re.compile("\'")
> > source3=p.sub( '`' , source2)
> > r =
> > subprocess.Popen(['/opt/scholarpack/ancillary/jre/bin/java','-jar','./saxon.jar',source3,style],
> > stdout = subprocess.PIPE,cwd = '/opt/scholarpack/ancillary/')
>
> Are you just using saxon for xslt processing? If so, IMO using an
> external process for that is severe overkill. Get lxml and try
> something like this:
>
> from lxml import etree
> style = etree.XSLT(etree.parse(some_file_object))
> transformed = style.apply(source_data)
Thank you I've never heard of this, I'll give it a try. I was using
4suite but it started to misbehave in the way that I wanted to use it.
regards
Garry
>
> but again, this doesn't address your immediate problem :)
>
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] external method strangeness
On Wed, Oct 18, 2006 at 06:13:38PM +0100, garry saddington wrote:
> Can anyone explain what is happening here. I am using Saxon to transform
> an xml file(source) using a stylesheet(style) the result is then written
> to Postgres. The contents of source are uploaded in Zope and passed to
> the external method as a string. When I run this external method:
>
> import psycopg, string, re, subprocess
> def scholarpack_xml_transform(source):
> f=open('/opt/scholarpack/ancillary/source.xml','w')
> f.write(source)
I have no idea what's causing your problem, but this raises
red flags for me: if two people upload sources at the same time,
you'll have problems here. If you really really must read/write
files on the filesystem, either use locks or use python's
tempfile module.
> f.close
> source= '/opt/scholarpack/ancillary/source.xml'
> style='/opt/scholarpack/ancillary/style.xml'
> source1=source.replace(''','`')
> source2=source1.replace('NLPGaddress','BS7666address')
> p=re.compile("\'")
> source3=p.sub( '`' , source2)
> r =
> subprocess.Popen(['/opt/scholarpack/ancillary/jre/bin/java','-jar','./saxon.jar',source3,style],
> stdout = subprocess.PIPE,cwd = '/opt/scholarpack/ancillary/')
Are you just using saxon for xslt processing? If so, IMO using an
external process for that is severe overkill. Get lxml and try
something like this:
from lxml import etree
style = etree.XSLT(etree.parse(some_file_object))
transformed = style.apply(source_data)
but again, this doesn't address your immediate problem :)
--
Paul Winkler
http://www.slinkp.com
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] external method strangeness
+---[ garry saddington ]-- | On Thu, 2006-10-19 at 03:18 +1000, Andrew Milton wrote: | > +---[ garry saddington ]-- | > | Can anyone explain what is happening here. I am using Saxon to transform | > | an xml file(source) using a stylesheet(style) the result is then written | > | to Postgres. The contents of source are uploaded in Zope and passed to | > | the external method as a string. When I run this external method: | > | > [snip] | > | > | Can anyone point me in the correct direction? | > | > Text encoding? | The same files are used both times. The error log is not very helpful | here but I enclose it for completeness. | regards | garry I'd add some debugging into the pyana.py file, at the point where it's raising the exception. -- Andrew Milton [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] external method strangeness
On Thu, 2006-10-19 at 03:18 +1000, Andrew Milton wrote:
> +---[ garry saddington ]--
> | Can anyone explain what is happening here. I am using Saxon to transform
> | an xml file(source) using a stylesheet(style) the result is then written
> | to Postgres. The contents of source are uploaded in Zope and passed to
> | the external method as a string. When I run this external method:
>
> [snip]
>
> | Can anyone point me in the correct direction?
>
> Text encoding?
The same files are used both times. The error log is not very helpful
here but I enclose it for completeness.
regards
garry
Log
Undo
Ownership
Security
Site Error Log Site Error Log at /error_log
Exception traceback
Time
2006/10/18 18:42:33.586 GMT+1
User Name (User Id)
scholarpack (scholarpack)
Request URL
http://localhost/scholarpack/admin/interoperability/upload/addCTF
Exception Type
ProgrammingError
Exception Value
Traceback (innermost last):
* Module ZPublisher.Publish, line 113, in publish
* Module ZPublisher.mapply, line 88, in mapply
* Module ZPublisher.Publish, line 40, in call_object
* Module Shared.DC.Scripts.Bindings, line 311, in __call__
* Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec
* Module Products.PythonScripts.PythonScript, line 323, in _exec
* Module None, line 11, in addCTF
Line 11
* Module OFS.DTMLMethod, line 144, in __call__
URL:
http://localhost/scholarpack/admin/interoperability/upload/result/manage_main
Physical Path:/scholarpack/admin/interoperability/upload/result
* Module DocumentTemplate.DT_String, line 476, in __call__
* Module Shared.DC.Scripts.Bindings, line 325, in
__render_with_namespace__
* Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec
* Module Products.PythonScripts.PythonScript, line 323, in _exec
* Module None, line 2, in test_html
Line 2
* Module Products.ExternalMethod.ExternalMethod, line 225, in
__call__
__traceback_info__: (('\n\n\n \n
U925540006009\n Abara\n
Tonna\n 1989-07-17\n
F\n \n
Abara\n
Tona\n
12\n BAFR\n
O\n \n \n
F\n
ENG\n \n \n
false\n
false\n
C\n \n \n
\n H61 Road 42\n Victoria
Garden City\n Box 72675\n Victoria
Island\n Lagos\n
Nigeria\n \n \n
\n \n Mr & Mrs\n
Abara\n FAM\n
true\n \n
true\n \n \n
\n H\n
014618466\n \n \n
H\n 0208 5750766\n
\n \n \n RMMS\n
1\n
true\n
true\n Mr &
Mrs Abara\n \n \n
\n \n RMMS\n
C\n \n
n/s\n
ENG\n \n
\n \n \n U92554000501A\n
Abdullah\n Safaris\n
1995-07-08\n M\n \n
Abdullah\n
Safaris\n
7\n MOTH\n
T\n \n \n
F\n
ENG\n \n \n
false\n
false\n
C\n \n \n
\n 43 St Margarets Avenue\n
\n \n Skegness\n
Lincolnshire\n PE25 2LX\n
England\n \n \n
\n \n Mrs\n
Lilley\n PAM\n
true\n \n
true\n \n \n
\n H\n 01754
760193\n \n \n
H\n 07783 538592\n
\n \n \n RMMS\n
1\n
true\n
true\n Mrs
Lilley\n \n \n \n
\n \n KS3\n
\n ENG\n 2006\n
ENG\n 47\n
SUB\n I\n
NS\n 0\n
\n \n \n
\n RMMS\n
C\n \n
n/s\n
ENG\n \n
\n \n\n\n',), {},
None)
* Module /opt/scholarpack/instances/scholarpack/Extensions/pyana.py, line
18, in scholarpack_xml_transform
ProgrammingError:
Display traceback as text
REQUEST
form
submit
' Add '
id
'ctfxml'
file
'\n\n\n \n
U925540006009\n Abara\n
Tonna\n 1989-07-17\n F\n
\n Abara\n
Tona\n 12\n
BAFR\n O\n
\n \n F\n
ENG\n \n \n
false\n false\n
C\n \n \n \n
H61 Road 42\n Victoria Garden City\n Box
72675\n Victoria Island\n Lagos\n
Nigeria\n \n \n \n
\n Mr & Mrs\n Abara\n
FAM\n true\n
\n true\n \n \n
\n H\n 014618466\n \n
\n H\n 0208 5750766\n
\n \n \n RMMS\n
1\n true\n
true\n Mr & Mrs
Abara\n \n \n \n \n
RMMS\n C\n
\n n/s\n
ENG\n \n \n
\n \n U92554000501A\n Abdullah\n
Safaris\n 1995-07-08\n M\n
\n Abdullah\n
Safaris\n
7\n MOTH\n
T\n \n \n
F\n ENG\n \n
\n false\n
false\n C\n
\n \n \n 43 St Margarets
Avenue\n \n \n
Skegness\n Lincolnshire\n PE25
2LX\n England\n \n \n
\n \n Mrs\n Lilley\n
PAM\n true\n
\n true\n \n \n
\n H\n 01754 760193\n
\n \n H\n 07783
538592\n \n \n \n RMMS\n
1\n true\n
true\n Mrs
Lilley\n \n \n \n
\n \n KS3\n \n
ENG\n 2006\n ENG\n
47\n SUB\n
I\n NS\n
0\n \n \n \n
\n RMMS\n
C\n \n
n/s\n ENG\n
\n \n \n\n\n'
cookies
dtpref_rows
'20'
dtpref_cols
'100%'
tree-s
'eJzTyCkw5NLIKT
Re: [Zope] external method strangeness
+---[ garry saddington ]-- | Can anyone explain what is happening here. I am using Saxon to transform | an xml file(source) using a stylesheet(style) the result is then written | to Postgres. The contents of source are uploaded in Zope and passed to | the external method as a string. When I run this external method: [snip] | Can anyone point me in the correct direction? Text encoding? Would probably be a lot easier if you told us the error message in full that you get. -- Andrew Milton [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] external method strangeness
Can anyone explain what is happening here. I am using Saxon to transform
an xml file(source) using a stylesheet(style) the result is then written
to Postgres. The contents of source are uploaded in Zope and passed to
the external method as a string. When I run this external method:
import psycopg, string, re, subprocess
def scholarpack_xml_transform(source):
f=open('/opt/scholarpack/ancillary/source.xml','w')
f.write(source)
f.close
source= '/opt/scholarpack/ancillary/source.xml'
style='/opt/scholarpack/ancillary/style.xml'
source1=source.replace(''','`')
source2=source1.replace('NLPGaddress','BS7666address')
p=re.compile("\'")
source3=p.sub( '`' , source2)
r =
subprocess.Popen(['/opt/scholarpack/ancillary/jre/bin/java','-jar','./saxon.jar',source3,style],
stdout = subprocess.PIPE,cwd = '/opt/scholarpack/ancillary/')
r.wait()
transformed = r.stdout.read()
transformed1=transformed.replace(',)',')')
conn = psycopg.connect("dbname=scholarpack user=scholarpack")
curs = conn.cursor()
do=curs.execute(transformed1)
do
conn.commit()
conn.close
I get a programming error thrown from the conn.commit() line and the
method fails. However when I run the following script externally to Zope
after the Zope failure (so that source.xml exists) the DB write is
successful.
#!/opt/scholarpack/python/bin/python
import psycopg, string, re, subprocess
source='source.xml'
style='style.xml'
source1=source.replace(''','`')
source2=source1.replace('NLPGaddress','BS7666address')
p=re.compile("\'")
source3=p.sub( '`' , source2)
r =
subprocess.Popen(['/opt/scholarpack/ancillary/jre/bin/java','-jar','./saxon.jar',source3,style],
stdout = subprocess.PIPE,cwd = '/opt/scholarpack/ancillary/')
r.wait()
transformed = r.stdout.read()
transformed1=transformed.replace(',)',')')
conn = psycopg.connect("dbname=scholarpack user=scholarpack")
curs = conn.cursor()
do=curs.execute(transformed1)
do
conn.commit()
conn.close
Can anyone point me in the correct direction?
regards
garry
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
