Re: [Zope] Zope FTP - OverflowError: requested number of bytes is more than a Python string can hold
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan Schulze wrote: > I am are using ZServer/Medusa for FTP-Uploads to our ZODB. This works > for smaller files, but the above error appears in instance.log, when > using a ~5 GB file. > > There's probably nothing I can do about it, as it's an internal > python marshalling limit. Or would there be a way to make uploading > big files work? > > > This is the complete error message from instance.log: > > 2009-12-04T12:34:49 ERROR Zope.SiteErrorLog > http://0.0.0.0/Zope2/sara/data/RudSoil_vs_lssu_160807.rma/PUT > Traceback (innermost last): > Module ZPublisher.Publish, line 119, in publish > Module ZPublisher.mapply, line 88, in mapply > Module ZPublisher.Publish, line 42, in call_object > Module webdav.NullResource, line 167, in PUT > Module Products.Archetypes.WebDAVSupport, line 116, in PUT > Module Products.Archetypes.utils, line 160, in mapply > Module Products.Marshall.marshaller, line 99, in demarshall > OverflowError: requested number of bytes is more than a Python string can hold > -- > 2009-12-04T12:34:57 ERROR ZServer uncaptured python exception, closing > channel 0x1e5c6a70> (socket.error:(9, 'Bad file descriptor') > [/raid/opt/Python-2.4.6/lib/python2.4/asynchat.py|initiate_send|219] > [/raid/opt/Python-2.4.6/lib/python2.4/asyncore.py|send|332] > [/raid/opt/Python-2.4.6/lib/python2.4/socket.py|_dummy|136]) You need to be using a blob-aware field type here. This is not a Python / medusa problem, but an Archetypes / Marshall problem: check with the Plone folks to see if there is a supported solution for the versino of Plone you are running, or look at Chris McDonough's blob package: http://plope.com/software/blob/ Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksZLXAACgkQ+gerLs4ltQ7AzQCghVeRiD3F1ETunBeQtwiZz636 f8sAnjmJNc46qxoSY2xP+NUJZEvgy9F6 =4jae -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope FTP - OverflowError: requested number of bytes is more than a Python string can hold
And here is a nice comment within marshaller.py:
93 def demarshall(self, obj, data, **kw):$
94 if kw.has_key('file'):$
95 if not data:$
96 # XXX Yuck! Shouldn't read the whole file, never.$
97 # OTOH, if you care about large files, you should be$
98 # using the PrimaryFieldMarshaller or something$
99 # similar.$
100 data = kw['file'].read()$
101 del kw['file']$
So when this is your own content-types: it's broken and should likely
use PrimaryFieldMarshaller, otherwise: Plone is guilty one.
In general I would upload large files to Zope/Plone only w/ blob support
or iw.fss support (Plone)
Andreas
Am 04.12.09 12:53, schrieb Andreas Jung:
> You're trying to upload a 5GB file into Zope?
>
> Andreas
>
> Am 04.12.09 12:47, schrieb Jan Schulze:
>
>> Hi all,
>>
>> I am are using ZServer/Medusa for FTP-Uploads to our ZODB. This works for
>> smaller files, but the above error appears in instance.log, when using a ~5
>> GB file.
>>
>> There's probably nothing I can do about it, as it's an internal python
>> marshalling limit. Or would there be a way to make uploading big files work?
>>
>>
>> This is the complete error message from instance.log:
>>
>> 2009-12-04T12:34:49 ERROR Zope.SiteErrorLog
>> http://0.0.0.0/Zope2/sara/data/RudSoil_vs_lssu_160807.rma/PUT
>> Traceback (innermost last):
>> Module ZPublisher.Publish, line 119, in publish
>> Module ZPublisher.mapply, line 88, in mapply
>> Module ZPublisher.Publish, line 42, in call_object
>> Module webdav.NullResource, line 167, in PUT
>> Module Products.Archetypes.WebDAVSupport, line 116, in PUT
>> Module Products.Archetypes.utils, line 160, in mapply
>> Module Products.Marshall.marshaller, line 99, in demarshall
>> OverflowError: requested number of bytes is more than a Python string can
>> hold
>> --
>> 2009-12-04T12:34:57 ERROR ZServer uncaptured python exception, closing
>> channel > 0x1e5c6a70> (socket.error:(9, 'Bad file descriptor')
>> [/raid/opt/Python-2.4.6/lib/python2.4/asynchat.py|initiate_send|219]
>> [/raid/opt/Python-2.4.6/lib/python2.4/asyncore.py|send|332]
>> [/raid/opt/Python-2.4.6/lib/python2.4/socket.py|_dummy|136])
>>
>>
>>
>> Best Regards,
>> Jan
>> ___
>> Zope maillist - Zope@zope.org
>> https://mail.zope.org/mailman/listinfo/zope
>> ** No cross posts or HTML encoding! **
>> (Related lists -
>> https://mail.zope.org/mailman/listinfo/zope-announce
>> https://mail.zope.org/mailman/listinfo/zope-dev )
>>
>>
>
>
>
>
> ___
> Zope maillist - Zope@zope.org
> https://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> https://mail.zope.org/mailman/listinfo/zope-announce
> https://mail.zope.org/mailman/listinfo/zope-dev )
>
--
ZOPYX Ltd. & Co KG \ zopyx group
Charlottenstr. 37/1 \ The full-service network for your
D-72070 Tübingen \ Python, Zope and Plone projects
www.zopyx.com, i...@zopyx.com \ www.zopyxgroup.com
E-Publishing, Python, Zope & Plone development, Consulting
<>___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope FTP - OverflowError: requested number of bytes is more than a Python string can hold
You're trying to upload a 5GB file into Zope? Andreas Am 04.12.09 12:47, schrieb Jan Schulze: > Hi all, > > I am are using ZServer/Medusa for FTP-Uploads to our ZODB. This works for > smaller files, but the above error appears in instance.log, when using a ~5 > GB file. > > There's probably nothing I can do about it, as it's an internal python > marshalling limit. Or would there be a way to make uploading big files work? > > > This is the complete error message from instance.log: > > 2009-12-04T12:34:49 ERROR Zope.SiteErrorLog > http://0.0.0.0/Zope2/sara/data/RudSoil_vs_lssu_160807.rma/PUT > Traceback (innermost last): > Module ZPublisher.Publish, line 119, in publish > Module ZPublisher.mapply, line 88, in mapply > Module ZPublisher.Publish, line 42, in call_object > Module webdav.NullResource, line 167, in PUT > Module Products.Archetypes.WebDAVSupport, line 116, in PUT > Module Products.Archetypes.utils, line 160, in mapply > Module Products.Marshall.marshaller, line 99, in demarshall > OverflowError: requested number of bytes is more than a Python string can hold > -- > 2009-12-04T12:34:57 ERROR ZServer uncaptured python exception, closing > channel 0x1e5c6a70> (socket.error:(9, 'Bad file descriptor') > [/raid/opt/Python-2.4.6/lib/python2.4/asynchat.py|initiate_send|219] > [/raid/opt/Python-2.4.6/lib/python2.4/asyncore.py|send|332] > [/raid/opt/Python-2.4.6/lib/python2.4/socket.py|_dummy|136]) > > > > Best Regards, > Jan > ___ > Zope maillist - Zope@zope.org > https://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > https://mail.zope.org/mailman/listinfo/zope-announce > https://mail.zope.org/mailman/listinfo/zope-dev ) > -- ZOPYX Ltd. & Co KG \ zopyx group Charlottenstr. 37/1 \ The full-service network for your D-72070 Tübingen \ Python, Zope and Plone projects www.zopyx.com, i...@zopyx.com \ www.zopyxgroup.com E-Publishing, Python, Zope & Plone development, Consulting <>___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope FTP - OverflowError: requested number of bytes is more than a Python string can hold
Hi all, I am are using ZServer/Medusa for FTP-Uploads to our ZODB. This works for smaller files, but the above error appears in instance.log, when using a ~5 GB file. There's probably nothing I can do about it, as it's an internal python marshalling limit. Or would there be a way to make uploading big files work? This is the complete error message from instance.log: 2009-12-04T12:34:49 ERROR Zope.SiteErrorLog http://0.0.0.0/Zope2/sara/data/RudSoil_vs_lssu_160807.rma/PUT Traceback (innermost last): Module ZPublisher.Publish, line 119, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 42, in call_object Module webdav.NullResource, line 167, in PUT Module Products.Archetypes.WebDAVSupport, line 116, in PUT Module Products.Archetypes.utils, line 160, in mapply Module Products.Marshall.marshaller, line 99, in demarshall OverflowError: requested number of bytes is more than a Python string can hold -- 2009-12-04T12:34:57 ERROR ZServer uncaptured python exception, closing channel (socket.error:(9, 'Bad file descriptor') [/raid/opt/Python-2.4.6/lib/python2.4/asynchat.py|initiate_send|219] [/raid/opt/Python-2.4.6/lib/python2.4/asyncore.py|send|332] [/raid/opt/Python-2.4.6/lib/python2.4/socket.py|_dummy|136]) Best Regards, Jan ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
