[Zope-dev] Re: serving BLOBs
Alan Milligan wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I know serving large BLOBs is rather frowned up using Z2, but I have a requirement to use the zope security machinery prior to serving a blob circa 500Mb. I thought I'd just wrap it in an ExternalFile and it would simply stream to the client. In fact I was even anticipating taking advantage of chunked transfer encoding for HTTP1.1 clients. However, I'm getting out of memory exceptions thrown by the server. This would indicate that no intermediate buffer flushing is occuring at all! What am I missing? I don't know the ExternalFile product, but Chris McDonough's presentation to the Plone conference in Vienna should be relevant: http://plope.com/Members/chrism/2004_plone_pres Also, you might want to check out Railroad, which is a different take on the problem: http://zope.org/Members/infrae/news/railroad_products_0_2 Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: serving BLOBs
What exactly is the filestream_iterator that the FileCacheManager uses? As I understand it this is a new feature? Johan -- Johan Carlsson Tel: + 46 8 31 24 94 Colliberty Mob: + 46 70 558 25 24 Torsgatan 72Email: [EMAIL PROTECTED] SE-113 37 STOCKHOLM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: serving BLOBs
Johan Carlsson wrote: What exactly is the filestream_iterator that the FileCacheManager uses? As I understand it this is a new feature? Of course the answer was in Chris' presentation you just sent :-) -- Johan Carlsson Tel: + 46 8 31 24 94 Colliberty Mob: + 46 70 558 25 24 Torsgatan 72Email: [EMAIL PROTECTED] SE-113 37 STOCKHOLM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: serving BLOBs
Johan Carlsson wrote: What exactly is the filestream_iterator that the FileCacheManager uses? As I understand it this is a new feature? For Zope = 2.7.2, ZPublisher defines an interface, ZPublisher.Iterators.IStreamIterator. An application can return an object implementing this interface to the publisher, instead of a string; if so, the publisher hands it directly to medusa, and frees up the application thread / database connection. medusa will use the iterator to send the response back to the user, without first converting it to a single big string. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: serving BLOBs
Tres Seaver wrote: Johan Carlsson wrote: What exactly is the filestream_iterator that the FileCacheManager uses? As I understand it this is a new feature? For Zope = 2.7.2, ZPublisher defines an interface, ZPublisher.Iterators.IStreamIterator. An application can return an object implementing this interface to the publisher, instead of a string; if so, the publisher hands it directly to medusa, and frees up the application thread / database connection. medusa will use the iterator to send the response back to the user, without first converting it to a single big string. That's actually brilliant. If I understand ExternalFile could benefit from using the IStreamIterator interface instead of RESPONSE.write? For instance, for someone that don't want to store blobs in the database temporarily :-) Johan -- Johan Carlsson Tel: + 46 8 31 24 94 Colliberty Mob: + 46 70 558 25 24 Torsgatan 72Email: [EMAIL PROTECTED] SE-113 37 STOCKHOLM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: serving BLOBs
Johan Carlsson wrote: Tres Seaver wrote: Johan Carlsson wrote: What exactly is the filestream_iterator that the FileCacheManager uses? As I understand it this is a new feature? For Zope = 2.7.2, ZPublisher defines an interface, ZPublisher.Iterators.IStreamIterator. An application can return an object implementing this interface to the publisher, instead of a string; if so, the publisher hands it directly to medusa, and frees up the application thread / database connection. medusa will use the iterator to send the response back to the user, without first converting it to a single big string. That's actually brilliant. If I understand ExternalFile could benefit from using the IStreamIterator interface instead of RESPONSE.write? For instance, for someone that don't want to store blobs in the database temporarily :-) Yup, that is what it is for. Chris has released a FileCacheManager which does this for objects which *are* stored in the ZODB; I think OFS.Image.File does try to work with the FileCacheManager on that. FileCacheManager is available at: http://www.dataflake.org/software/filecachemanager/ with CVS at: http://cvs.zope.org/Products/FileCacheManager/ Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
