Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: I've just installed Zope 2.8 with Apache 2 and I made a download library (a CMS site is too much for me and I can't find a simple "download area" product, so I've done it!) What about railorad: http://www.infrae.com/products/railroad How can i manage downloads uploads? In general it is bad idea to hold and serve large files directly from ZODB. This should be done by Apache etc. Large files aren't store in ZODB but in a shared folder connected with LOCALFS. Maybe Zope load a file in zodb before begin the download? (Are there other storage product that work only in zope - without plone?) Tramline is an Apache add-on but my problem still remain without Apache so this product can help me after I resolved the Zope issue. Thank you Illorca ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Large files aren't store in ZODB but in a shared folder connected with LOCALFS. Maybe Zope load a file in zodb before begin the download? (Are there other storage product that work only in zope - without plone?) Tramline is an Apache add-on but my problem still remain without Apache so this product can help me after I resolved the Zope issue. I think that if you'll use tramline you'll have no 'Zope issue' anymore. -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: Large files aren't store in ZODB but in a shared folder connected with LOCALFS. Maybe Zope load a file in zodb before begin the download? (Are there other storage product that work only in zope - without plone?) Tramline is an Apache add-on but my problem still remain without Apache so this product can help me after I resolved the Zope issue. I think that if you'll use tramline you'll have no 'Zope issue' anymore. Looking LocalFS specs I find the issue: This version of LocalFS takes advantanges of Zope 2.7.1 's new filestream_iterator feature and is believed to server large files better than before. However, my initial testing showed that it still doesn't prevent the whole file being loaded into memory. When I opened a URL pointing to a mpg file served by LocalFS, the file was loaded into memory and then immediately freeed twice. I think that's the problem... load 300Mb into memory Maybe I look for other fs products. Illorca ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Looking LocalFS specs I find the issue: This version of LocalFS takes advantanges of Zope 2.7.1 http://www.zope.org/Members/hewei/zoperpms/zope2.7 's new /filestream_iterator/ feature and is believed to server large files better than before. However, my initial testing showed that *it still doesn't prevent the whole file being loaded into memory*. When I opened a URL pointing to a mpg file served by LocalFS, the file was loaded into memory and then immediately freeed twice. I think that's the problem... load 300Mb into memory Maybe I look for other fs products. AFAIK it uses tempfile for serving such files. This allows Zope resources to be not blocked while returning big amounts of data. Although creation of tempfile or something like that (I'm not sure how this works in details) may still be slow. In general serving static files is best done by servers like Apache and I think this shouldn't be done by application server like Zope (possibly until it has blobs support). I don't understand why you can't use tramline. It may be connected with zope easily. It gives you very fast filesystem storage. Even more, seems for me that it is designed to do exacly what you're asking for. You say that tramline may help you after resolving zope issues, but I don't get it... By using tramline you will have no files in Zope (ZODB or LocalFS) so you will have no problem to eliminate. But maybe I missed something in your's use case. -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: Looking LocalFS specs I find the issue: This version of LocalFS takes advantanges of Zope 2.7.1 http://www.zope.org/Members/hewei/zoperpms/zope2.7 's new /filestream_iterator/ feature and is believed to server large files better than before. However, my initial testing showed that *it still doesn't prevent the whole file being loaded into memory*. When I opened a URL pointing to a mpg file served by LocalFS, the file was loaded into memory and then immediately freeed twice. I think that's the problem... load 300Mb into memory Maybe I look for other fs products. AFAIK it uses tempfile for serving such files. This allows Zope resources to be not blocked while returning big amounts of data. Although creation of tempfile or something like that (I'm not sure how this works in details) may still be slow. In general serving static files is best done by servers like Apache and I think this shouldn't be done by application server like Zope (possibly until it has blobs support). I don't understand why you can't use tramline. It may be connected with zope easily. It gives you very fast filesystem storage. Even more, seems for me that it is designed to do exacly what you're asking for. You say that tramline may help you after resolving zope issues, but I don't get it... By using tramline you will have no files in Zope (ZODB or LocalFS) so you will have no problem to eliminate. But maybe I missed something in your's use case. I want to use localfs instead of Apache becouse: 1) upload is done by web too 2) upload/download operations cause db operation (files names and descriptions are stored in sql server) Now I find a product (StreamingFS) that patch LocalFS for do this but doesn't work. At the moment I'm trying to fix its code. Illorca ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
I want to use localfs instead of Apache becouse: 1) upload is done by web too tramline supports uploads and downloads 2) upload/download operations cause db operation (files names and descriptions are stored in sql server) With tramline: The only thing the appserver sees is a unique identifier of the uploaded file. I don't know whether it is a file name, but seems that while uploading/downloading you may do some writes into db. Good luck! -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
I don't understand your paste. I'll try to describe better my situation - server A with Apache(80) and a rewrite rule to Zope(8080) - server B with a shared folder storage - *without any webserver* - server C with MS SQL Server - *without any webserver* - connected with ZODBC - firewall that send any incoming connection for www.mydomain.it to server A when protocol is http or https - in every server is active a ZOPEUSER (local administrator) with the same password. An incoming request to download will cause: - server A ask server C for filename - server A ask server B for file (using LocalFS object that stay in ZODB linked to server B's shared folder) I really can't understand how implement the same process using Apache for file storage. Please stay on zope list too! I've never used tramline, but after reading short description on their page seems that you'll have simpler configuration: - server A with Apache(and tramline) and a rewrite rule to Zope(8080) - server C with MS SQL Server - *without any webserver* - connected with ZODBC - firewall that send any incoming connection for www.mydomain.it to server A when protocol is http or https In general seems that instead of real, large files, app server receives only small files that contain only unique identifiers of these real ones. Real files are handled by tramline and stored in specific filesystem folders. When someone requests a file for download, application server sets special header that contains unique file identifier and this causes that tramline will return real, big file to the browser. Upload similarly uses headers to allow or disallow uploading. -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: I don't understand your paste. I'll try to describe better my situation - server A with Apache(80) and a rewrite rule to Zope(8080) - server B with a shared folder "storage" - *without any webserver* - server C with MS SQL Server - *without any webserver* - connected with ZODBC - firewall that send any incoming connection for www.mydomain.it to server A when protocol is http or https - in every server is active a ZOPEUSER (local administrator) with the same password. An incoming request to download will cause: - server A ask server C for filename - server A ask server B for file (using LocalFS object that stay in ZODB linked to server B's shared folder) I really can't understand how implement the same process using Apache for file storage. Please stay on zope list too! I've never used tramline, but after reading short description on their page seems that you'll have simpler configuration: - server A with Apache(and tramline) and a rewrite rule to Zope(8080) - server C with MS SQL Server - *without any webserver* - connected with ZODBC - firewall that send any incoming connection for www.mydomain.it to server A when protocol is http or https In general seems that instead of real, large files, app server receives only small files that contain only unique identifiers of these real ones. Real files are handled by tramline and stored in specific filesystem folders. When someone requests a file for download, application server sets special header that contains unique file identifier and this causes that tramline will return real, big file to the browser. Upload similarly uses headers to allow or disallow uploading. Sorry, I push the wrong button when I reply I'll try your configuration, but - where are files stored? (I dont't see server B in your post) - it seems I'll lose Zope User Restrictions ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Sorry, I push the wrong button when I reply I'll try your configuration, but - where are files stored? (I dont't see server B in your post) Read http://www.infrae.com/products/tramline There is description where files are stored - it seems I'll lose Zope User Restrictions Possibly not. This is application server that decides if user is allowed to upload/download file (by setting proper headers). -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Jary Busato schrieb: Maciej Wisniowski ha scritto: Large files aren't store in ZODB but in a shared folder connected with LOCALFS. Maybe Zope load a file in zodb before begin the download? (Are there other storage product that work only in zope - without plone?) Tramline is an Apache add-on but my problem still remain without Apache so this product can help me after I resolved the Zope issue. I think that if you'll use tramline you'll have no 'Zope issue' anymore. Looking LocalFS specs I find the issue: This version of LocalFS takes advantanges of Zope 2.7.1 http://www.zope.org/Members/hewei/zoperpms/zope2.7 's new /filestream_iterator/ feature and is believed to server large files better than before. However, my initial testing showed that *it still doesn't prevent the whole file being loaded into memory*. When I opened a URL pointing to a mpg file served by LocalFS, the file was loaded into memory and then immediately freeed twice. I think that's the problem... load 300Mb into memory Maybe I look for other fs products. However download of files off ZODB isnt such a big problem (you should mount an extra storage for the files to avoid overly cache trashing) but the performance is more then acceptable. ZODB/Filestorage is currely only a problem if you regulary update those files (or objects anyway) because it would add a complete new version. Regards Tino Wildenhain ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: Sorry, I push the wrong button when I reply I'll try your configuration, but - where are files stored? (I dont't see server B in your post) Read http://www.infrae.com/products/tramline There is description where files are stored - it seems I'll lose Zope User Restrictions Possibly not. This is application server that decides if user is allowed to upload/download file (by setting proper headers). [SOLVED] http://www.slinkp.com/code/zopestuff/blobnotes Your solution was good but I don't want to do all again so I tried all products mentioned above. The new version of ExtFile (1.54) have resolved the problem. It implements files download streaming as well as I want. Thank you Illorca ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
- Original Message - From: [EMAIL PROTECTED] To: zope zope@zope.org Sent: Tuesday, March 06, 2007 2:49 PM Subject: [Zope] Download Large Files in Zope Hi, I've just installed Zope 2.8 with Apache 2 and I made a download library (a CMS site is too much for me and I can't find a simple download area product, so I've done it!) Now, when I click on a large .exe file link (href) it seems it load all data before prompt for download and the zope service memory usage grow until 1/2 Gb. How can i manage downloads uploads? This is one possible solution: http://www.infrae.com/download/tramline Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
I've just installed Zope 2.8 with Apache 2 and I made a download library (a CMS site is too much for me and I can't find a simple download area product, so I've done it!) What about railorad: http://www.infrae.com/products/railroad How can i manage downloads uploads? In general it is bad idea to hold and serve large files directly from ZODB. This should be done by Apache etc. -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
What about railorad: http://www.infrae.com/products/railroad I meant tramline: http://www.infrae.com/products/tramline -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
