Re: [Dspace-tech] Using ftp to upload files into DSpace
Thanks to Emilio and helix for their answers. I'll consider to solve this using HTML5 File API. If I get some useful solutions, I'll share them of course... Regards, Pascal -Original Message- From: ivan.ma...@gmail.com [mailto:ivan.ma...@gmail.com] On Behalf Of helix84 Sent: Monday, April 29, 2013 12:44 PM To: Becker, Pascal-Nicolas Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Using ftp to upload files into DSpace Hi Pascal, the problem you're trying to cope with is that HTTP doesn't have a native way of resuming uploads (at least not that browsers actually implement). This is the only advantage of FTP over HTTP, other than that, FTP has only disadvantages - the need for an active port, unfamiliarity to most modern users. I'd argue that you don't want to use a different protocol, only a frontend that allows you to resume, show progress and upload multiple files in a series. This can be done using HTTP as the protocol - look at e.g. Gmail attachment or Youtube video uploads. Yes, this requires an extra programmatic support on both server and client side. There are quite a lot of existing solutions out there (here's a random example, although it doesn't have all the features: http://www.solmetra.com/en/disp.php/en_products/flash_uploader/uploa der_intro). But if you are going to solve this problem, why not solve it in an ideal way that will also benefit other DSpace users (assuming you'll share your work). If this is, for whatever reason, not the way you want to go, I can list the different ways how you can get bitstreams to DSpace (but all of them will require some customization of your DSpace). As a sidenote, here's a different solution actively being worked on: http://blogs.exeter.ac.uk/openexeterrdm/blog/2012/10/19/big-data- submission-tool-at-the-idcc/ This should be covered by the OR13 presentation Moving BIG DATA into DSpace by Lee Taylor from University of Exeter. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET Get 100% visibility into your production application - at no cost. Code-level diagnostics for performance bottlenecks with 2% overhead Download for free and get started troubleshooting in minutes. http://p.sf.net/sfu/appdyn_d2d_ap1 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
[Dspace-tech] Using ftp to upload files into DSpace
Hi, I'm looking for a possibility to load a lot of (50) small files or just some big (1 GB) files into an Item of DSpace. Loading them using a HTML form is not a good solution for both cases. Does one had the same problem already? Does anyone knows a solution for this? The best idea I had yet is to setup an ftp server on the same* server DSpace is running. When a user wants to publish data into DSpace he/she can either upload files using the web interface or may choose files out of his/her user folder on the ftp server. DSpace then uses these files as if they had been uploaded by the web interface. Before I start to write code I would like to know if anyone else has done something similarly or has a better idea for the problem mentioned above? Of course I won't use unencrypted ftp, but for the DSpace part it is not important if the files where uploaded using sftp or ftp-ssl. We are still setting up our DSpace repository, currently we are running tests using DSpace 3.1 and XMLUI. Regards, Pascal Becker *Either the ftp server and DSpace runs on the same machine or both uses the same network storage so that DSpace can read the files uploaded using the ftp server. -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Using ftp to upload files into DSpace
Hi Pascal-Nicolas With that aproach you will have to cope with the file Layer of Dspace, That is not obvious My suggestion is that you better try import via SIP or AIP. There are a number of small utilities (even based on excel) built around Dspace batch import function. https://wiki.duraspace.org/display/DSDOC18/Importing+and+Exporting+Items+via+Simple+Archive+Format Best Luck Emilio Becker, Pascal-Nicolas Hi, I'm looking for a possibility to load a lot of (50) small files or just some big (1 GB) files into an Item of DSpace. Loading them using a HTML form is not a good solution for both cases. Does one had the same problem already? Does anyone knows a solution for this? The best idea I had yet is to setup an ftp server on the same* server DSpace is running. When a user wants to publish data into DSpace he/she can either upload files using the web interface or may choose files out of his/her user folder on the ftp server. DSpace then uses these files as if they had been uploaded by the web interface. Before I start to write code I would like to know if anyone else has done something similarly or has a better idea for the problem mentioned above? Of course I won't use unencrypted ftp, but for the DSpace part it is not important if the files where uploaded using sftp or ftp-ssl. We are still setting up our DSpace repository, currently we are running tests using DSpace 3.1 and XMLUI. Regards, Pascal Becker *Either the ftp server and DSpace runs on the same machine or both uses the same network storage so that DSpace can read the files uploaded using the ftp server. -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Using ftp to upload files into DSpace
Hi Pascal, the problem you're trying to cope with is that HTTP doesn't have a native way of resuming uploads (at least not that browsers actually implement). This is the only advantage of FTP over HTTP, other than that, FTP has only disadvantages - the need for an active port, unfamiliarity to most modern users. I'd argue that you don't want to use a different protocol, only a frontend that allows you to resume, show progress and upload multiple files in a series. This can be done using HTTP as the protocol - look at e.g. Gmail attachment or Youtube video uploads. Yes, this requires an extra programmatic support on both server and client side. There are quite a lot of existing solutions out there (here's a random example, although it doesn't have all the features: http://www.solmetra.com/en/disp.php/en_products/flash_uploader/uploader_intro). But if you are going to solve this problem, why not solve it in an ideal way that will also benefit other DSpace users (assuming you'll share your work). If this is, for whatever reason, not the way you want to go, I can list the different ways how you can get bitstreams to DSpace (but all of them will require some customization of your DSpace). As a sidenote, here's a different solution actively being worked on: http://blogs.exeter.ac.uk/openexeterrdm/blog/2012/10/19/big-data-submission-tool-at-the-idcc/ This should be covered by the OR13 presentation Moving BIG DATA into DSpace by Lee Taylor from University of Exeter. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
