Hi Tony & Gary, Thanks very much - So...I had checked the permission on the files to be imported but not on the assetstore directory. As I'm looking at it, it appears that some of the earlier directories/files are owned by root - could be an issue. I need to re-import my files again so I'll change permissions and recreate my content files to add bundle:ORIGINAL, a suggestion emailed by Graeme Fox. Thanks! Susan
Tony Kyne wrote: > Gary > > To be pedantically correct, I should have said the "store_id" field of the > "bitstream" > table. It seems to be "randomly" generated - its is a key to the bucket > where the data > is kept. And a store_id number translates into a directory path to the > bitstream: > > So a store_id of 129683849070764175216956097535764632127 means that the > pathname to > the object in the assetstore is > > assetstore/12/96/83/129683849070764175216956097535764632127 > That is, the first three tuples of the store number become subdirectory paths > in the > assetstore > > And it appears that if you try different times with the same entry, you > will get a > different store number. I am willing to bet that this is all designed to > spread the > object around the buckets efficiently to speed access (computer science data > structure > and storage lectures from uni courses decades ago are being salvaged from > deep memory in > the brain here....) > > And so Susan's problems probably stem from ownerships and permissions of > the creation > of the original three tuples of directory path from previous loads by > processes with > other ownership and permissions. > > Happy sleeping tonight ... > > Gary Browne wrote: > >> Hang on a minute... >> >> Tony Kyne wrote: >> "From what we can gather, >> the bitstream identifier created has some random aspect to it." >> >> >> Are you talking about the sequence ID for the bitstream, or the bundle's >> handle? I >> > know I'm obsessed with the sequence ID generation process but my belief was > that it is > (somehow) sequentially generated (hence the name, I guess). > >> Can anyone confirm or deny so I can sleep tonight? >> >> Cheers >> Gary >> >> >> Gary Browne >> Development Programmer >> Library IT Services >> University of Sydney >> Australia >> ph: 61-2-9351 5946 >> >> -----Original Message----- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of Tony Kyne >> Sent: Tuesday, 8 May 2007 9:44 AM >> To: susan rector >> Cc: [email protected]; George Kozak >> Subject: Re: [Dspace-tech] help with import - bitstream error >> >> Susan and others >> >> We had that problem with an 11,000+ upload. Drove us spare - we even >> thought the >> there was a hardware problem given the random nature of the problem: it >> bombs out >> with no such file error, but immediately on restart it works. Even tried >> different >> version of Java, operating system patches etc,etc >> >> However, we ultimately worked it out what the problem was. From what >> we can gather, >> the bitstream identifier created has some random aspect to it. The lead >> numbers of >> this identifier give the start of the directory path in the assetstore to >> store the >> images etc. And so if you upload a file on two separate occassions, you get >> different magic numbers and hence different directory paths into the >> assetstore. So >> the problem is directory paths. And if you have the same problem as we had, >> bits of >> the some of the directory paths existed before the current upload, and thay >> had >> owenerships and permissions that the owner of the current upload process did >> not have >> access to: the "No such file..." error, is really saying "Permissions for >> the >> required directory access not valid ...." [ or from our scouring the web, it >> is the >> Java catch all error message for any sort of possible file access problems] >> >> So look at the ownership and permissions of the various bits of your >> assetstore. In >> our case, the problem was a few test examples we did way back at the start >> of setting >> up and testing Dspace, and these were out of sync with our current >> development >> environment ownership/permission regime. >> >> susan rector wrote: >> >>> Thanks George, >>> I just got them all in the system (467). It seemed to bomb out when >>> >> the >> >>> directory had a lot of images or .mov files. I think for our >>> >> production >> >>> version, I'm going to just import by year from etd-db so that I can avoid >>> the >>> > continuous restart of of the program > >>> thanks again, >>> Susan >>> >>> George Kozak wrote: >>> >>>> Susan: >>>> >>>> I have run into this problem running big batches. Every so often, >>>> >> the >> >>>> batch importer comes up with the error (at least for me). If I then batch >>>> upload >>>> > with smaller groups of items (without making other changes) everything runs > fine. > >>>> At 11:38 AM 5/7/2007, susan rector wrote: >>>> >>>>> Hi all, >>>>> I fixed the last error with the ^M - thanks for all your help >>>>> >>>>> Now, I'm running into another strange import error. I run the dspace >>>>> importer in >>>>> > test mode and it works fine and imports 466 distinct directories with > metadata and > files. > >>>>> Test mode: /dspace/bin/dsrun org.dspace.app.itemimport.ItemImport >>>>> >> --test >> >>>>> --add [EMAIL PROTECTED] --collection=5 >>>>> --source=/usr/local/archive_directory --mapfile mapfile.txt >>>>> >>>>> When I run the same command just not in test mode, I get the >>>>> >> following >> >>>>> error stating that the contents file isn't there. In fact, it is >>>>> >> there >> >>>>> and has a listing of the files in that directory >>>>> Add: /dspace/bin/dsrun org.dspace.app.itemimport.ItemImport --add >>>>> > [EMAIL PROTECTED] --collection=5 > >>>>> --source=/usr/local/archive_directory --mapfile mapfile.txt >>>>> >>>>> I've changed all the file permissions thinking that might be it - >>>>> >> didn't >> >>>>> work. It seems to get stuck on the jpeg's. >>>>> >>>>> Thanks for any suggestions! >>>>> Susan Rector >>>>> [EMAIL PROTECTED] >>>>> >>>>> >>>>> Error: >>>>> Processing contents file: >>>>> /usr/local/archive_directory/etd-05152006-125030/contents >>>>> Bitstream: benitezjm_image06.jpg >>>>> Bitstream: benitezjm_thesis.pdf >>>>> Bitstream: benitezjm_image04.jpg >>>>> Bitstream: benitezjm_image02.jpg >>>>> java.io.IOException: No such file or directory >>>>> at java.io.UnixFileSystem.createFileExclusively(Native >>>>> >> Method) >> >>>>> at java.io.File.createNewFile(File.java:883) >>>>> at >>>>> edu.sdsc.grid.io.local.LocalFile.createNewFile(LocalFile.java:486) >>>>> at >>>>> >>>>> >> org.dspace.storage.bitstore.BitstreamStorageManager.store(BitstreamStora >> > geManager.java:332) > >>>>> at org.dspace.content.Bitstream.create(Bitstream.java:184) at >>>>> >> org.dspace.content.Bundle.createBitstream(Bundle.java:362) >> >>>>> at >>>>> >>>>> >> org.dspace.app.itemimport.ItemImport.processContentFileEntry(ItemImport. >> java:1103) >> >>>>> at >>>>> >>>>> >> org.dspace.app.itemimport.ItemImport.processContentsFile(ItemImport.java >> :1030) >> >>>>> at >>>>> org.dspace.app.itemimport.ItemImport.addItem(ItemImport.java:631) >>>>> at >>>>> org.dspace.app.itemimport.ItemImport.addItems(ItemImport.java:498) >>>>> at >>>>> org.dspace.app.itemimport.ItemImport.main(ItemImport.java:407) >>>>> java.io.IOException: >>>>> > No such file or directory > >>>>> >>>>> Contents file: >>>>> benitezjm_image06.jpg >>>>> benitezjm_thesis.pdf >>>>> benitezjm_image04.jpg >>>>> benitezjm_image02.jpg >>>>> benitezjm_image05.jpg >>>>> benitezjm_image01.jpg >>>>> benitezjm_image03.jpg >>>>> >>>>> >>>>> >>>>> >> ------------------------------------------------------------------------ - >> >>>>> This SF.net email is sponsored by DB2 Express >>>>> Download DB2 Express C - the FREE version of DB2 express and take control >>>>> of your >>>>> > XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > >>>>> _______________________________________________ >>>>> DSpace-tech mailing list >>>>> [email protected] >>>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech >>>>> >>>> *************************** >>>> George Kozak >>>> Coordinator >>>> Web Development and Management >>>> Digital Media Group >>>> 501 Olin Library >>>> Cornell University >>>> 607-255-8924 >>>> *************************** >>>> [EMAIL PROTECTED] >>>> >>> >>> >> ------------------------------------------------------------------------ - >> >>> This SF.net email is sponsored by DB2 Express >>> Download DB2 Express C - the FREE version of DB2 express and take control >>> of your >>> > XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > >>> _______________________________________________ >>> DSpace-tech mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/dspace-tech >>> >>> >> regards, Tony >> >> ======================================================================== === >> Dr. Tony Kyne, Head, Information Technology Services, >> The Walter and Eliza Hall Institute of Medical Research, 1G >> Royal >> > Parade, Parkville, Victoria, 3050, Australia. > >> Phone: International +61-3-9345-2586 FAX: International >> +61-3-9347-0852 >> National 03-9345-2586 National >> 03-9347-0852 >> Email: [EMAIL PROTECTED] >> ======================================================================== == >> >> >> ------------------------------------------------------------------------ - >> This SF.net email is sponsored by DB2 Express >> Download DB2 Express C - the FREE version of DB2 express and take control of >> your XML. >> > No limits. Just data. Click to get it now. > >> http://sourceforge.net/powerbar/db2/ >> _______________________________________________ >> DSpace-tech mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/dspace-tech >> >> ------------------------------------------------------------------------- >> This SF.net >> > email is sponsored by DB2 Express > >> Download DB2 Express C - the FREE version of DB2 express and take control of >> your XML. >> > No limits. Just data. Click to get it now. > >> http://sourceforge.net/powerbar/db2/ >> _______________________________________________ >> DSpace-tech mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/dspace-tech >> >> > > > regards, Tony > > =========================================================================== > Dr. Tony > Kyne, Head, Information Technology Services, > The Walter and Eliza Hall Institute of Medical Research, 1G > Royal Parade, > Parkville, Victoria, 3050, Australia. > Phone: International +61-3-9345-2586 FAX: International +61-3-9347-0852 > National 03-9345-2586 National 03-9347-0852 > Email: [EMAIL PROTECTED] > ========================================================================== > > > > regards, Tony > > =========================================================================== > Dr. Tony Kyne, Head, Information Technology Services, > The Walter and Eliza Hall Institute of Medical Research, > 1G Royal Parade, Parkville, Victoria, 3050, Australia. > Phone: International +61-3-9345-2586 FAX: International +61-3-9347-0852 > National 03-9345-2586 National 03-9347-0852 > Email: [EMAIL PROTECTED] > ========================================================================== > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > DSpace-tech mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/dspace-tech > ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ DSpace-tech mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspace-tech
