So, Adam (and others), it goes like this: We have two properly configured Fedora 3.4 servers, 'test' and 'prod'. By that I mean the fcfg file has a proper address for the server, not just 'localhost'.
If, on my desktop machine's Fedora Java admin client, I log into test and try to do 'ingest > one object > from repository' from prod (giving correct passwords and asking for 'migrate' format) I get an 'HttpServiceNotFoundException' (full error listing below) for a metadata-only object (this for a 'managed' metadata datastream). If we use the admin client software actually on the test server, the exact same thing. If we use a browser on the test server to access http://....../fedora/search on prod and attempt to download the failing datastream or file from there - it works fine (which makes us think it isn't a firewall thing). The log on the sending server (prod) reports a successful export. If the attempted ingest involves a content datastream (which, for us means an MD5 checksum is set), we get a checksum failure rather than an HttpService error, but I'd like to bet that the checksum failure is actually because the file can't be retrieved. 'Prod' really is our repository production server so we know that we're starting with valid objects. The httpService error stack is below (I've just removed part of the server address for security). If I paste the full address from the error message into a browser I get an authentication challenge and then the datastream content. (That won't work for you because the Fedora server is behind the University firewall.) When we set up the prod server 10 weeks ago we tried to transfer content in the reverse direction (test to prod) but failed (we ended up copying the data directories and (re)building prod's Fedora over them - probably quicker anyway). I can't now swear we got the exact same error, but I'd be fairly certain. I'm very reluctant to try a 'test to prod' transfer now, precisely because it is the production server we're dealing with. Any insights welcome!!! ================================= org.fcrepo.server.errors.HttpServiceNotFoundException: [DefaultExternalContentManager] returned an error. The underlying error was a org.fcrepo .server.errors.GeneralException The message was "Error getting http://fedora-prod-vm.hull.ac.uk:8080/fedora/get/hull:4994/descMetadata/ 20 11-11-28T10:22:40.549Z" . at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.ja va:222) at org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.jav a:129) at org.apache.axis.encoding.DeserializationContext.endElement(Deserializati onContext.java:1087) at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDis patcher.dispatch(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unkno wn Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at javax.xml.parsers.SAXParser.parse(Unknown Source) at org.apache.axis.encoding.DeserializationContext.parse(DeserializationCon text.java:227) at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696) at org.apache.axis.Message.getSOAPEnvelope(Message.java:435) at org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstan dChecker.java:62) at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206) at org.apache.axis.client.Call.invokeEngine(Call.java:2784) at org.apache.axis.client.Call.invoke(Call.java:2767) at org.apache.axis.client.Call.invoke(Call.java:2443) at org.apache.axis.client.Call.invoke(Call.java:2366) at org.apache.axis.client.Call.invoke(Call.java:1812) at fedora.server.management.FedoraAPIMBindingSOAPHTTPStub.ingest(FedoraAPIM BindingSOAPHTTPStub.java:537) at fedora.client.APIMStubWrapper$1.construct(APIMStubWrapper.java:31) at fedora.client.SwingWorker$2.run(SwingWorker.java:131) at java.lang.Thread.run(Unknown Source) ___________________________________________________________________ Richard Green Consultant to Library & Learning Innovation, University of Hull managing the History DMP and Hydra (Hull) Projects http://hydra.hull.ac.uk http://hydrangeainhull.wordpress.com http://projecthydra.org http://historydmp.wordpress.com -----Original Message----- From: aj...@virginia.edu [mailto:aj...@virginia.edu] Sent: 29 November 2011 8:45 PM To: Support and info exchange list for Fedora users. Subject: Re: [fcrepo-user] Ingest Question Richard-- I'm frankly very dubious that you're being stupid. {grin} If you don't trace the problem to some kind of network issue, please let us have some stacktraces and FOXML. It's far from impossible that there is some kind of issue here that gets masked by other problems. If nothing else, perhaps we need to augment the documentation to explain more clearly that a "migration"-style FOXML serialization of an object has URL-dependencies on the repository from which it was exported... --- A. Soroka Online Library Environment the University of Virginia Library On Nov 29, 2011, at 2:17 PM, Richard Green wrote: > Rich > > That's exactly the error we've been seeing. Between the two of us I > think we have probably stumbled upon a real issue. Either that, or > we're both capable of being stupid - certainly not beyond the bounds > of possibility in my case!!! > > Richard > > -----Original Message----- > From: Burgis, Richard [mailto:burgi...@ais.msu.edu] > Sent: 29 November 2011 6:49 PM > To: Support and info exchange list for Fedora users. > Subject: Re: [fcrepo-user] Ingest Question > > Thanks. > > The error is > org.fcrepo.server.errors.HttpServiceNotFoundException:[DefaultExternal > Co ntentManager] returned an error. The underlying error was a > org.fcrepo.server.errors.GeneralException The message was "Error > getting > http://localhost:8080//fedora/get/uahc:AP00001/source/2011-10-19T18:14 > :3 > 4.840Z" > > > The Fedora log at this point just contains a Java exception dump that > starts out exactly as the above: > > ERROR 2011-11-29 13:42:36.394 [http-bio-8080-exec-4] > (FedoraAPIMBindingSOAPHTTPImpl) Error ingesting > org.fcrepo.server.errors.HttpServiceNotFoundException: > [DefaultExternalContentManager] returned an error. The underlying > error was a org.fcrepo.server.errors.GeneralException The message was > "Error getting > http://localhost:8080/fedora/get/uahc:AP00001/source/2011-10-19T18:14: > 34 > .840Z" . > at > org.fcrepo.server.storage.DefaultExternalContentManager.getExternalCon > te > nt(DefaultExternalContentManager.java:155) [fcrepo-server-3.5.jar:na] > at > org.fcrepo.server.storage.DefaultDOManager.doCommit(DefaultDOManager.j > av > a:1203) [fcrepo-server-3.5.jar:na] > at > org.fcrepo.server.storage.SimpleDOWriter.commit(SimpleDOWriter.java:50 > 9) > [fcrepo-server-3.5.jar:na] > at > org.fcrepo.server.management.DefaultManagement.ingest(DefaultManagement. > java:177) [fcrepo-server-3.5.jar:na] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) > [na:na] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > or > Impl.java:25) [na:1.6.0_25] > at java.lang.reflect.Method.invoke(Method.java:597) > [na:1.6.0_25] > at > org.fcrepo.server.messaging.NotificationInvocationHandler.invoke(Notif > ic > ationInvocationHandler.java:68) [fcrepo-server-3.5.jar:na] > at $Proxy5.ingest(Unknown Source) [na:na] > at > org.fcrepo.server.management.ManagementModule.ingest(ManagementModule. > ja > va:354) [fcrepo-server-3.5.jar:na] > at > org.fcrepo.server.management.FedoraAPIMBindingSOAPHTTPImpl.ingest(Fedo > ra > APIMBindingSOAPHTTPImpl.java:83) [fcrepo-server-3.5.jar:na] > at > org.fcrepo.server.management.FedoraAPIMBindingSOAPHTTPSkeleton.ingest( > Fe > doraAPIMBindingSOAPHTTPSkeleton.java:355) [fcrepo-common-3.5.jar:na] > at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) > [na:na] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > or > Impl.java:25) [na:1.6.0_25] > > There is a great deal more of course. > > Rich > -----Original Message----- > From: aj...@virginia.edu [mailto:aj...@virginia.edu] > Sent: Tuesday, November 29, 2011 1:30 PM > To: Support and info exchange list for Fedora users. > Subject: Re: [fcrepo-user] Ingest Question > > One natural question is why you are wanting to use file:// URLs? Is it > because you are dealing with very large datastreams (which is often > the motivation)? Or for some other reason? As for using http:// URLs > for external datastreams, it's not difficult to create objects that so do. > The pattern of URLs to use is essentially arbitrary, as long as Fedora > can dereference them when appropriate, and the specifics really depend > on your larger system design criteria. Keep in mind that when using > external datastreams, you give up some of Fedora's abilities to manage > that content, e.g. eventing via JMS or content versioning. > > As to the specific problem-- it would be helpful if you could provide > the section of Fedora's log in which the new repository fails to > retrieve the managed datastreams from the old repository. Then we > might be able to determine just why that's happening in your > situation. If it makes you feel any better about the time you are > spending with this problem, I move objects between repositories by > this means all the time, so it is possible, and there is some definite > reason it isn't working for you, and we can find out what that is and fix it. > > --- > A. Soroka > Online Library Environment > the University of Virginia Library > > > > > On Nov 29, 2011, at 1:20 PM, Burgis, Richard wrote: > >> Thanks. That's one of the things that I checked. It makes that much > more >> frustrating when I can get to the file via the web or a service. >> >> But this leads to a bigger question. My background is in big systems > and >> I'm feeling my way around in the new world. I read in the section >> that > >> Mr. Armintor kindly pointed out that it seems like a very bad idea to >> use file URI's for external datastreams. I'm am going to be using > those >> extensively in the future and would appreciate it if anyone could >> suggest how I configure my server (or create the URI's) so that I can >> use an http based URI. >> >> This is at a level of ignorance so high that I would think that a >> reference to a web site or book would be the easiest way to answer. >> >> Thank you very much >> Rich >> >> -----Original Message----- >> From: aj...@virginia.edu [mailto:aj...@virginia.edu] >> Sent: Tuesday, November 29, 2011 12:57 PM >> To: Support and info exchange list for Fedora users. >> Subject: Re: [fcrepo-user] Ingest Question >> >> If I understand you rightly you are using the "Migrate" style of > export, >> so that managed datastreams will be expressed in the exported FOXML >> as > >> URLs back into the original repository. If, by any chance, the > original >> repository URLs are inaccessible at the time of ingest (e.g., because > of >> XACML policy), you may see some funny behavior. It's something that > has >> bitten me before because of my own absent-mindedness and you might > want >> to check to make sure it's not happening to you. It's easy to check >> by > >> taking one of those URLs and retrieving it _from the machine on which >> the new repository is running_ right before you start the ingest, > using >> a tool like 'wget' or 'curl'. If this fails, it should at least give > you >> more information about why it's happening (whether you have an XACML >> policy problem or, like me, are a bit absentminded and prone to > turning >> things off without remembering you did {grin}). >> >> --- >> A. Soroka >> Online Library Environment >> the University of Virginia Library >> >> >> >>> They are both using Akubra module. From the logs it appears that the >> object was not in low level storage. >>> >>> I did try using the export file directly as ingest. That seemed the >> most straight forward way to do this. However it refers to locations > in >> the source repository for the managed datastreams and the ingest >> could > >> not find them. That's why I tried changing to a file URL. >>> >>> I have not changed the policy file to allow this. Thank you very >>> much >> this sounds promising. >>> >>> My other question relates to why I cannot ingest objects with >>> managed >> datastreams directly from the original repository. >>> >>> My biggest problem is that Fedora comes with a huge set of concepts > to >> digest all at once in support of it capabilities. Getting by that >> will > >> be the challenge. >>> >>> Thank you >>> Rich >>> -----Original Message----- >>> From: Benjamin Armintor [mailto:armin...@gmail.com] >>> Sent: Tuesday, November 29, 2011 12:25 PM >>> To: Support and info exchange list for Fedora users. >>> Subject: Re: [fcrepo-user] Ingest Question >>> >>> Also, just in case: >>> https://wiki.duraspace.org/display/FEDORA35/Using+File+URIs >>> >>> On Tue, Nov 29, 2011 at 12:19 PM, Benjamin Armintor >> <armin...@gmail.com> wrote: >>>> Richard (the first): >>>> Can you provide a bit more detail about the configuration of the >>>> two > >>>> fedoras (are they both using the same storage module?), what >>>> exactly > >>>> it is you're trying to (it sounds like you're trying to submit the >>>> exported foxml as the body of an ingest request), and the type of > 500 >>>> errors you're getting (or any information from the logs)? >>>> >>>> If you're changing the contentLocations to be file uris, did you > also >>>> update the relevant policy to allow resolution of those uris? >>>> >>>> - Ben >>>> >>>> On Tue, Nov 29, 2011 at 12:00 PM, Richard Green >>>> <r.gr...@hull.ac.uk> >> wrote: >>>>> Rich >>>>> >>>>> >>>>> >>>>> Don't assume that it's just you. This sounds rather familiar and >> we've been >>>>> wondering if it was *us*. Are you trying this with the Fedora >>>>> Java >> admin >>>>> client (ingest object(s) from repository)? If so - exactly what >> error >>>>> message do you get? What version of Fedora are you using? >>>>> >>>>> >>>>> >>>>> Richard >>>>> >>>>> __________________________________________________________________ >>>>> _ >>>>> >>>>> >>>>> >>>>> Richard Green >>>>> >>>>> Consultant to Library & Learning Innovation, University of Hull >>>>> >>>>> managing the History DMP and Hydra (Hull) Projects >>>>> >>>>> >>>>> >>>>> http://hydra.hull.ac.uk >>>>> >>>>> http://hydrangeainhull.wordpress.com >>>>> >>>>> http://projecthydra.org >>>>> >>>>> http://historydmp.wordpress.com >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> From: Burgis, Richard [mailto:burgi...@ais.msu.edu] >>>>> Sent: 29 November 2011 4:04 PM >>>>> To: Support and info exchange list for Fedora users. >>>>> Subject: [fcrepo-user] Ingest Question >>>>> >>>>> >>>>> >>>>> I'm moving a test repository and trying various methods to get it > to >> work. >>>>> >>>>> >>>>> >>>>> For objects with embedded content (X), I have no problem >>>>> ingesting >> from the >>>>> source repository. But when I try ingesting objects with Managed >> content >>>>> (M), I get errors saying that the managed content cannot be found. >>>>> >>>>> >>>>> >>>>> I tried exporting the objects and ingesting the exported objects, >> but I get >>>>> the same result. >>>>> >>>>> >>>>> >>>>> I modified the contentLocation to use file URLs pointing to the >> location of >>>>> the content in the file system. This time the ingest succeeded, >>>>> but >> I got >>>>> errors (500?) when I tried to edit or view the content. I got the >> same error >>>>> when I attempted to re-import it via the Admin program. I was >> finally able >>>>> to get the import to work after purging the items. >>>>> >>>>> >>>>> >>>>> This seems unreasonable as a process, so I would assume that I am >> missing >>>>> something critical. >>>>> >>>>> >>>>> >>>>> Any suggestions? >>>>> >>>>> Thanks >>>>> >>>>> Rich >>>>> >>>>> >>>>> >>>>> >> > ---------------------------------------------------------------------- > -- >> ------ >>>>> All the data continuously generated in your IT infrastructure >>>>> contains a definitive record of customers, application >>>>> performance, > >>>>> security threats, fraudulent activity, and more. Splunk takes this >>>>> data and makes sense of it. IT sense. And common sense. >>>>> http://p.sf.net/sfu/splunk-novd2d >>>>> _______________________________________________ >>>>> Fedora-commons-users mailing list >>>>> Fedora-commons-users@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users >>>>> >>>>> >>>> >>> >>> >> > ---------------------------------------------------------------------- > -- >> ------ >>> All the data continuously generated in your IT infrastructure >>> contains a definitive record of customers, application performance, >>> security threats, fraudulent activity, and more. Splunk takes this >>> data and makes sense of it. IT sense. And common sense. >>> http://p.sf.net/sfu/splunk-novd2d >>> _______________________________________________ >>> Fedora-commons-users mailing list >>> Fedora-commons-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users >>> >>> >> > ---------------------------------------------------------------------- > -- >> ------ >>> All the data continuously generated in your IT infrastructure >>> contains a definitive record of customers, application performance, >>> security threats, fraudulent activity, and more. Splunk takes this >>> data and makes sense of it. IT sense. And common sense. >>> http://p.sf.net/sfu/splunk-novd2d >>> _______________________________________________ >>> Fedora-commons-users mailing list >>> Fedora-commons-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users >> >> >> > ---------------------------------------------------------------------- > -- >> ------ >> All the data continuously generated in your IT infrastructure >> contains > >> a definitive record of customers, application performance, security >> threats, fraudulent activity, and more. Splunk takes this data and >> makes sense of it. IT sense. And common sense. >> http://p.sf.net/sfu/splunk-novd2d >> _______________________________________________ >> Fedora-commons-users mailing list >> Fedora-commons-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users >> >> > ---------------------------------------------------------------------- > -- > ------ >> All the data continuously generated in your IT infrastructure >> contains > >> a definitive record of customers, application performance, security >> threats, fraudulent activity, and more. Splunk takes this data and >> makes sense of it. IT sense. And common sense. >> http://p.sf.net/sfu/splunk-novd2d >> _______________________________________________ >> Fedora-commons-users mailing list >> Fedora-commons-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > > > ---------------------------------------------------------------------- > -- > ------ > All the data continuously generated in your IT infrastructure contains > a definitive record of customers, application performance, security > threats, fraudulent activity, and more. Splunk takes this data and > makes sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-novd2d > _______________________________________________ > Fedora-commons-users mailing list > Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > > ---------------------------------------------------------------------- > -- > ------ > All the data continuously generated in your IT infrastructure contains > a definitive record of customers, application performance, security > threats, fraudulent activity, and more. Splunk takes this data and > makes sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-novd2d > _______________________________________________ > Fedora-commons-users mailing list > Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > > ---------------------------------------------------------------------- > -------- All the data continuously generated in your IT infrastructure > contains a definitive record of customers, application performance, > security threats, fraudulent activity, and more. Splunk takes this > data and makes sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-novd2d > _______________________________________________ > Fedora-commons-users mailing list > Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users ------------------------------------------------------------------------ ------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d _______________________________________________ Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-users ------------------------------------------------------------------------------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d _______________________________________________ Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
