I opened an issue in the Akubra issue tracking system:
https://jira.duraspace.org/browse/AKUBRA-1
Issue number 1 :)
-- Scott
On 10/05/10, "Scott Prater" <pra...@wisc.edu> wrote:
> I think I may have found the source of the problem.
>
> In org.akubraproject.fs.FSBlob, this is what happens when an object gets
> moved:
>
> if (!file.renameTo(other)) {
> if (!file.exists())
> throw new MissingBlobException(getId());
>
> throw new IOException("Rename failed for an unknown reason.");
> }
>
> file.renameTo() is a java.io.File method that may or may not move files
> across symlinks and filesystems, depending on your platform (I'm running
> Solaris 10, so go figure):
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4073756
>
> Googling turns up a rich history of problems with the java.io.File renameTo()
> method when other filesystems and symlinks are involved. Best practice these
> days seems to be to use java.nio.file instead:
>
> http://download.oracle.com/javase/tutorial/essential/io/legacy.html
>
> Chris: shall I open to a bug report in the Akubra JIRA issue tracker?
>
> -- Scott
>
> On 10/05/10, "Scott Prater" <pra...@wisc.edu> wrote:
>
> > Another item of information: Ingesting FOXML files works without a
> > problem, induicating that this may be a problem with the copy/replace
> > functionality?
> >
> > Also, I simplified the process somewhat: really, the new data is spread
> > across three different filesystems, with the symlinks occuring at the
> > top-level hash character directories:
> >
> > /fedora/data/objects/0 -> /fedora/db01/objects/0
> > /fedora/data/objects/1 -> /fedora/db02/objects/1
> > etc.
> >
> > I mention this, as it occurs to me that perhaps Akubra, as part of its safe
> > move/rename function, is attempting to create temporary hard links across
> > filesystems? Just a guess...
> >
> > -- Scott
> >
> > On 10/05/10, "Scott Prater" <pra...@wisc.edu> wrote:
> >
> > > We encountered a worrisome problem today after filling up a filesystem
> > > and being forced to move our data to another filesystem. After the move
> > > and rebuild, we can no longer add datastreams to objects.
> > >
> > > I'm running Fedora 3.4, with the default Akubra low-level storage module.
> > >
> > > Here's what happened:
> > >
> > > 1. filesystem /fedora/data filled up, Fedora (and Tomcat) hung.
> > > 2. I shut down Fedora, Tomcat, Postgres.
> > > 3. I moved the directory /fedora/data/objects to another filesystem,
> > > /fedora/db01/objects.
> > > 4. I moved the directory /fedora/data/datastreams to another filesystem,
> > > /fedora/db01/datastreams.
> > > 5. I made symlinks from the original locations to the new locations:
> > > /fedora/data/objects -> /fedora/db01/objects
> > > /fedora/data/datastreams -> /fedora/db01/datastreams
> > > 6. I started up Postgres, and ran fedora-rebuild.sh twice, once to
> > > rebuild the database and once to rebuild the resource index. The rebuild
> > > completed successfully.
> > > 7. I then started up Tomcat + Fedora, and I was able to view all the
> > > objects.
> > > 8. I then did a test ingest of an object, and a stub object was
> > > successfully created, with a DC datastream. I verified that I could find
> > > the object and the datastream on the filesystem (both through the direct
> > > path and through the symlinked path). I verified that permsissions were
> > > correct at every directory level, and at the file level.
> > >
> > > I then try to add a datastream to the test object, and I receive this
> > > error (truncated):
> > >
> > > ERROR 2010-10-05 14:51:16.424 [http-9090-1] (WebApplicationImpl) Internal
> > > server error
> > > javax.ws.rs.WebApplicationException:
> > > org.fcrepo.server.errors.GeneralException: Unable to add or modify object
> > > (commit canceled)
> > > [...]
> > > Caused by: java.io.IOException: Rename failed for an unknown reason.
> > > at org.akubraproject.fs.FSBlob.moveTo(FSBlob.java:151)
> > > [akubra-fs-0.3.jar:na]
> > >
> > > I've tested this with the REST API, through the web admin interface, and
> > > through the java GUI admin tool, and the results are the same in all
> > > three cases. This error occurs regardless of whether I'm adding a
> > > datastream to a newly-created object or attempting to add a datastream to
> > > an older object.
> > >
> > > This may properly be an Akubra matter, but I thought I would bring it up
> > > here, first. Any suggestions or help would be greatly appreciated.
> > >
> > > thanks,
> > >
> > > -- Scott
> > >
> > > Following is the full stack trace:
> > >
> > > ERROR 2010-10-05 14:51:16.424 [http-9090-1] (WebApplicationImpl) Internal
> > > server error
> > > javax.ws.rs.WebApplicationException:
> > > org.fcrepo.server.errors.GeneralException: Unable to add or modify object
> > > (commit canceled)
> > > at
> > > org.fcrepo.server.rest.BaseRestResource.handleException(BaseRestResource.java:168)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.rest.DatastreamResource.addOrUpdateDatastream(DatastreamResource.java:583)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.rest.DatastreamResource.addDatastream(DatastreamResource.java:358)
> > > [fcrepo-server-3.4.jar:na]
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > [na:1.6.0_21]
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > [na:1.6.0_21]
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > [na:1.6.0_21]
> > > at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_21]
> > > at
> > > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:175)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:163)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:71)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:63)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:689)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:647)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:638)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:309)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:425)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at
> > > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:590)
> > > [jersey-bundle-1.0.3.1.jar:1.0.3.1]
> > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> > > [servlet-api.jar:na]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.fcrepo.server.security.servletfilters.FilterRestApiFlash.doFilter(FilterRestApiFlash.java:66)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.fcrepo.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:235)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.fcrepo.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:235)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.fcrepo.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:235)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.fcrepo.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:235)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:269)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:81)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> > > [catalina.jar:6.0.29]
> > > at
> > > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
> > > [tomcat-coyote.jar:6.0.29]
> > > at
> > > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> > > [tomcat-coyote.jar:6.0.29]
> > > at
> > > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> > > [tomcat-coyote.jar:6.0.29]
> > > at java.lang.Thread.run(Thread.java:619) [na:1.6.0_21]
> > > Caused by: org.fcrepo.server.errors.GeneralException: Unable to add or
> > > modify object (commit canceled)
> > > at
> > > org.fcrepo.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1438)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.storage.SimpleDOWriter.commit(SimpleDOWriter.java:508)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.management.DefaultManagement.addDatastream(DefaultManagement.java:572)
> > > [fcrepo-server-3.4.jar:na]
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > [na:1.6.0_21]
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > [na:1.6.0_21]
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > [na:1.6.0_21]
> > > at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_21]
> > > at
> > > org.fcrepo.server.messaging.NotificationInvocationHandler.invoke(NotificationInvocationHandler.java:68)
> > > [fcrepo-server-3.4.jar:na]
> > > at $Proxy0.addDatastream(Unknown Source) [na:na]
> > > at
> > > org.fcrepo.server.management.ManagementModule.addDatastream(ManagementModule.java:227)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.rest.DatastreamResource.addOrUpdateDatastream(DatastreamResource.java:467)
> > > [fcrepo-server-3.4.jar:na]
> > > ... 50 common frames omitted
> > > Caused by: org.fcrepo.common.FaultException: java.io.IOException: Rename
> > > failed for an unknown reason.
> > > at
> > > org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.rename(AkubraLowlevelStorage.java:363)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.safeOverwrite(AkubraLowlevelStorage.java:306)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.replace(AkubraLowlevelStorage.java:244)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.replaceObject(AkubraLowlevelStorage.java:150)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorageModule.replaceObject(AkubraLowlevelStorageModule.java:79)
> > > [fcrepo-server-3.4.jar:na]
> > > at
> > > org.fcrepo.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1334)
> > > [fcrepo-server-3.4.jar:na]
> > > ... 60 common frames omitted
> > > Caused by: java.io.IOException: Rename failed for an unknown reason.
> > > at org.akubraproject.fs.FSBlob.moveTo(FSBlob.java:151)
> > > [akubra-fs-0.3.jar:na]
> > > at
> > > org.akubraproject.map.IdMappingBlob.moveTo(IdMappingBlob.java:69)
> > > [akubra-map-0.3.jar:na]
> > > at
> > > org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.rename(AkubraLowlevelStorage.java:361)
> > > [fcrepo-server-3.4.jar:na]
> > > ... 65 common frames omitted
> > > --
> > > Scott Prater
> > > Library, Instructional, and Research Applications (LIRA)
> > > Division of Information Technology (DoIT)
> > > University of Wisconsin - Madison
> > > pra...@wisc.edu
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
