[
https://issues.apache.org/jira/browse/JCLOUDS-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francois Rigault updated JCLOUDS-1137:
--------------------------------------
Description:
When listing files in a local blobstore, we are sometimes hitting the following
exception:
{noformat}
java.nio.file.NoSuchFileException: /opt/data/./ggg/fff_local_db.db-wal
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.getBlob(FilesystemStorageStrategyImpl.java:373)
at
org.jclouds.blobstore.config.LocalBlobStore.loadBlob(LocalBlobStore.java:414)
...
Caused by: java.nio.file.NoSuchFileException:
/opt/data/./ggg/fff_local_db.db-wal
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixPath.openForAttributeAccess(UnixPath.java:787)
at
sun.nio.fs.LinuxUserDefinedFileAttributeView.list(LinuxUserDefinedFileAttributeView.java:100)
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.getBlob(FilesystemStorageStrategyImpl.java:333)
{noformat}
There is some kind of race condition when a file is being removed in the middle
of a getBlob: to reproduce, one can put a breakpoint in
FilesystemStorageStrategyImpl before the call to
UserDefinedFileAttributeView.list, and remove the current file from the file
system when hitting the breakpoint.
The workaround we are using is to provide our own version of
FileSystemStorageStrategyImpl that patches the getBlob method in this fashion:
{code:java}
@Override
public Blob getBlob(String container, String key) {
try {
return super.getBlob(container, key);
} catch (RuntimeException t) {
if (t.getCause() instanceof NoSuchFileException) {
log.warn("file `" + key + "' has been removed during the
getBlob", t);
return newBlob("deleted." + key + "." +
System.currentTimeMillis());
} else {
throw t;
}
}
}
{code}
This ensures a blob is returned. Returning null is not enough as
LocalBlobStore.list would thrown an exception "blob is not present although it
was in the list of container".
was:
When listing files in a local blobstore, we are sometimes hitting the following
exception:
{noformat}
java.nio.file.NoSuchFileException: /opt/data/./ggg/fff_local_db.db-wal
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.getBlob(FilesystemStorageStrategyImpl.java:373)
at
org.jclouds.blobstore.config.LocalBlobStore.loadBlob(LocalBlobStore.java:414)
...
Caused by: java.nio.file.NoSuchFileException:
/opt/data/./ggg/fff_local_db.db-wal
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixPath.openForAttributeAccess(UnixPath.java:787)
at
sun.nio.fs.LinuxUserDefinedFileAttributeView.list(LinuxUserDefinedFileAttributeView.java:100)
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.getBlob(FilesystemStorageStrategyImpl.java:333)
{noformat}
There is some kind of race condition when a file is being removed in the middle
of a getBlob: to reproduce, one can put a breakpoint in
FilesystemStorageStrategyImpl before the call to
UserDefinedFileAttributeView.list, and remove the current file from the file
system when hitting the breakpoint.
We are having this error multiple times a day and don't really know how to
proceed. Ideally, we would expect getBlob to return null in this case, instead
of throwing a NoSuchFileException.
> FilesystemStorageStrategyImpl throws NoSuchFileException when listing files
> ---------------------------------------------------------------------------
>
> Key: JCLOUDS-1137
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1137
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-core
> Affects Versions: 1.9.2
> Reporter: Francois Rigault
>
> When listing files in a local blobstore, we are sometimes hitting the
> following exception:
> {noformat}
> java.nio.file.NoSuchFileException: /opt/data/./ggg/fff_local_db.db-wal
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> at
> org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.getBlob(FilesystemStorageStrategyImpl.java:373)
> at
> org.jclouds.blobstore.config.LocalBlobStore.loadBlob(LocalBlobStore.java:414)
> ...
> Caused by: java.nio.file.NoSuchFileException:
> /opt/data/./ggg/fff_local_db.db-wal
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at sun.nio.fs.UnixPath.openForAttributeAccess(UnixPath.java:787)
> at
> sun.nio.fs.LinuxUserDefinedFileAttributeView.list(LinuxUserDefinedFileAttributeView.java:100)
> at
> org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.getBlob(FilesystemStorageStrategyImpl.java:333)
> {noformat}
> There is some kind of race condition when a file is being removed in the
> middle of a getBlob: to reproduce, one can put a breakpoint in
> FilesystemStorageStrategyImpl before the call to
> UserDefinedFileAttributeView.list, and remove the current file from the file
> system when hitting the breakpoint.
> The workaround we are using is to provide our own version of
> FileSystemStorageStrategyImpl that patches the getBlob method in this fashion:
> {code:java}
> @Override
> public Blob getBlob(String container, String key) {
> try {
> return super.getBlob(container, key);
> } catch (RuntimeException t) {
> if (t.getCause() instanceof NoSuchFileException) {
> log.warn("file `" + key + "' has been removed during the
> getBlob", t);
> return newBlob("deleted." + key + "." +
> System.currentTimeMillis());
> } else {
> throw t;
> }
> }
> }
> {code}
> This ensures a blob is returned. Returning null is not enough as
> LocalBlobStore.list would thrown an exception "blob is not present although
> it was in the list of container".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
