I opened a pull request:
https://github.com/apache/jclouds/pull/94
On Fri, Jan 08, 2021 at 10:01:56AM +0900, Andrew Gaul wrote:
> Hi John, the code in LocalBlobStore.putBlob looks like this:
>
> String eTag = storageStrategy.putBlob(containerName, blob);
> setBlobAccess(containerName, blobKey, options.getBlobAccess());
>
> I suspect that the object is being removed by another client in between
> these calls. Instead FilesystemStorageStrategyImp.putBlob should allow
> an options parameter to set the blobAccess on the temporary file which
> then gets renamed to the real file name. Would you like to submit a PR
> for this?
>
> On Thu, Jan 07, 2021 at 10:09:29PM -0000, John Calcote wrote:
> > We use the filesystem (LocalBlobStore) implementation for integration
> > testing in our code.
> >
> > I'm seeing the following behavior in version 2.2.1 which I recently
> > upgraded to:
> >
> > ```
> > 2021-01-07 20:03:43.503,7964896021968234 {} DEBUG
> > c.p.d.m.j.AbstractUploadCallable [cpu-normalpri-1] [task 1] State change:
> > (data) chunk [0-1048575] - uploading
> > 2021-01-07 20:03:43.575,7964896093429743 {} DEBUG
> > o.j.f.s.i.FilesystemStorageStrategyImpl [cpu-normalpri-1] Creating
> > container test-container
> > 2021-01-07 20:03:43.575,7964896093478721 {} DEBUG
> > o.j.f.s.i.FilesystemStorageStrategyImpl [cpu-normalpri-1] Creating
> > directory //tmp/16100498223118761388361431415820/test-container
> > 2021-01-07 20:03:43.575,7964896093881924 {} DEBUG
> > c.p.d.m.j.AbstractUploadCallable [cpu-normalpri-1] [task 1] State change:
> > (data) chunk [1048576-2097151] - checksummed
> > checksum=D8ACDD71E006E090F4FBF32CAF0627A3
> > 2021-01-07 20:03:43.575,7964896093909591 {} DEBUG
> > c.p.d.m.j.AbstractUploadCallable [cpu-normalpri-1] [task 1] State change:
> > (data) chunk [1048576-2097151] - uploading
> > ..
> > 2021-01-07 20:03:43.576,7964896094167857 {} DEBUG o.j.b.c.LocalBlobStore
> > [cloud-normalpri-0] Put blob with key [byHashV1/0001000000000010000020
> > B3A2D81C390E0531DBCF0DEC082C4CA96D26F26AA9A26A26E80B5F00FA9F48E3 ] to
> > container [test-container]
> > ..
> > 2021-01-07 20:03:43.577,7964896095288960 {} DEBUG o.j.b.c.LocalBlobStore
> > [cloud-normalpri-0] Put blob with key [byHashV1/0001000000000010000020
> > B3A2D81C390E0531DBCF0DEC082C4CA96D26F26AA9A26A26E80B5F00FA9F48E3 ] to
> > container [test-container]
> > ..
> > 2021-01-07 20:03:43.584,7964896102791478 {} DEBUG o.j.b.c.LocalBlobStore
> > [cloud-normalpri-1] Put blob with key [byHashV1/0001000000000010000020
> > B3A2D81C390E0531DBCF0DEC082C4CA96D26F26AA9A26A26E80B5F00FA9F48E3 ] to
> > container [test-container]
> > ..
> > 2021-01-07 20:03:43.597,7964896115564074 {} DEBUG o.j.b.c.LocalBlobStore
> > [cloud-normalpri-1] Put blob with key [byHashV1/0001000000000010000020
> > B3A2D81C390E0531DBCF0DEC082C4CA96D26F26AA9A26A26E80B5F00FA9F48E3 ] to
> > container [test-container]
> > ..
> > 2021-01-07 20:03:43.605,7964896123355487 {} ERROR
> > c.p.d.m.j.AbstractUploadCallable [cloud-normalpri-0] [task 1] Upload
> > failed: chunk [1048576-2097151] - uploading
> > java.lang.RuntimeException: java.nio.file.NoSuchFileException:
> > /tmp/16100498223118761388361431415820/test-container/byHashV1/0001000000000010000020
> > B3A2D81C390E0531DBCF0DEC082C4CA96D26F26AA9A26A26E80B5F00FA9F48E3
> > at com.google.common.base.Throwables.propagate(Throwables.java:240)
> > ~[guava-21.0.jar:?]
> > at
> > org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.setBlobAccess(FilesystemStorageStrategyImpl.java:694)
> > ~[filesystem-2.2.1.jar:2.2.1]
> > at
> > org.jclouds.blobstore.config.LocalBlobStore.setBlobAccess(LocalBlobStore.java:409)
> > ~[jclouds-blobstore-2.2.1.jar:2.2.1]
> > at
> > org.jclouds.blobstore.config.LocalBlobStore.putBlob(LocalBlobStore.java:794)
> > ~[jclouds-blobstore-2.2.1.jar:2.2.1]
> > at
> > org.jclouds.blobstore.config.LocalBlobStore.putBlob(LocalBlobStore.java:533)
> > ~[jclouds-blobstore-2.2.1.jar:2.2.1]
> > at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> > ~[?:?]
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > ~[?:1.8.0_181]
> > at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
> > at
> > com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37)
> > ~[guice-3.0.jar:?]
> > at com.sun.proxy.$Proxy66.putBlob(Unknown Source) ~[?:?]
> > at
> > com.hammerspace.csp.rest.client.jclouds.JCloudsCspRestClient$1UploadRunnable.run(JCloudsCspRestClient.java:454)
> > [csp-rest-client-4.5.1.jar:?]
> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > [?:1.8.0_181]
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > [?:1.8.0_181]
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > [?:1.8.0_181]
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > [?:1.8.0_181]
> > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
> > Caused by: java.nio.file.NoSuchFileException:
> > /tmp/16100498223118761388361431415820/test-container/byHashV1/0001000000000010000020
> > B3A2D81C390E0531DBCF0DEC082C4CA96D26F26AA9A26A26E80B5F00FA9F48E3
> > at
> > sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> > ~[?:1.8.0_181]
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> > ~[?:1.8.0_181]
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> > ~[?:1.8.0_181]
> > at
> > sun.nio.fs.UnixFileAttributeViews$Posix.readAttributes(UnixFileAttributeViews.java:218)
> > ~[?:1.8.0_181]
> > at
> > sun.nio.fs.UnixFileAttributeViews$Posix.readAttributes(UnixFileAttributeViews.java:131)
> > ~[?:1.8.0_181]
> > at
> > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
> > ~[?:1.8.0_181]
> > at
> > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
> > ~[?:1.8.0_181]
> > at java.nio.file.Files.readAttributes(Files.java:1737)
> > ~[?:1.8.0_181]
> > at java.nio.file.Files.getPosixFilePermissions(Files.java:2004)
> > ~[?:1.8.0_181]
> > at
> > org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.setBlobAccess(FilesystemStorageStrategyImpl.java:686)
> > ~[filesystem-2.2.1.jar:2.2.1]
> > ... 14 more
> > ```
> >
> > JClouds seems to be having trouble reading the existing POSIX attributes on
> > the file it just wrote (so it can modify them).
> >
> > Has anyone else seen this, or have there been changes between 2.1 and 2.2.0
> > to the filesystem implementation?
--
Andrew Gaul
http://gaul.org/
