[jira] [Commented] (JCLOUDS-1505) BlobStore.blobMetadata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServ
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948716#comment-16948716
]
Xavier BOURGOUIN commented on JCLOUDS-1505:
---
I've made a proposal fix for this issue:
https://github.com/apache/jclouds/pull/48
Could you please review it when time permits ?
Thanks!
Brgds,
Xavier
> BlobStore.blobMetadata(container, object) returns a StorageMetadata object
> with empty size when using
> org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule
> ---
>
> Key: JCLOUDS-1505
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1505
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-drivers
>Reporter: Xavier BOURGOUIN
>Priority: Major
> Attachments: jclouds_null_size_reproducer.tar.gz
>
>
> Hello,
>
> When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
> blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
> BlobMetadata object has a size attribute which is null.
> Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
> seems like it is reproducible with any other S3 provider, and potentially
> others OS providers like GCS, to be confirmed).
> In essence, the minimal reproducer pseudo-code is:
> {code:java}
> BlobStoreContext context = ContextBuilder.newBuilder(provider)
> .credentials(identity, credential)
> .endpoint(endpoint)
> // usage of ApacheHCHttp module
> .modules(ImmutableList. of(new
> ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
> .buildView(BlobStoreContext.class);
>
> try {
> BlobStore blobStore = context.getBlobStore();
> // output is: "File size from metadata: null"
> System.out.println("File size from metadata: " +
> blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
> } finally {
> context.close();
> }
> {code}
>
> For convenience, I've attached a ready to run reproducer to this ticket,
> here's how to run it:
>
> {code:bash}
> > mkdir reproducer
> > tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> > cd reproducer
> > mvn install
> # note: pre-req from this stage is to have an S3 Object Store available, you
> can use minio one if you have docker (docker run -it -p 9000:9000 -v
> /mnt/data:/data minio/minio server /data)
> # replace with your Object Store endpoint, identity and access key
> > java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> > http://10.66.33.1:9000
> File size from metadata: 8
> File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
> {code}
> The reproducer code has jclouds-wire logs enabled (inside log folder) and
> I've also added the log files inside the attached tarball for convenience. We
> see that in the HTTP HEAD response (the 2nd one, which correspond to my
> .blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
> Content-* headers are missing :
> {code:bash}
> >cat log/jclouds-wire.log
> # note: first .blobMetadata() call without
> ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
> headers
> 2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
> http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
> 8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
> MinIO/RELEASE.2019-07-24T02-02-23Z
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
> "35a41457219c775659b6018fb7f465a1-1"
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection:
> 1; mode=block
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
> Content-Security-Policy: block-all-mixed-content
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
> 31 Jul 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
> 15B687995977CC49
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Type:
> application/unknown
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Length: 8
> # note: second
[jira] [Updated] (JCLOUDS-1505) BlobStore.blobMetadata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServic
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xavier BOURGOUIN updated JCLOUDS-1505:
--
Summary: BlobStore.blobMetadata(container, object) returns a
StorageMetadata object with empty size when using
org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule
(was: BlobStore.blobMetata(container, object) returns a StorageMetadata object
with empty size when using
org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule)
> BlobStore.blobMetadata(container, object) returns a StorageMetadata object
> with empty size when using
> org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule
> ---
>
> Key: JCLOUDS-1505
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1505
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-drivers
>Reporter: Xavier BOURGOUIN
>Priority: Major
> Attachments: jclouds_null_size_reproducer.tar.gz
>
>
> Hello,
>
> When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
> blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
> BlobMetadata object has a size attribute which is null.
> Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
> seems like it is reproducible with any other S3 provider, and potentially
> others OS providers like GCS, to be confirmed).
> In essence, the minimal reproducer pseudo-code is:
> {code:java}
> BlobStoreContext context = ContextBuilder.newBuilder(provider)
> .credentials(identity, credential)
> .endpoint(endpoint)
> // usage of ApacheHCHttp module
> .modules(ImmutableList. of(new
> ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
> .buildView(BlobStoreContext.class);
>
> try {
> BlobStore blobStore = context.getBlobStore();
> // output is: "File size from metadata: null"
> System.out.println("File size from metadata: " +
> blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
> } finally {
> context.close();
> }
> {code}
>
> For convenience, I've attached a ready to run reproducer to this ticket,
> here's how to run it:
>
> {code:bash}
> > mkdir reproducer
> > tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> > cd reproducer
> > mvn install
> # note: pre-req from this stage is to have an S3 Object Store available, you
> can use minio one if you have docker (docker run -it -p 9000:9000 -v
> /mnt/data:/data minio/minio server /data)
> # replace with your Object Store endpoint, identity and access key
> > java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> > http://10.66.33.1:9000
> File size from metadata: 8
> File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
> {code}
> The reproducer code has jclouds-wire logs enabled (inside log folder) and
> I've also added the log files inside the attached tarball for convenience. We
> see that in the HTTP HEAD response (the 2nd one, which correspond to my
> .blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
> Content-* headers are missing :
> {code:bash}
> >cat log/jclouds-wire.log
> # note: first .blobMetadata() call without
> ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
> headers
> 2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
> http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
> 8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
> MinIO/RELEASE.2019-07-24T02-02-23Z
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
> "35a41457219c775659b6018fb7f465a1-1"
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection:
> 1; mode=block
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
> Content-Security-Policy: block-all-mixed-content
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
> 31 Jul 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
> 15B687995977CC49
>
[jira] [Updated] (JCLOUDS-1505) BlobStore.blobMetata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceM
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xavier BOURGOUIN updated JCLOUDS-1505:
--
Description:
Hello,
When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
BlobMetadata object has a size attribute which is null.
Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
seems like it is reproducible with any other S3 provider, and potentially
others OS providers like GCS, to be confirmed).
In essence, the minimal reproducer pseudo-code is:
{code:java}
BlobStoreContext context = ContextBuilder.newBuilder(provider)
.credentials(identity, credential)
.endpoint(endpoint)
// usage of ApacheHCHttp module
.modules(ImmutableList. of(new
ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
.buildView(BlobStoreContext.class);
try {
BlobStore blobStore = context.getBlobStore();
// output is: "File size from metadata: null"
System.out.println("File size from metadata: " +
blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
} finally {
context.close();
}
{code}
For convenience, I've attached a ready to run reproducer to this ticket, here's
how to run it:
{code:bash}
> mkdir reproducer
> tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> cd reproducer
> mvn install
# note: pre-req from this stage is to have an S3 Object Store available, you
can use minio one if you have docker (docker run -it -p 9000:9000 -v
/mnt/data:/data minio/minio server /data)
# replace with your Object Store endpoint, identity and access key
> java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> http://10.66.33.1:9000
File size from metadata: 8
File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
{code}
The reproducer code has jclouds-wire logs enabled (inside log folder) and I've
also added the log files inside the attached tarball for convenience. We see
that in the HTTP HEAD response (the 2nd one, which correspond to my
.blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
Content-* headers are missing :
{code:java}
>cat log/jclouds-wire.log
# note: first .blobMetadata() call without
ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
headers
2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection: 1;
mode=block
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
15B687995977CC49
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Type:
application/unknown
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Length: 8
# note: second call with ApacheHCHttpCommandExecutorServiceModule, the reponse
is missing the content-* headers
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,396
[jira] [Updated] (JCLOUDS-1505) BlobStore.blobMetata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceM
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xavier BOURGOUIN updated JCLOUDS-1505:
--
Description:
Hello,
When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
BlobMetadata object has a size attribute which is null.
Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
seems like it is reproducible with any other S3 provider, and potentially
others OS providers like GCS, to be confirmed).
In essence, the minimal reproducer pseudo-code is:
{code:java}
BlobStoreContext context = ContextBuilder.newBuilder(provider)
.credentials(identity, credential)
.endpoint(endpoint)
// usage of ApacheHCHttp module
.modules(ImmutableList. of(new
ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
.buildView(BlobStoreContext.class);
try {
BlobStore blobStore = context.getBlobStore();
// output is: "File size from metadata: null"
System.out.println("File size from metadata: " +
blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
} finally {
context.close();
}
{code}
For convenience, I've attached a ready to run reproducer to this ticket, here's
how to run it:
{code:bash}
> mkdir reproducer
> tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> cd reproducer
> mvn install
# note: pre-req from this stage is to have an S3 Object Store available, you
can use minio one if you have docker (docker run -it -p 9000:9000 -v
/mnt/data:/data minio/minio server /data)
# replace with your Object Store endpoint, identity and access key
> java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> http://10.66.33.1:9000
File size from metadata: 8
File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
{code}
The reproducer code has jclouds-wire logs enabled (inside log folder) and I've
also added the log files inside the attached tarball for convenience. We see
that in the HTTP HEAD response (the 2nd one, which correspond to my
.blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
Content-* headers are missing :
{code:bash}
>cat log/jclouds-wire.log
# note: first .blobMetadata() call without
ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
headers
2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection: 1;
mode=block
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
15B687995977CC49
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Type:
application/unknown
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Length: 8
# note: second call with ApacheHCHttpCommandExecutorServiceModule, the reponse
is missing the content-* headers
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,396
[jira] [Updated] (JCLOUDS-1505) BlobStore.blobMetata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceM
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xavier BOURGOUIN updated JCLOUDS-1505:
--
Description:
Hello,
When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
BlobMetadata object has a size attribute which is null.
Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
seems like it is reproducible with any other S3 provider, and potentially
others OS providers like GCS, to be confirmed).
In essence, the minimal reproducer pseudo-code is:
{code:java}
BlobStoreContext context = ContextBuilder.newBuilder(provider)
.credentials(identity, credential)
.endpoint(endpoint)
// usage of ApacheHCHttp module
.modules(ImmutableList. of(new
ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
.buildView(BlobStoreContext.class);
try {
BlobStore blobStore = context.getBlobStore();
// output is: "File size from metadata: null"
System.out.println("File size from metadata: " +
blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
} finally {
context.close();
}
{code}
For convenience, I've attached a ready to run reproducer to this ticket, here's
how to run it:
{code:shell}
> mkdir reproducer
> tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> cd reproducer
> mvn install
# note: pre-req from this stage is to have an S3 Object Store available, you
can use minio one if you have docker (docker run -it -p 9000:9000 -v
/mnt/data:/data minio/minio server /data)
# replace with your Object Store endpoint, identity and access key
> java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> http://10.66.33.1:9000
File size from metadata: 8
File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
{code}
The reproducer code has jclouds-wire logs enabled (inside log folder) and I've
also added the log files inside the attached tarball for convenience. We see
that in the HTTP HEAD response (the 2nd one, which correspond to my
.blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
Content-* headers are missing :
{code:java}
>cat log/jclouds-wire.log
# note: first .blobMetadata() call without
ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
headers
2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection: 1;
mode=block
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
15B687995977CC49
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Type:
application/unknown
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Length: 8
# note: second call with ApacheHCHttpCommandExecutorServiceModule, the reponse
is missing the content-* headers
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,396
[jira] [Updated] (JCLOUDS-1505) BlobStore.blobMetata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceM
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xavier BOURGOUIN updated JCLOUDS-1505:
--
Description:
Hello,
When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
BlobMetadata object has a size attribute which is null.
Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
seems like it is reproducible with any other S3 provider, and potentially
others OS providers like GCS, to be confirmed).
In essence, the minimal reproducer pseudo-code is:
{code:java}
BlobStoreContext context = ContextBuilder.newBuilder(provider)
.credentials(identity, credential)
.endpoint(endpoint)
// usage of ApacheHCHttp module
.modules(ImmutableList. of(new
ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
.buildView(BlobStoreContext.class);
try {
BlobStore blobStore = context.getBlobStore();
// output is: "File size from metadata: null"
System.out.println("File size from metadata: " +
blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
} finally {
context.close();
}
{code}
For convenience, I've attached a ready to run reproducer to this ticket, here's
how to run it:
{code:java}
> mkdir reproducer
> tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> cd reproducer
> mvn install
# note: pre-req from this stage is to have an S3 Object Store available, you
can use minio one if you have docker (docker run -it -p 9000:9000 -v
/mnt/data:/data minio/minio server /data)
# replace with your Object Store endpoint, identity and access key
> java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> http://10.66.33.1:9000
File size from metadata: 8
File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
{code}
The reproducer code has jclouds-wire logs enabled (inside log folder) and I've
also added the log files inside the attached tarball for convenience. We see
that in the HTTP HEAD response (the 2nd one, which correspond to my
.blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
Content-* headers are missing :
{code:java}
>cat log/jclouds-wire.log
# note: first .blobMetadata() call without
ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
headers
2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection: 1;
mode=block
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
15B687995977CC49
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Type:
application/unknown
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Length: 8
# note: second call with ApacheHCHttpCommandExecutorServiceModule, the reponse
is missing the content-* headers
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,396
[jira] [Created] (JCLOUDS-1505) BlobStore.blobMetata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceM
Xavier BOURGOUIN created JCLOUDS-1505:
-
Summary: BlobStore.blobMetata(container, object) returns a
StorageMetadata object with empty size when using
org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule
Key: JCLOUDS-1505
URL: https://issues.apache.org/jira/browse/JCLOUDS-1505
Project: jclouds
Issue Type: Bug
Components: jclouds-drivers
Reporter: Xavier BOURGOUIN
Attachments: jclouds_null_size_reproducer.tar.gz
Hello,
When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
StorageMetadata object has a size attribute which is null.
Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
seems like it is reproducible with any other S3 provider, and potentially
others OS providers like GCS, to be confirmed).
In essence, the minimal reproducer pseudo-code is:
{code:java}
BlobStoreContext context = ContextBuilder.newBuilder(provider)
.credentials(identity, credential)
.endpoint(endpoint)
// usage of ApacheHCHttp module
.modules(ImmutableList. of(new
ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
.buildView(BlobStoreContext.class);
try {
BlobStore blobStore = context.getBlobStore();
// output is: "File size from metadata: null"
System.out.println("File size from metadata: " +
blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
} finally {
context.close();
}
{code}
For convenience, I've attached a ready to run reproducer to this ticket, here's
how to run it:
{code:java}
> mkdir reproducer
> tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> cd reproducer
> mvn install
# note: pre-req from this stage is to have an S3 Object Store available, you
can use minio one if you have docker (docker run -it -p 9000:9000 -v
/mnt/data:/data minio/minio server /data)
# replace with your Object Store endpoint, identity and access key
> java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> http://10.66.33.1:9000 8A0CESVSLMU5HSSBIOB0
> SIdpPHZ3vEoLyqrL4f8maA9XqJVaKipMoElW0ssW
File size from metadata: 8
File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
{code}
The reproducer code has jclouds-wire logs enabled (inside log folder) and I've
also added the log files inside the attached tarball for convenience. We see
that in the HTTP HEAD response (the 2nd one, which correspond to my
.blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
Content-* headers are missing :
{code:java}
>cat log/jclouds-wire.log
# note: first .blobMetadata() call without
ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
headers
2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
MinIO/RELEASE.2019-07-24T02-02-23Z
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
"35a41457219c775659b6018fb7f465a1-1"
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection: 1;
mode=block
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
Content-Security-Policy: block-all-mixed-content
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
31 Jul 2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
15B687995977CC49
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Type:
application/unknown
2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Length: 8
# note: second call with ApacheHCHttpCommandExecutorServiceModule, the reponse
is missing the content-* headers
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> HEAD
http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
2019 15:30:13 GMT
2019-07-31 17:30:13,355 DEBUG [jclouds.headers] [main] >> Authorization: AWS
8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
2019-07-31 17:30:13,396 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
2019-07-31
