Gautam,
Thanks for your response, S3 plugin config is below.
I am not sure if there is an property in fs.s3a.* to set the signature
version ( in core-site.xml). Any recommendation on how to apply it.
{
"type": "file",
"enabled": true,
"connection": "s3a://abcd1234/",
"config": null,
"workspaces": {
"root": {
"location": "/EMR/Data/Test/",
"writable": false,
"defaultInputFormat": null
},
"omniture": {
"location": "/EMR/Data/Omniture_parquet/Omniture_global_data",
"writable": false,
"defaultInputFormat": null
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null
}
},
"formats": {
"psv": {
"type": "text",
"extensions": [
"tbl"
],
"delimiter": "|"
},
"csv": {
"type": "text",
"extensions": [
"csv"
],
"delimiter": ","
},
"tsv": {
"type": "text",
"extensions": [
"tsv"
],
"delimiter": "\t"
},
"httpd": {
"type": "httpd",
"logFormat": "%h %t \"%r\" %>s %b \"%{Referer}i\"",
"timestampFormat": null
},
"parquet": {
"type": "parquet"
},
"json": {
"type": "json",
"extensions": [
"json"
]
},
"avro": {
"type": "avro"
},
"sequencefile": {
"type": "sequencefile",
"extensions": [
"seq"
]
},
"csvh": {
"type": "text",
"extensions": [
"csvh"
],
"extractHeader": true,
"delimiter": ","
}
}
}
On Tue, May 23, 2017 at 7:48 PM, Gautam Parai <[email protected]> wrote:
> Hi Chandrakanth,
>
>
> A quick google got me this result from AWS:
>
>
> "We are sorry you have trouble getting this to work. The error is
> consistent with KMS requiring SigV4. We are still looking at why, by
> default, the s3 client for node.js doesn't do that. In the interim, you can
> explicitly set the signatureVersion when creating the s3 client object as
> follows:
>
> var s3Obj = new AWS.S3({maxRetries: 10, "signatureVersion":"v4"});
>
> Please let us know if this fixed or didn't fix your problem."
>
> Can you try setting an equivalent S3 storage plugin configuration for
> `signatureVersion`? Please see here for an example
> https://drill.apache.org/docs/s3-storage-plugin/
>
> Also, it would be good if you could share your S3 storage plugin
> configuration here.
>
>
> Thanks,
>
> Gautam
>
> ________________________________
> From: Chandrakanth Akkinepalli <[email protected]>
> Sent: Tuesday, May 23, 2017 1:34:25 PM
> To: [email protected]
> Subject: querying aws s3 objects that are server side encrypted
>
> Hi All,
> I have configured aws s3 storage plugin to access data in my s3 bucket.
>
> I have placed both un-encrypted and aws-kms sse objects in the bucket.
>
> I am able to query the un-encrypted objects just fine via apache drill.
> But i am getting following error when querying aws-kms encrypted objects.
>
> SQL Error: VALIDATION ERROR: Status Code: 400, AWS Service: Amazon S3, AWS
> Request ID: ABCDEFGH12345, AWS Error Code: InvalidArgument, AWS Error
> Message: Requests specifying Server Side Encryption with AWS KMS managed
> keys require AWS Signature Version 4.
>
> SQL Query null
>
>
> Did anyone else face this issue? How do i resolve it?
>
> Thanks,
> Chandra
>
