If you were getting similar exception given below with original AWS jars with 
Drill 1.11, you can set DRILLBIT_JAVA_OPTS as mentioned in the previous mail. 
Earlier, there were regions that support only V2 and setting this parameter may 
be problematic with those regions. Now I see that all the regions support V4. 
So setting this property should be fine.


-- Without 'com.amazonaws.services.s3.enableV4=true' set in dril-env.sh


0: jdbc:drill:schema=dfs> select * from s3.root.`0_0_0.parquet` limit 10;
Error: [MapR][DrillJDBCDriver](500165) Query execution error: SYSTEM ERROR: 
AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3, AWS Request ID: 
1A036C181EA7B8D4, AWS Error Code: null, AWS Error Message: Bad Request
[Error Id: bbe7d74a-b191-4ab2-99d7-10c5092c31d3 on arjun-lab-72:31010] 
(state=HY000,code=500165)
0: jdbc:drill:schema=dfs>


-- With 'com.amazonaws.services.s3.enableV4=true' set in dril-env.sh


"what ever the mind of man can conceive and believe, drill can query"
0: jdbc:drill:schema=dfs> select * from s3.root.`0_0_0.parquet` limit 10;
+-----+
| id  |
+-----+
| 1   |
+-----+
1 row selected (7.975 seconds)
0: jdbc:drill:schema=dfs>

https://docs.aws.amazon.com/general/latest/gr/rande.html



Thanks,


Arjun

________________________________
From: Arjun kr <arjun...@outlook.com>
Sent: Tuesday, February 13, 2018 8:47 PM
To: user@drill.apache.org
Subject: Re: S3 Connection Issues

Hi Anup,


Please see if below steps help.


1) Add below option $DRILL_HOME/conf/drill-env.sh


 export DRILLBIT_JAVA_OPTS="$DRILLBIT_JAVA_OPTS 
-Dcom.amazonaws.services.s3.enableV4=true"

2) Restart the drillbit service and try querying S4 region.

Thanks,

Arjun
________________________________
From: Anup Tiwari <anup.tiw...@games24x7.com>
Sent: Tuesday, February 13, 2018 3:16 PM
To: user@drill.apache.org
Subject: Re: S3 Connection Issues

Hi Padma,
As you have mentioned "Last time I tried, using Hadoop 2.8.1 worked for me." so
have you build drill with hadoop 2.8.1 ? If yes then can you provide steps ?
Since i have downloaded tar ball of 1.11.0 and replaced hadoop-aws-2.7.1.jar
with hadoop-aws-2.9.0.jar but still not able to query successfully to s3 bucket;
queries are going in starting state.
We are trying to query : "ap-south-1" region which supports only v4 signature.





On Thu, Oct 19, 2017 9:44 AM, Padma Penumarthy ppenumar...@mapr.com  wrote:
Which AWS region are you trying to connect to ?

We have a problem connecting to regions which support only v4 signature

since the version of hadoop we include in Drill is old.

Last time I tried, using Hadoop 2.8.1 worked for me.



Thanks

Padma





> On Oct 18, 2017, at 8:14 PM, Charles Givre <cgi...@gmail.com> wrote:

>

> Hello all,

> I’m trying to use Drill to query data in an S3 bucket and running into some
issues which I can’t seem to fix. I followed the various instructions online to
set up Drill with S3, and put my keys in both the conf-site.xml and in the
plugin config, but every time I attempt to do anything I get the following
errors:

>

>

> jdbc:drill:zk=local> show databases;

> Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon
S3, AWS Request ID: 56D1999BD1E62DEB, AWS Error Code: null, AWS Error Message:
Forbidden

>

>

> [Error Id: 65d0bb52-a923-4e98-8ab1-65678169140e on
charless-mbp-2.fios-router.home:31010] (state=,code=0)

> 0: jdbc:drill:zk=local> show databases;

> Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon
S3, AWS Request ID: 4D2CBA8D42A9ECA0, AWS Error Code: null, AWS Error Message:
Forbidden

>

>

> [Error Id: 25a2d008-2f4d-4433-a809-b91ae063e61a on
charless-mbp-2.fios-router.home:31010] (state=,code=0)

> 0: jdbc:drill:zk=local> show files in s3.root;

> Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon
S3, AWS Request ID: 2C635944EDE591F0, AWS Error Code: null, AWS Error Message:
Forbidden

>

>

> [Error Id: 02e136f5-68c0-4b47-9175-a9935bda5e1c on
charless-mbp-2.fios-router.home:31010] (state=,code=0)

> 0: jdbc:drill:zk=local> show schemas;

> Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon
S3, AWS Request ID: 646EB5B2EBCF7CD2, AWS Error Code: null, AWS Error Message:
Forbidden

>

>

> [Error Id: 954aaffe-616a-4f40-9ba5-d4b7c04fe238 on
charless-mbp-2.fios-router.home:31010] (state=,code=0)

>

> I have verified that the keys are correct but using the AWS CLI and downloaded
some of the files, but I’m kind of at a loss as to how to debug. Any
suggestions?

> Thanks in advance,

> — C







Regards,
Anup Tiwari

Sent with Mixmax

Reply via email to