Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

2018-04-12 Thread Anup Tiwari
We are not using EMR. Also we have set below params for accessing s3 bucket in hive-site.xml which are same as what we have set in hive 2.1.1. - fs.s3a.access.key - fs.s3a.secret.key - fs.s3a.connection.maximum - fs.s3a.impl Regards, Anup Tiwari On Thu, Apr 12, 2018 at 7:19 PM,

Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

2018-04-12 Thread Anup Tiwari
Hi Richard, After looking at hive *2.3.3* logs i found that when we are loading all configuration parameters then below message is getting printed :- Values omitted for security reason if present: [fs.s3n.awsAccessKeyId, fs.s3a.access.key, fs.s3.awsAccessKeyId, hive.server2.keystore.password, f

Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

2018-04-12 Thread Richard A. Bross
I hear you, but given the exception log, it does seem that it can't authenticate you. You can try using the AWS environment variables. If that resolves the issue then you'll have some more to go on. According to Hortonworks here: https://hortonworks.github.io/hdp-aws/s3-security/ "AWS CLI

Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

2018-04-12 Thread Anup Tiwari
Hi All, When i am trying to read s3 linked external table in 2.3.3 ; i am getting errors.It was working properly in 2.1.1. please find below details and let me know if i am missing something :- *Hadoop Version :- *2.8.0 *Query :-* select log_date,count(1) as cnt from test.tt1 group by

Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

2018-04-12 Thread Richard A. Bross
Based on the exception, it looks more like an AWS credentials issue than a Hive issue. Are you running in AWS EMR, on-prem? In AWS the resource accessing the S3 bucket would have to have an IAM that gave permission. If you are running somewhere else whatever AWS login you use would have to

Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

2018-04-12 Thread Anup Tiwari
looping dev in for suggestions. Regards, Anup Tiwari On Thu, Apr 12, 2018 at 8:39 PM, Richard A. Bross wrote: > To be honest, my advice was just based on your post - we only use Hive in > AWS EMR, so I couldn't tell you. > > Glad that at least you're back up though. > >

Re: Unable to read hive external table data which is linked to s3 after upgradation from 2.1.1 to 2.3.3

2018-04-12 Thread Richard A. Bross
To be honest, my advice was just based on your post - we only use Hive in AWS EMR, so I couldn't tell you. Glad that at least you're back up though. - Original Message - From: "Anup Tiwari" To: user@hive.apache.org Sent: Thursday, April 12, 2018 10:50:23 AM