Re: rename output error during hive query on AWSs3-external table

2020-02-04 Thread Sungwoo Park
Not a solution, but looking at the source code of S3AFileSystem.java (Hadoop 2.8.5), I think the Exception raised inside S3AFileSystem.rename() is swallowed and only a new HiveException is reported. So, in order to find out the root cause, I guess you might need to set Log level to DEBUG and see

RE: rename output error during hive query on AWSs3-external table

2020-02-04 Thread Aaron Grubb
Check this thread: https://forums.aws.amazon.com/thread.jspa?messageID=922594 From: Souvikk Roy Sent: Tuesday, February 4, 2020 3:06 AM To: user@hive.apache.org Subject: rename output error during hive query on AWSs3-external table Hello, We are using some external tables backed by aws S3.

rename output error during hive query on AWSs3-external table

2020-02-04 Thread Souvikk Roy
Hello, We are using some external tables backed by aws S3. And we are intermittently getting this error, most likely at the last stage of the reduce, I see some similar posts in net but could not find any solution, Is there any way yo solve it: org.apache.hadoop.hive.ql.metadata.HiveException: