Re: ORC ACID table returning Array Index Out of Bounds

2018-02-15 Thread Aviral Agarwal
Hive version is 1.2.1000.2.6.1.0-0129 ( HDP 2.6.1.0) For now I have mitigated the problem by recreating the table. So, I don't have the relevant ORC files right now. Also, I am curious, how would "*hive.acid.key.index*" help in debugging this problem ? I was going through the source code and it

ORC ACID table returning Array Index Out of Bounds

2018-02-15 Thread Aviral Agarwal
Hi guys, I am running into the following error when querying a ACID table : Caused by: java.lang.RuntimeException: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 8 at

Re: Hive beeline and ACID tables.

2018-02-15 Thread Colin Williams
Hi Johannes. I did set the "transactional" property to true like: tblproperties("transactional"="true"); And also stored as orc and bucketed and partitioned the table. Finally I did use double -- for setting the hiveconf. Then there must be some other issue regarding using the ACID tables. On

Re: Hive beeline and ACID tables.

2018-02-15 Thread Colin Williams
Looking back I may have omitted the --hiveconf setting. I will try creating the table again today and see if that was the issue. On Thu, Feb 15, 2018 at 8:51 AM, Colin Williams < colin.williams.seat...@gmail.com> wrote: > Hi Johannes. I did set the "transactional" property to true like: > >

Re: Hive beeline and ACID tables.

2018-02-15 Thread Eugene Koifman
“This command is not allowed on an ACID table analytics_user_vws.liveramp_events with a non-ACID transaction manager.” Indicates that “hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager” is not taking effect. When enabling acid you usually want to set this (and other relevant

Re: ORC ACID table returning Array Index Out of Bounds

2018-02-15 Thread Eugene Koifman
What version of Hive is this? Can you isolate this to a specific partition? The table/partition you are reading should have a directory called base_x/ with several bucket_N files. (if you see more than 1 base_x, take one with highest x) Each bucket_N should have a

Re: Hive beeline and ACID tables.

2018-02-15 Thread Colin Williams
Hi all. Thanks for the help. I was able to set the properties with multiple --hiveconf flags. It appears to have worked. However I do think it's a good idea to set the properties globally in hive-site.conf. On Thu, Feb 15, 2018 at 9:32 AM, Eugene Koifman wrote: > “This