Yes below are the values set in hive. Initally I haven’t mentioned NO_AUTO_COMPACTION in my table definition, which didn’t work so I have put it with value as false.
hive.compactor.initiator.on hive.compactor.worker.threads hive.compactor.worker.timeout hive.compactor.check.interval hive.compactor.delta.num.threshold hive.compactor.delta.pct.threshold Thanks, Sachin From: Alan Gates <alanfga...@gmail.com<mailto:alanfga...@gmail.com>> Reply-To: "dev@hive.apache.org<mailto:dev@hive.apache.org>" <dev@hive.apache.org<mailto:dev@hive.apache.org>> Date: Tuesday, 15 September 2015 10:30 pm To: "dev@hive.apache.org<mailto:dev@hive.apache.org>" <dev@hive.apache.org<mailto:dev@hive.apache.org>> Subject: Re: How to enable compaction for table with external data? If you want it to compact automatically you should not put NO_AUTO_COMPACTION in the table properties. First question, did you turn on the compactor on your metastore thrift server? To do this you need to set a couple of values in the metastore's hive-site.xml: hive.compactor.initiator.on=true hive.compactor.worker.threads=1 # or more Alan. [cid:part1.07070900.07000506@gmail.com] Sachin Pasalkar<mailto:sachin_pasal...@symantec.com> September 14, 2015 at 3:03 Hi, We are writing direct orc file from storm topology instead of using hive streaming (Due to performance issue with our data). However, we want to compact the data. So we have added the "NO_AUTO_COMPACTION"=“false” option in table which we created to read data(1.6 GB scattered in multiple small files) in ORC file. Does “NO_AUTO_COMPACTION” means it will not compact data while hive streaming is used? If no, why it did not compact our data into 1 file? We also tried manually calling compaction from java code using org.apache.hadoop.hive.metastore.txn.TxnHandler’s compact API which shows it has started compaction, when we execute command Show compactions. But still does not work. I don’t want to execute the manual commands from command line. Is there any way? PS: We are writing all files in one directory only. Thanks, Sachin
