Alan, Thank you very much for the quick response. The hive jira: HIVE-8966 ( https://issues.apache.org/jira/browse/HIVE-8966) has been created. This is really a big issue. We are using hive hcatalog streaming to populate data into hive. It will be very bad if the data cannot be compacted. So please fix it as early as possible. Thank you very much for your help Jihong
On Tue, Nov 25, 2014 at 11:02 AM, Alan Gates <ga...@hortonworks.com> wrote: > Can you file a JIRA for this. We can get it fixed and included in the > 0.14 branch. > > Alan. > > Jihong Liu <jihongl...@gmail.com> > November 25, 2014 at 10:54 > hive hcatalog streaming will also create a file like bucket_n_flush_length > in each delta directory. Where "n" is the bucket number. But > the compactor.CompactorMR think this file also needs to compact. However > this file of course cannot be compacted, so compactor.CompactorMR will not > continue to do the compaction. > > Did a test, after removed the bucket_n_flush_length file, then the "alter > table partition compact" finished successfully. If don't delete that file, > nothing will be compacted. > This is probably a very severity bug. Both 0.13 and 0.14 have this issue. > > Thanks > Jihong > > > -- > Sent with Postbox <http://www.getpostbox.com> > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
