Re: compaction issue: Compaction cannot compact above this txnid

Yes Peter, we're working on it. We try to make compaction work automatically. With crontab otherwise. Thanks for your help David Le mar. 2 juin 2020 à 14:48, Peter Vary a écrit : > Hi David, > > Maybe this can help: >

Re: compaction issue: Compaction cannot compact above this txnid

Hi David, Maybe this can help: https://docs.cloudera.com/HDPDocuments/HDP2/HDP-2.6.5/bk_data-access/content/understanding-administering-compactions.html Also if

Re: compaction issue: Compaction cannot compact above this txnid

Yes Peter. We can wait a little. We use streaming so we have to merge files regularly. If we can reduce the duration for some big Merge queries and make auto compaction works properly it should be ok. Problem: at the moment, compactions are not triggered automatically. Have you an idea ? Le mar.

Re: compaction issue: Compaction cannot compact above this txnid

Hi David, You do not really need to run compaction every time. Is it possible to wait for the compaction to start automatically next time? Thanks, Peter > On Jun 2, 2020, at 12:51, David Morin wrote: > > Thanks Peter, > > Any workaround on HDP 2.6.x with Hive 2 ? > Otherwise, the only way is

Re: compaction issue: Compaction cannot compact above this txnid

Thanks Peter, Any workaround on HDP 2.6.x with Hive 2 ? Otherwise, the only way is to reduce time it takes for this "merge" queries in order to cancel locks and related transactions. Am I right ? Le mar. 2 juin 2020 à 11:52, Peter Vary a écrit : > Hi David, > > I think this jira describes your

Re: compaction issue: Compaction cannot compact above this txnid

Hi David, I think this jira describes your situation: https://issues.apache.org/jira/browse/HIVE-16360 "The reason is that compactor won't compact anything above the level of an open transaction. So if there is a very long running txn, it may

Re: compaction issue: Compaction cannot compact above this txnid

I don't get it. The transaction id in the error message "No delta files or original files found to compact in hdfs://... with min Open txnid:6704143" is related to a current transaction (transaction + lock) for another user, another database and table. But still it blocks the compaction for the

compaction issue: Compaction cannot compact above this txnid

Hi, I have a compaction issue on my cluster. When I force a compaction (major) on one table I get this error in Metastore logs: 2020-06-01 19:49:35,512 ERROR [-78]: compactor.CompactorMR (CompactorMR.java:run(264)) - No delta files or original files found to compact in