[jira] [Commented] (HIVE-20901) running compactor when there is nothing to do produces duplicate data

2019-04-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813942#comment-16813942 ] Eugene Koifman commented on HIVE-20901: --- I'd suggest that {{msg.append("Skipping minor compaction

[jira] [Commented] (HIVE-20901) running compactor when there is nothing to do produces duplicate data

2019-04-04 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810093#comment-16810093 ] Eugene Koifman commented on HIVE-20901: --- go ahead.  Looking at the description, there is no data

[jira] [Commented] (HIVE-13479) Relax sorting requirement in ACID tables

2019-03-20 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-13479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797769#comment-16797769 ] Eugene Koifman commented on HIVE-13479: --- There is no sorting restriction on insert-only ACID

[jira] [Commented] (HIVE-21165) ACID: pass query hint to the writers to write hive.acid.key.index

2019-03-18 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795644#comment-16795644 ] Eugene Koifman commented on HIVE-21165: --- The logic in HIVE-20738 relies on this index. > ACID:

[jira] [Commented] (HIVE-20580) OrcInputFormat.isOriginal() should not rely on hive.acid.key.index

2019-03-18 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795638#comment-16795638 ] Eugene Koifman commented on HIVE-20580: --- isOriginal should mean files w/o acid metadata columns in

[jira] [Commented] (HIVE-20580) OrcInputFormat.isOriginal() should not rely on hive.acid.key.index

2019-03-17 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794511#comment-16794511 ] Eugene Koifman commented on HIVE-20580: --- note that Query based compactor doesn't produce

[jira] [Commented] (HIVE-20436) Lock Manager scalability - linear

2019-02-19 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772344#comment-16772344 ] Eugene Koifman commented on HIVE-20436: --- {{RewriteSemanticAnalyzer.updateOutputs}} is relevant >

[jira] [Updated] (HIVE-21266) Issue with single delta file

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21266: -- Component/s: Transactions > Issue with single delta file > > >

[jira] [Commented] (HIVE-21254) Pre-upgrade tool should handle exceptions and skip db/tables

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767735#comment-16767735 ] Eugene Koifman commented on HIVE-21254: --- +1 patch 5 pending tests > Pre-upgrade tool should handle

[jira] [Updated] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21177: -- Release Note: I messed up the commit msg for this. The Jira number is correct, but the bug

[jira] [Commented] (HIVE-21254) Pre-upgrade tool should handle exceptions and skip db/tables

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767572#comment-16767572 ] Eugene Koifman commented on HIVE-21254: --- I would think it's a security hole if you can set this

[jira] [Updated] (HIVE-21266) Issue with single delta file

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21266: -- Description:

[jira] [Assigned] (HIVE-21266) Issue with single delta file

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-21266: - > Issue with single delta file > > > Key:

[jira] [Commented] (HIVE-21254) Pre-upgrade tool should handle exceptions and skip db/tables

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767539#comment-16767539 ] Eugene Koifman commented on HIVE-21254: --- Compactor has to have {{Table}} object to do anything.  If

[jira] [Commented] (HIVE-21254) Pre-upgrade tool should handle exceptions and skip db/tables

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767507#comment-16767507 ] Eugene Koifman commented on HIVE-21254: --- this seems dangerous.  If the tools misses any tables that

[jira] [Commented] (HIVE-21058) Make Compactor run in a transaction (Umbrella)

2019-02-13 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767499#comment-16767499 ] Eugene Koifman commented on HIVE-21058: --- [~asomani] - no concrete plans > Make Compactor run in a

[jira] [Updated] (HIVE-9995) ACID compaction tries to compact a single file

2019-02-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-9995: - Target Version/s: 4.0.0 Status: Patch Available (was: Open) > ACID compaction

[jira] [Updated] (HIVE-20901) running compactor when there is nothing to do produces duplicate data

2019-02-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-20901: -- Description: suppose we run minor compaction 2 times, via alter table The 2nd request to

[jira] [Updated] (HIVE-9995) ACID compaction tries to compact a single file

2019-02-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-9995: - Attachment: HIVE-9995.01.patch > ACID compaction tries to compact a single file >

[jira] [Updated] (HIVE-20901) running compactor when there is nothing to do produces duplicate data

2019-02-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-20901: -- Description: suppose we run minor compaction 2 times, via alter table The 2nd request to

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

2019-02-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763911#comment-16763911 ] Eugene Koifman commented on HIVE-21225: --- perhaps an easier/better solution is to add another suffix

[jira] [Updated] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-02-07 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21177: -- Resolution: Fixed Fix Version/s: 4.0.0 Release Note: n/a Status:

[jira] [Commented] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-02-07 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762905#comment-16762905 ] Eugene Koifman commented on HIVE-21177: --- failures not related committed to master thanks Prasanth

[jira] [Commented] (HIVE-21222) ACID: When there are no delete deltas skip finding min max keys

2019-02-06 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762394#comment-16762394 ] Eugene Koifman commented on HIVE-21222: --- failure not related > ACID: When there are no delete

[jira] [Commented] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-02-06 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762302#comment-16762302 ] Eugene Koifman commented on HIVE-21177: --- patch 3 - some more refactoring to use Path rather than

[jira] [Updated] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-02-06 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21177: -- Attachment: HIVE-21177.03.patch > Optimize AcidUtils.getLogicalLength() >

[jira] [Updated] (HIVE-21222) ACID: When there are no delete deltas skip finding min max keys

2019-02-05 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21222: -- Component/s: Transactions > ACID: When there are no delete deltas skip finding min max keys >

[jira] [Commented] (HIVE-21222) ACID: When there are no delete deltas skip finding min max keys

2019-02-05 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761374#comment-16761374 ] Eugene Koifman commented on HIVE-21222: --- +1 > ACID: When there are no delete deltas skip finding

[jira] [Commented] (HIVE-20699) Query based compactor for full CRUD Acid tables

2019-02-04 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760381#comment-16760381 ] Eugene Koifman commented on HIVE-20699: --- +1 patch 11 > Query based compactor for full CRUD Acid

[jira] [Commented] (HIVE-20699) Query based compactor for full CRUD Acid tables

2019-02-04 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760259#comment-16760259 ] Eugene Koifman commented on HIVE-20699: --- There are a few unused imports in SplitGrouper

[jira] [Commented] (HIVE-21172) DEFAULT keyword handling in MERGE UPDATE clause issues

2019-02-04 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760191#comment-16760191 ] Eugene Koifman commented on HIVE-21172: --- [~vgarg], HIVE-21159 is in. thank you > DEFAULT keyword

[jira] [Updated] (HIVE-21159) Modify Merge statement logic to perform Update split early

2019-02-04 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21159: -- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch

[jira] [Commented] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-01-30 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756350#comment-16756350 ] Eugene Koifman commented on HIVE-21177: --- ParsedDeltaLight pd =

[jira] [Commented] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-01-29 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755600#comment-16755600 ] Eugene Koifman commented on HIVE-21177: --- fixed tests - all were test issues all

[jira] [Updated] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-01-29 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21177: -- Attachment: HIVE-21177.02.patch > Optimize AcidUtils.getLogicalLength() >

[jira] [Comment Edited] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-01-29 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755453#comment-16755453 ] Eugene Koifman edited comment on HIVE-21177 at 1/29/19 10:54 PM: - I added

[jira] [Updated] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-01-29 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21177: -- Status: Patch Available (was: Open) I added checks so that we don't look for the side file if

[jira] [Updated] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-01-29 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21177: -- Attachment: HIVE-21177.01.patch > Optimize AcidUtils.getLogicalLength() >

[jira] [Assigned] (HIVE-21177) Optimize AcidUtils.getLogicalLength()

2019-01-29 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-21177: - > Optimize AcidUtils.getLogicalLength() > - > >

[jira] [Commented] (HIVE-20699) Query based compactor for full CRUD Acid tables

2019-01-28 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754507#comment-16754507 ] Eugene Koifman commented on HIVE-20699: --- I left a few of minor comments on RB for patch 9.  Overall

[jira] [Commented] (HIVE-21159) Modify Merge statement logic to perform Update split early

2019-01-28 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754349#comment-16754349 ] Eugene Koifman commented on HIVE-21159: --- fixed test failures in patch 3. [~vgumashta] could you

[jira] [Updated] (HIVE-21159) Modify Merge statement logic to perform Update split early

2019-01-28 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21159: -- Attachment: HIVE-21159.03.patch > Modify Merge statement logic to perform Update split early >

[jira] [Updated] (HIVE-21159) Modify Merge statement logic to perform Update split early

2019-01-25 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21159: -- Attachment: HIVE-21159.02.patch > Modify Merge statement logic to perform Update split early >

[jira] [Updated] (HIVE-21158) Perform update split early

2019-01-25 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21158: -- Description: Currently Acid 2.0 does U=D+I in the OrcRecordUpdater. This means that all

[jira] [Updated] (HIVE-21172) DEFAULT keyword handling in MERGE UPDATE clause issues

2019-01-25 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21172: -- Description: once HIVE-21159 lands, enable {{HiveConf.MERGE_SPLIT_UPDATE}} and run these

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-23 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750653#comment-16750653 ] Eugene Koifman commented on HIVE-21052: --- Suppose you have p-type clean on table T that is running

[jira] [Commented] (HIVE-21118) Make sure if dynamic partitions is true only there's only one writeId allocated

2019-01-23 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750636#comment-16750636 ] Eugene Koifman commented on HIVE-21118: --- The current model in Acid subsystem is that there is 1

[jira] [Updated] (HIVE-21159) Modify Merge statement logic to perform Update split early

2019-01-23 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21159: -- Attachment: HIVE-21159.01.patch > Modify Merge statement logic to perform Update split early >

[jira] [Updated] (HIVE-21159) Modify Merge statement logic to perform Update split early

2019-01-23 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21159: -- Status: Patch Available (was: Open) WIP - see what breaks > Modify Merge statement logic to

[jira] [Assigned] (HIVE-21159) Modify Merge statement logic to perform Update split early

2019-01-23 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-21159: - > Modify Merge statement logic to perform Update split early >

[jira] [Assigned] (HIVE-21158) Perform update split early

2019-01-23 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-21158: - > Perform update split early > -- > > Key:

[jira] [Comment Edited] (HIVE-20699) Query based compactor for full CRUD Acid tables

2019-01-23 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749370#comment-16749370 ] Eugene Koifman edited comment on HIVE-20699 at 1/23/19 7:41 PM: left some

[jira] [Commented] (HIVE-20699) Query based compactor for full CRUD Acid tables

2019-01-22 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749370#comment-16749370 ] Eugene Koifman commented on HIVE-20699: --- left some comments on RB > Query based compactor for full

[jira] [Comment Edited] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-22 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749161#comment-16749161 ] Eugene Koifman edited comment on HIVE-21052 at 1/22/19 9:21 PM: not sure

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-22 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749161#comment-16749161 ] Eugene Koifman commented on HIVE-21052: --- not sure that this is enough. Suppose you have 2 p-type

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-22 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749022#comment-16749022 ] Eugene Koifman commented on HIVE-21052: --- You are right. I forgot that {{findReadyToClean()}}

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-18 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746903#comment-16746903 ] Eugene Koifman commented on HIVE-21052: --- We'd like to prevent 2 concurrent p-type cleans of the

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-18 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746897#comment-16746897 ] Eugene Koifman commented on HIVE-21052: --- I misunderstood which locks you were talking about.  I'll

[jira] [Issue Comment Deleted] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-18 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21052: -- Comment: was deleted (was: Compaction (any part of it) never acquires any locks - it runs

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-18 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746854#comment-16746854 ] Eugene Koifman commented on HIVE-21052: --- Compaction (any part of it) never acquires any locks - it

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-17 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745713#comment-16745713 ] Eugene Koifman commented on HIVE-21052: --- I left some comments on RB. I think the way Cleaner

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-17 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745627#comment-16745627 ] Eugene Koifman commented on HIVE-21052: --- there is 1 \{{writeId}} per (table, txnid) - HIVE-21118 >

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-17 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745593#comment-16745593 ] Eugene Koifman commented on HIVE-21052: --- [~jmarhuen], I'm not sure I understand your 1st 2 bullet

[jira] [Commented] (HIVE-20198) Constant time table drops/renames

2019-01-17 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745538#comment-16745538 ] Eugene Koifman commented on HIVE-20198: --- FYI, {{TBLS.TBL_ID}} is exposed via Thrift since

[jira] [Updated] (HIVE-20960) Make MM compactor run in a transaction and remove CompactorMR.createCompactorMarker()

2019-01-11 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-20960: -- Resolution: Fixed Fix Version/s: 4.0.0 Release Note: n/a Status:

[jira] [Commented] (HIVE-21115) Add support for object versions in metastore

2019-01-11 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740793#comment-16740793 ] Eugene Koifman commented on HIVE-21115: --- Isn't there a direct SQL path somewhere that modifies HMS

[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking

2019-01-11 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740788#comment-16740788 ] Eugene Koifman commented on HIVE-20801: --- [~gopalv] what do you mean locks are advisory? By

[jira] [Updated] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-11 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21052: -- Affects Version/s: (was: 3.1.1) 3.0.0 > Make sure transactions get

[jira] [Updated] (HIVE-21118) Make sure if dynamic partitions is true only there's only one writeId allocated

2019-01-11 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21118: -- Component/s: Transactions > Make sure if dynamic partitions is true only there's only one

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-11 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740528#comment-16740528 ] Eugene Koifman commented on HIVE-21052: --- we should never allocate > 1 writeId per (table,txn).

[jira] [Updated] (HIVE-21114) Create read-only transactions

2019-01-10 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21114: -- Description: With HIVE-21036 we have a way to indicate that a txn is read only. We should (at

[jira] [Updated] (HIVE-21036) extend OpenTxnRequest with transaction type

2019-01-10 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21036: -- Resolution: Fixed Fix Version/s: 4.0.0 Release Note: n/a Status:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-10 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739823#comment-16739823 ] Eugene Koifman commented on HIVE-21052: --- it would be useful to add some tests of partitioned

[jira] [Comment Edited] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-10 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739799#comment-16739799 ] Eugene Koifman edited comment on HIVE-21052 at 1/10/19 9:52 PM: I don't

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

2019-01-10 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739799#comment-16739799 ] Eugene Koifman commented on HIVE-21052: --- I don't think testing the manually is sufficient. See

[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type

2019-01-10 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739772#comment-16739772 ] Eugene Koifman commented on HIVE-21036: --- +1 patch 4 pending tests > extend OpenTxnRequest with

[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type

2019-01-10 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739586#comment-16739586 ] Eugene Koifman commented on HIVE-21036: --- I don't understand your explanation about the

[jira] [Updated] (HIVE-21052) Make sure transaction get cleaned if they are aborted before addPartitions is called

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21052: -- Attachment: Aborted Txn w_Direct Write.pdf > Make sure transaction get cleaned if they are

[jira] [Assigned] (HIVE-21052) Make sure transaction get cleaned if they are aborted before addPartitions is called

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-21052: - Assignee: Jaume M (was: Eugene Koifman) > Make sure transaction get cleaned if they

[jira] [Assigned] (HIVE-21052) Make sure transaction get cleaned if they are aborted before addPartitions is called

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-21052: - Assignee: Eugene Koifman (was: Jaume M) > Make sure transaction get cleaned if they

[jira] [Resolved] (HIVE-12451) Orc fast file merging/concatenation should be disabled for ACID tables

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman resolved HIVE-12451. --- Resolution: Fixed Fix Version/s: 3.0.0 Release Note: Concatenate is now

[jira] [Assigned] (HIVE-12451) Orc fast file merging/concatenation should be disabled for ACID tables

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-12451: - Assignee: Eugene Koifman (was: Alan Gates) > Orc fast file merging/concatenation

[jira] [Updated] (HIVE-16258) Suggestion: simplify type 2 SCDs with this non-standard extension to MERGE

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16258: -- Issue Type: New Feature (was: Improvement) > Suggestion: simplify type 2 SCDs with this

[jira] [Resolved] (HIVE-17284) remove OrcRecordUpdater.deleteEventIndexBuilder

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman resolved HIVE-17284. --- Resolution: Won't Fix hive.acid.key.index is actually need for delete event filtering

[jira] [Resolved] (HIVE-18221) test acid default

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman resolved HIVE-18221. --- Resolution: Won't Fix this was just to run all tests > test acid default >

[jira] [Resolved] (HIVE-20435) Failed Dynamic Partition Insert into insert only table may loose transaction metadata

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman resolved HIVE-20435. --- Resolution: Won't Fix Fix Version/s: 4.0.0 Release Note: n/a address in

[jira] [Resolved] (HIVE-19955) ACID: Pre-filter the delete event registry using insert delta ranges

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-19955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman resolved HIVE-19955. --- Resolution: Won't Fix Fix Version/s: 4.0.0 Release Note: n/a fixed as part of

[jira] [Assigned] (HIVE-19955) ACID: Pre-filter the delete event registry using insert delta ranges

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-19955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-19955: - Assignee: Eugene Koifman > ACID: Pre-filter the delete event registry using insert

[jira] [Comment Edited] (HIVE-20435) Failed Dynamic Partition Insert into insert only table may loose transaction metadata

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738774#comment-16738774 ] Eugene Koifman edited comment on HIVE-20435 at 1/9/19 11:30 PM: addressed

[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-20919: -- Resolution: Fixed Release Note: n/a Status: Resolved (was: Patch Available) >

[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738767#comment-16738767 ] Eugene Koifman commented on HIVE-20919: --- committed to master thanks Miklos for the contribution >

[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-20919: -- Fix Version/s: (was: 3.2.0) 4.0.0 > Break up

[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738740#comment-16738740 ] Eugene Koifman commented on HIVE-20919: --- +1 > Break up UpdateDeleteSemanticAnalyzer >

[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738671#comment-16738671 ] Eugene Koifman commented on HIVE-20919: --- [~mgergely] is this complete or does more need to be done?

[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738653#comment-16738653 ] Eugene Koifman commented on HIVE-21036: --- n/m the RB. could {{optional TxnType txn_type,}} in

[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type

2019-01-09 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738642#comment-16738642 ] Eugene Koifman commented on HIVE-21036: --- [~ikryvenko] could you create a ReviewBoard please >

[jira] [Resolved] (HIVE-21106) Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry

2019-01-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman resolved HIVE-21106. --- Resolution: Won't Fix Fix Version/s: 4.0.0 Release Note: n/a already addressed

[jira] [Assigned] (HIVE-21106) Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry

2019-01-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-21106: - Assignee: Eugene Koifman > Potential NEP in

[jira] [Updated] (HIVE-21106) Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry

2019-01-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-21106: -- Description: {{VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry()}} {noformat}

[jira] [Commented] (HIVE-21100) Allow flattening of table subdirectories resulted when using TEZ engine and UNION clause

2019-01-08 Thread Eugene Koifman (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737666#comment-16737666 ] Eugene Koifman commented on HIVE-21100: --- What is the motivation for this change? Doing

  1   2   3   4   5   6   7   8   9   10   >