[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071138#comment-17071138
]
Marton Bod commented on HIVE-23052:
---
Benchmark test results:
[
https://issues.apache.org/jira/browse/HIVE-23067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23067:
--
Attachment: HIVE-23067.1.patch
> Use batch DB calls in TxnHandler for commitTxn and abortTxns
>
[
https://issues.apache.org/jira/browse/HIVE-23067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23067:
--
Attachment: HIVE-23067.1.patch
> Use batch DB calls in TxnHandler for commitTxn and abortTxns
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.9.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23067:
--
Attachment: HIVE-23067.1.patch
Status: Patch Available (was: Open)
> Use batch DB calls in
[
https://issues.apache.org/jira/browse/HIVE-23067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23067:
--
Attachment: HIVE-23067.1.patch
> Use batch DB calls in TxnHandler for commitTxn and abortTxns
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.8.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.8.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.8.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod reassigned HIVE-23093:
-
> Create new metastore config value for jdbc max batch size
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.7.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23067:
--
Summary: Use batch DB calls in TxnHandler for commitTxn and abortTxns
(was: Use batch inserts in
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.6.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065038#comment-17065038
]
Marton Bod commented on HIVE-23067:
---
Yes, sure :) thanks for the heads-up
> Use batch inserts in
[
https://issues.apache.org/jira/browse/HIVE-23067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23067:
--
Description:
To reduce the number of database calls and network roundtrips, we could use
more
[
https://issues.apache.org/jira/browse/HIVE-23067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod reassigned HIVE-23067:
-
> Use batch inserts in TxnHandler
> ---
>
> Key:
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.5.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Description:
* Reduce scope of next_lock_id select-for-update by moving the txn_component
inserts
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.4.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Description:
* Reduce scope of next_lock_id select-for-update by moving the txn_component
inserts
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.3.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.2.patch
> Optimize lock enqueueing in TxnHandler
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Attachment: HIVE-23052.1.patch
Status: Patch Available (was: In Progress)
> Optimize lock
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
--
Description:
TxnHandler::enqueueLockWithRetry uses select for update (S4U) to retrieve the
next lock
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-23052 started by Marton Bod.
-
> Optimize lock enqueueing in TxnHandler
> --
>
>
[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod reassigned HIVE-23052:
-
> Optimize lock enqueueing in TxnHandler
> --
>
>
[
https://issues.apache.org/jira/browse/HIVE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17051326#comment-17051326
]
Marton Bod edited comment on HIVE-22938 at 3/4/20, 3:33 PM:
Removing the
[
https://issues.apache.org/jira/browse/HIVE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17051326#comment-17051326
]
Marton Bod commented on HIVE-22938:
---
Removing the empty code part breaks 400+ tests. At this point with
[
https://issues.apache.org/jira/browse/HIVE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22938:
--
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
> Investigate possibility of
[
https://issues.apache.org/jira/browse/HIVE-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod resolved HIVE-22918.
---
Resolution: Done
> Investigate empty bucket file creation for ACID tables
>
[
https://issues.apache.org/jira/browse/HIVE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22938:
--
Attachment: HIVE-22938.1.patch
Assignee: Marton Bod
Status: Patch Available (was: Open)
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.9.patch
> Parallelise direct insert directory cleaning process
>
[
https://issues.apache.org/jira/browse/HIVE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046776#comment-17046776
]
Marton Bod commented on HIVE-22938:
---
As an addition: even for MR, the current state now is that when
[
https://issues.apache.org/jira/browse/HIVE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046751#comment-17046751
]
Marton Bod commented on HIVE-22938:
---
[~ashutoshc], [~gopalv] - I'm investigating whether we can stop
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.8.patch
> Parallelise direct insert directory cleaning process
>
[
https://issues.apache.org/jira/browse/HIVE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22938:
--
Description:
As a follow-up to HIVE-22918, this ticket is to investigate whether the empty
bucket
[
https://issues.apache.org/jira/browse/HIVE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22938:
--
Summary: Investigate possibility of removing empty bucket file creation
mechanism in Hive-on-MR (was:
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.8.patch
> Refactor Hive::listFilesCreatedByQuery to make it faster for object
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.7.patch
> Refactor Hive::listFilesCreatedByQuery to make it faster for object
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.7.patch
> Parallelise direct insert directory cleaning process
>
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.6.patch
> Refactor Hive::listFilesCreatedByQuery to make it faster for object
[
https://issues.apache.org/jira/browse/HIVE-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044598#comment-17044598
]
Marton Bod commented on HIVE-22918:
---
Based on discussions with [~lpinter], in theory, the lack of empty
[
https://issues.apache.org/jira/browse/HIVE-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044586#comment-17044586
]
Marton Bod commented on HIVE-22918:
---
Based on my investigation, the empty bucket files are only created
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.6.patch
> Parallelise direct insert directory cleaning process
>
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.5.patch
> Refactor Hive::listFilesCreatedByQuery to make it faster for object
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.5.patch
> Parallelise direct insert directory cleaning process
>
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.4.patch
> Parallelise direct insert directory cleaning process
>
[
https://issues.apache.org/jira/browse/HIVE-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22918 started by Marton Bod.
-
> Investigate empty bucket file creation for ACID tables
>
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.3.patch
> Parallelise direct insert directory cleaning process
>
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.4.patch
> Refactor Hive::listFilesCreatedByQuery to make it faster for object
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Description: Inside Utilities::handleDirectInsertTableFinalPath, the
cleanDirectInsertDirectories
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Summary: Parallelise direct insert directory cleaning process (was:
Parallelise mm directory cleaning
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.2.patch
> Parallelise mm directory cleaning process
>
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.3.patch
> Refactor Hive::listFilesCreatedByQuery to make it faster for object
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22832:
--
Attachment: HIVE-22832.1.patch
Status: Patch Available (was: Open)
> Parallelise mm directory
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod reassigned HIVE-22832:
-
> Parallelise mm directory cleaning process
> -
>
>
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.2.patch
> Refactor Hive::listFilesCreatedByQuery to make it faster for object
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.1.patch
Status: Patch Available (was: Open)
> Refactor
[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod reassigned HIVE-22819:
-
> Refactor Hive::listFilesCreatedByQuery to make it faster for object stores
>
301 - 359 of 359 matches
Mail list logo
