[jira] [Created] (HIVE-13315) Option to reuse existing restored HBase snapshots

Liyin Tang created HIVE-13315: - Summary: Option to reuse existing restored HBase snapshots Key: HIVE-13315 URL: https://issues.apache.org/jira/browse/HIVE-13315 Project: Hive Issue Type:

[jira] [Commented] (HIVE-2095) auto convert map join bug

[ https://issues.apache.org/jira/browse/HIVE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13016871#comment-13016871 ] Liyin Tang commented on HIVE-2095: -- I will take a look auto convert map join bug

[jira] [Commented] (HIVE-1965) Auto convert mapjoin should not throw exception if the top operator is union operator.

[ https://issues.apache.org/jira/browse/HIVE-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13010508#comment-13010508 ] Liyin Tang commented on HIVE-1965: -- +1 Auto convert mapjoin should not throw exception

[jira] [Commented] (HIVE-1966) mapjoin operator should not load hashtable for each new inputfile if the hashtable to be loaded is already there.

[ https://issues.apache.org/jira/browse/HIVE-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13010510#comment-13010510 ] Liyin Tang commented on HIVE-1966: -- +1 mapjoin operator should not load hashtable for

[jira] Updated: (HIVE-1845) Some attributes in the Eclipse template file is deprecated

[ https://issues.apache.org/jira/browse/HIVE-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1845: - Attachment: hive-1845-1.patch Update all the eclipse template files. Some attributes in the Eclipse

[jira] Updated: (HIVE-1845) Some attributes in the Eclipse template file is deprecated

[ https://issues.apache.org/jira/browse/HIVE-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1845: - Status: Patch Available (was: Open) Some attributes in the Eclipse template file is deprecated

[jira] Updated: (HIVE-1830) mappers in group followed by joins may die OOM

[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1830: - Attachment: hive-1830-5.patch 1) Remove the debug statements 2) Add the memory threshold to group by

[jira] Created: (HIVE-1842) Add the local flag to all the map red tasks, if the query is running locally.

Add the local flag to all the map red tasks, if the query is running locally. - Key: HIVE-1842 URL: https://issues.apache.org/jira/browse/HIVE-1842 Project: Hive

[jira] Updated: (HIVE-1842) Add the local flag to all the map red tasks, if the query is running locally.

[ https://issues.apache.org/jira/browse/HIVE-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1842: - Attachment: hive-1842-1.patch Add the local flag to all the map red tasks, if the query is running

[jira] Created: (HIVE-1832) Dynamically allocate and measure memory usage when a map join op followed by a group by op

Dynamically allocate and measure memory usage when a map join op followed by a group by op -- Key: HIVE-1832 URL: https://issues.apache.org/jira/browse/HIVE-1832

[jira] Commented: (HIVE-1832) Dynamically allocate and measure memory usage when a map join op followed by a group by op

[ https://issues.apache.org/jira/browse/HIVE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12967268#action_12967268 ] Liyin Tang commented on HIVE-1832: -- Duplicate of Hive-1830 Dynamically allocate and

[jira] Updated: (HIVE-1827) Audit how many queries will be run in the local mode

[ https://issues.apache.org/jira/browse/HIVE-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1827: - Attachment: hive-1827-1.patch Add a new attribute isLocalMode in Task. Audit how many queries will be

[jira] Updated: (HIVE-1830) mappers in group followed by joins may die OOM

[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1830: - Attachment: hive-1830-2.patch Add a new test: auto_join26.q mappers in group followed by joins may die

[jira] Updated: (HIVE-1830) mappers in group followed by joins may die OOM

[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1830: - Attachment: hive-1830-3.patch Carefully measure the memory usage of map side group by. Flush frequently,

[jira] Resolved: (HIVE-1700) Optimiza JDBM to make mapjoin faster

[ https://issues.apache.org/jira/browse/HIVE-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1700. -- Resolution: Won't Fix Release Note: The JDBM component has been removed from Hive. No need to

[jira] Assigned: (HIVE-1700) Optimiza JDBM to make mapjoin faster

[ https://issues.apache.org/jira/browse/HIVE-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang reassigned HIVE-1700: Assignee: Liyin Tang Optimiza JDBM to make mapjoin faster

[jira] Created: (HIVE-1811) Show the time the local task takes

Show the time the local task takes -- Key: HIVE-1811 URL: https://issues.apache.org/jira/browse/HIVE-1811 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions:

[jira] Updated: (HIVE-1804) Mapjoin will fail if there are no files associating with the join tables

[ https://issues.apache.org/jira/browse/HIVE-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1804: - Attachment: hive-1804-3.patch Since there are some other patches committed recently, I regenerate the

[jira] Updated: (HIVE-1811) Show the time the local task takes

[ https://issues.apache.org/jira/browse/HIVE-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1811: - Attachment: hive-1811-1.patch The original showTime code has potential bug if the local task takes more

[jira] Updated: (HIVE-1804) Mapjoin will fail if there are no files associating with the join tables

[ https://issues.apache.org/jira/browse/HIVE-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1804: - Attachment: hive-1804-2.patch Remove all the debug print statements. Please review Mapjoin will fail if

[jira] Updated: (HIVE-1792) track the joins which are being converted to map-join automatically

[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1792: - Attachment: (was: hive-1792-2.patch) track the joins which are being converted to map-join

[jira] Assigned: (HIVE-1808) but in auto_join25.q

[ https://issues.apache.org/jira/browse/HIVE-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang reassigned HIVE-1808: Assignee: Liyin Tang but in auto_join25.q Key: HIVE-1808

[jira] Commented: (HIVE-1792) track the joins which are being converted to map-join automatically

[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12935234#action_12935234 ] Liyin Tang commented on HIVE-1792: -- There will be 2 cases to run the common join. One is

[jira] Created: (HIVE-1804) Mapjoin will fail if there are no files associating with the join tables

Mapjoin will fail if there are no files associating with the join tables Key: HIVE-1804 URL: https://issues.apache.org/jira/browse/HIVE-1804 Project: Hive Issue Type:

[jira] Updated: (HIVE-1797) Compressed the hashtable dump file before put into distributed cache

[ https://issues.apache.org/jira/browse/HIVE-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1797: - Status: Patch Available (was: Open) Compressed the hashtable dump file before put into distributed

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Status: Patch Available (was: Open) change Pre/Post Query Hooks to take in 1 parameter: HookContext

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive-1785_3.patch In order to be compatible, we check whether the hook implements the

[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12933096#action_12933096 ] Liyin Tang commented on HIVE-1785: -- How about adding one more layer over Pre/PostExecute

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive-1785_4.patch In this patch, I add the Hook interface over Pre/PostExecute and

[jira] Created: (HIVE-1797) Compressed the hashtable dump file before put into distributed cache

Compressed the hashtable dump file before put into distributed cache Key: HIVE-1797 URL: https://issues.apache.org/jira/browse/HIVE-1797 Project: Hive Issue Type:

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive_1642_2.patch Thanks for the comments. I have updated the patch according to the review

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive_1642_4.patch This patch formats the output of local task. Convert join queries to

[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932623#action_12932623 ] Liyin Tang commented on HIVE-1785: -- I generate the diff based on the Hive-1642. Please

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_5.patch Add more descriptions to the configuration files. Revert the DriverContext.

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: (was: hive-1642_5.patch) Convert join queries to map-join based on size of table/row

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_6.patch Remove the getBackupTask interface from all the Conditional Resolver

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_9.patch some minor changes in ConditionalResolverCommonJoin.java Convert join

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_10.patch After discussing, we think the function: replaceWithConditionalTask is not

[jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932132#action_12932132 ] Liyin Tang commented on HIVE-1642: -- There are 2 kinds of backup. 1) task level 2) branch

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive_1785_1.patch In this patch I have changed the interface of pre hoook and post hook. So

[jira] Commented: (HIVE-1688) In the MapJoinOperator, the code uses tag as alias, which is not always true

[ https://issues.apache.org/jira/browse/HIVE-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12931710#action_12931710 ] Liyin Tang commented on HIVE-1688: -- This bug has been fixed in Hive-1641 earlier and that

[jira] Resolved: (HIVE-1688) In the MapJoinOperator, the code uses tag as alias, which is not always true

[ https://issues.apache.org/jira/browse/HIVE-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1688. -- Resolution: Fixed Fix Version/s: 0.7.0 In the MapJoinOperator, the code uses tag as alias,

[jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row

[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12931599#action_12931599 ] Liyin Tang commented on HIVE-1642: -- I just finished converting common join into map join

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: hive-1754_7.patch Change the code style according to the reviewer comments Remove JDBM

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: hive-1754_4.patch Resolved all the output conflicts in this patch Remove JDBM component

[jira] Resolved: (HIVE-1775) Assertation on inputObjInspectors.length in Groupy operator

[ https://issues.apache.org/jira/browse/HIVE-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1775. -- Resolution: Fixed Release Note: This bug is fixed in Hive-1754 Assertation on

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: Hive-1754_3.patch Fix a bug when join value is null Also fix the hive-1775 Remove JDBM

[jira] Commented: (HIVE-1775) Assertation on inputObjInspectors.length in Groupy operator

[ https://issues.apache.org/jira/browse/HIVE-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12929869#action_12929869 ] Liyin Tang commented on HIVE-1775: -- resolved in patch hive-1754 Assertation on

[jira] Created: (HIVE-1775) Assertation on inputObjInspectors.length in Groupy operator

Assertation on inputObjInspectors.length in Groupy operator --- Key: HIVE-1775 URL: https://issues.apache.org/jira/browse/HIVE-1775 Project: Hive Issue Type: Bug Components:

[jira] Commented: (HIVE-1775) Assertation on inputObjInspectors.length in Groupy operator

[ https://issues.apache.org/jira/browse/HIVE-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12929480#action_12929480 ] Liyin Tang commented on HIVE-1775: -- Yes. I can just comment out this assertion.

[jira] Commented: (HIVE-1754) Remove JDBM component from Map Join

[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12928262#action_12928262 ] Liyin Tang commented on HIVE-1754: -- This patch has some potential bugs. I will fix it today

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: Hive-1754_2.patch Remove JDBM from Hive completely Remove JDBM component from Map Join

[jira] Resolved: (HIVE-1702) optimize JDBM to make mapjoin faster

[ https://issues.apache.org/jira/browse/HIVE-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1702. -- Resolution: Won't Fix Release Note: JDBM will be removed from Hive optimize JDBM to make mapjoin

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Status: Patch Available (was: Open) This patch modifies the following things 1) Remove the JDBM from

[jira] Updated: (HIVE-1756) failures in fatal.q in TestNegativeCliDriver

[ https://issues.apache.org/jira/browse/HIVE-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1756: - Attachment: Hive-1756.patch remove fatal.q failures in fatal.q in TestNegativeCliDriver

[jira] Updated: (HIVE-1757) test cleanup for Hive-1641

[ https://issues.apache.org/jira/browse/HIVE-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1757: - Attachment: Hive-1757.patch Remove some unnecessary print out statements test cleanup for Hive-1641

[jira] Commented: (HIVE-1641) add map joined table to distributed cache

[ https://issues.apache.org/jira/browse/HIVE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12925035#action_12925035 ] Liyin Tang commented on HIVE-1641: -- The patch without jdbm is also ready. shall I submit

[jira] Created: (HIVE-1733) Make the bucket size of JDBM configurable

Make the bucket size of JDBM configurable -- Key: HIVE-1733 URL: https://issues.apache.org/jira/browse/HIVE-1733 Project: Hive Issue Type: Task Components: Query Processor Affects

[jira] Updated: (HIVE-1641) add map joined table to distributed cache

[ https://issues.apache.org/jira/browse/HIVE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1641: - Attachment: Hive-1641(4).patch add map joined table to distributed cache

[jira] Updated: (HIVE-1641) add map joined table to distributed cache

[ https://issues.apache.org/jira/browse/HIVE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1641: - Attachment: Hive-1641(5).patch add map joined table to distributed cache

[jira] Updated: (HIVE-1641) add map joined table to distributed cache

[ https://issues.apache.org/jira/browse/HIVE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1641: - Attachment: Hive-1641(3).txt add map joined table to distributed cache

[jira] Commented: (HIVE-1723) The result of left semi join is not correct

[ https://issues.apache.org/jira/browse/HIVE-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12922193#action_12922193 ] Liyin Tang commented on HIVE-1723: -- The root cause of missing some part of join result is

[jira] Created: (HIVE-1730) Keep the mapping between jdbm file name with bucket file name during the compile time

Keep the mapping between jdbm file name with bucket file name during the compile time - Key: HIVE-1730 URL: https://issues.apache.org/jira/browse/HIVE-1730 Project:

[jira] Assigned: (HIVE-1730) Keep the mapping between jdbm file name with bucket file name during the compile time

[ https://issues.apache.org/jira/browse/HIVE-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang reassigned HIVE-1730: Assignee: Liyin Tang Keep the mapping between jdbm file name with bucket file name during the

[jira] Created: (HIVE-1732) Do not assume if there is a local work, then there will be a map join operator in this work

Do not assume if there is a local work, then there will be a map join operator in this work --- Key: HIVE-1732 URL: https://issues.apache.org/jira/browse/HIVE-1732

[jira] Updated: (HIVE-1732) Do not assume if there is a local work, then there will be a map join operator in this work

[ https://issues.apache.org/jira/browse/HIVE-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1732: - Component/s: Query Processor Affects Version/s: 0.7.0 0.6.0 Do not