[jira] [Commented] (HIVE-14237) Move stats collection logic out of transaction for create table

2016-10-14 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576563#comment-15576563
 ] 

Ashutosh Chauhan commented on HIVE-14237:
-

IIRC client was sending table object with stats in some cases, so its not 
needed to collect stats on server. But I forgot which cases were those or if 
its true in all cases.

> Move stats collection logic out of transaction for create table
> ---
>
> Key: HIVE-14237
> URL: https://issues.apache.org/jira/browse/HIVE-14237
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 1.0.0, 1.2.0, 1.1.0, 2.0.0, 2.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-14237.1.patch, HIVE-14237.patch
>
>
> Stats collection logic interacts with FS which may take long time. There is 
> no need to do that inside DB transaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14237) Move stats collection logic out of transaction for create table

2016-10-14 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574390#comment-15574390
 ] 

Thejas M Nair commented on HIVE-14237:
--

[~ashutoshc] The call to MetaStoreUtils.updateTableStatsFast from 
HiveMetaStore.create_table_core no longer there after this change. That should 
also move to before the openTransaction, is that right ?


> Move stats collection logic out of transaction for create table
> ---
>
> Key: HIVE-14237
> URL: https://issues.apache.org/jira/browse/HIVE-14237
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 1.0.0, 1.2.0, 1.1.0, 2.0.0, 2.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-14237.1.patch, HIVE-14237.patch
>
>
> Stats collection logic interacts with FS which may take long time. There is 
> no need to do that inside DB transaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14237) Move stats collection logic out of transaction for create table

2016-09-03 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15461684#comment-15461684
 ] 

Hive QA commented on HIVE-14237:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12826711/HIVE-14237.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 10443 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineWithArgs - did not produce a TEST-*.xml file
TestHiveCli - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_table_like_stats]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testSimpleFunction
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testSimpleFunction
org.apache.hadoop.hive.metastore.TestRetryingHMSHandler.testRetryingHMSHandler
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleFunction
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1100/testReport
Console output: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1100/console
Test logs: 
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1100/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12826711 - PreCommit-HIVE-MASTER-Build

> Move stats collection logic out of transaction for create table
> ---
>
> Key: HIVE-14237
> URL: https://issues.apache.org/jira/browse/HIVE-14237
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 1.0.0, 1.2.0, 1.1.0, 2.0.0, 2.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-14237.1.patch, HIVE-14237.patch
>
>
> Stats collection logic interacts with FS which may take long time. There is 
> no need to do that inside DB transaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14237) Move stats collection logic out of transaction for create table

2016-07-28 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398567#comment-15398567
 ] 

Pengcheng Xiong commented on HIVE-14237:


The patch looks good to me. Could u double check testCliDriver_create_like and 
testCliDriver_index_skewtable. They seem related? +1 pending tests.

> Move stats collection logic out of transaction for create table
> ---
>
> Key: HIVE-14237
> URL: https://issues.apache.org/jira/browse/HIVE-14237
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 1.0.0, 1.2.0, 1.1.0, 2.0.0, 2.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-14237.patch
>
>
> Stats collection logic interacts with FS which may take long time. There is 
> no need to do that inside DB transaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14237) Move stats collection logic out of transaction for create table

2016-07-14 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15378793#comment-15378793
 ] 

Hive QA commented on HIVE-14237:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12817889/HIVE-14237.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 10321 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_index
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_like
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_default_file_format
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_skewtable
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestMinimrCliDriver.org.apache.hadoop.hive.cli.TestMinimrCliDriver
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testCheckPermissions
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testGetToken
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testSimpleFunction
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testConnections
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleFunction
org.apache.hadoop.hive.ql.metadata.TestHiveRemote.testThriftTable
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/518/testReport
Console output: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/518/console
Test logs: 
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-518/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12817889 - PreCommit-HIVE-MASTER-Build

> Move stats collection logic out of transaction for create table
> ---
>
> Key: HIVE-14237
> URL: https://issues.apache.org/jira/browse/HIVE-14237
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 1.0.0, 1.2.0, 1.1.0, 2.0.0, 2.1.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-14237.patch
>
>
> Stats collection logic interacts with FS which may take long time. There is 
> no need to do that inside DB transaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)