[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14727709#comment-14727709 ] Sushanth Sowmyan commented on HIVE-11657: - +1. > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725778#comment-14725778 ] Sergey Shelukhin commented on HIVE-11657: - [~navis] np :) can you review the change? > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725024#comment-14725024 ] Navis commented on HIVE-11657: -- [~sershe] Making static call in Hive was one of the my worst decision on hive. I was so tired of rebasing the patch for years and new things like permanent function bothered too much of me and I couldn't think of any better idea in that day. > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724500#comment-14724500 ] Sergey Shelukhin commented on HIVE-11657: - [~sushanth] can you take a look? > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[
https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14721006#comment-14721006
]
Hive QA commented on HIVE-11657:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12753084/HIVE-11657.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9380 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
org.apache.hive.hcatalog.streaming.TestStreaming.testRemainingTransactions
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5111/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5111/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5111/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12753084 - PreCommit-HIVE-TRUNK-Build
HIVE-2573 introduces some issues during metastore init (and CLI init)
-
Key: HIVE-11657
URL: https://issues.apache.org/jira/browse/HIVE-11657
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Critical
Attachments: HIVE-11657.patch
HIVE-2573 introduced static reload functions call.
It has a few problems:
1) When metastore client is initialized using an externally supplied config
(i.e. Hive.get(HiveConf)), it still gets called during static init using the
main service config. In my case, even though I have uris in the supplied
config to connect to remote MS (which eventually happens), the static call
creates objectstore, which is undesirable.
2) It breaks compat - old metastores do not support this call so new clients
will fail, and there's no workaround like not using a new feature because the
static call is always made
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717877#comment-14717877 ] Sergey Shelukhin commented on HIVE-11657: - I will take a look tomorrow. This is causing annoying and unobvious problems on some setups. HIVE-2573 introduces some issues Key: HIVE-11657 URL: https://issues.apache.org/jira/browse/HIVE-11657 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Critical HIVE-2573 introduced static reload functions call. It has a few problems: 1) When metastore client is initialized using an externally supplied config (i.e. Hive.get(HiveConf)), it still gets called during static init using the main service config. In my case, even though I have uris in the supplied config to connect to remote MS (which eventually happens), the static call creates objectstore, which is undesirable. 2) It breaks compat - old metastores do not support this call so new clients will fail, and there's no workaround like not using a new feature because the static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715601#comment-14715601 ] Sergey Shelukhin commented on HIVE-11657: - [~gopalv] fyi... I might take a look at (1) at some point, will probably fix (2) with it, unless something happens earlier HIVE-2573 introduces some issues Key: HIVE-11657 URL: https://issues.apache.org/jira/browse/HIVE-11657 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Priority: Critical HIVE-2573 introduced static reload functions call. It has a few problems: 1) When metastore client is initialized using an externally supplied config, it still gets called during static init using the main service config. In my case, even though I have uris in the supplied config to connect to remote MS (which eventually happens), the static call creates objectstore, which is undesirable. 2) It breaks compat - old metastores do not support this call so new clients will fail, and there's no workaround like not using a new feature because the static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues
[
https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715668#comment-14715668
]
Sergey Shelukhin commented on HIVE-11657:
-
[~navis] fyi if you have some easy fixes in mind :)
Example of logging I get
{noformat}
2015-08-26 17:52:16,695 INFO [InputInitializer [Map 1] #0]
metastore.HiveMetaStore: 0: Opening raw store with implemenation
class:org.apache.hadoop.hive.metastore.ObjectStore
2015-08-26 17:52:16,730 INFO [InputInitializer [Map 1] #0]
metastore.ObjectStore: ObjectStore, initialize called
[snip]
{noformat}
The call stack for the above is:
{noformat}
at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3054)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3074)
at
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3299)
at
org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:175)
at org.apache.hadoop.hive.ql.metadata.Hive.clinit(Hive.java:167)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$MetastoreCache.getHive(OrcInputFormat.java:1544)
{noformat}
[snip]
{noformat}
2015-08-26 17:52:16,761 INFO [InputInitializer [Map 1] #0] hive.metastore:
Trying to connect to metastore with URI thrift://[snip]:9085
2015-08-26 17:52:16,782 INFO [InputInitializer [Map 1] #0] hive.metastore:
Opened a connection to metastore, current connections: 1
{noformat}
HIVE-2573 introduces some issues
Key: HIVE-11657
URL: https://issues.apache.org/jira/browse/HIVE-11657
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Priority: Critical
HIVE-2573 introduced static reload functions call.
It has a few problems:
1) When metastore client is initialized using an externally supplied config
(i.e. Hive.get(HiveConf)), it still gets called during static init using the
main service config. In my case, even though I have uris in the supplied
config to connect to remote MS (which eventually happens), the static call
creates objectstore, which is undesirable.
2) It breaks compat - old metastores do not support this call so new clients
will fail, and there's no workaround like not using a new feature because the
static call is always made
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
