[
https://issues.apache.org/jira/browse/HIVE-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622972#comment-14622972
]
Hive QA commented on HIVE-11229:
--------------------------------
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12744722/HIVE-11229.1.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9160 tests executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4575/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4575/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-4575/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12744722 - PreCommit-HIVE-TRUNK-Build
> Mutation API: Coordinator communication with meta store should be optional
> --------------------------------------------------------------------------
>
> Key: HIVE-11229
> URL: https://issues.apache.org/jira/browse/HIVE-11229
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
> Affects Versions: 2.0.0
> Reporter: Elliot West
> Assignee: Elliot West
> Labels: streaming_api
> Attachments: HIVE-11229.1.patch
>
>
> [~ekoifman] raised a theoretical issue with the streaming mutation API
> (HIVE-10165) where worker nodes operating in a distributed cluster might
> overwhelm a meta store while trying to obtain partition locks. Although this
> does not happen in practice (see HIVE-11228), the API does communicate with
> the meta store in this manner to obtain partition paths and create new
> partitions. Therefore the issue described does in fact exist in the current
> implementation, albeit in a different code path. I’d like to make such
> communication optional like so:
> * When the user chooses not to create partitions on demand, no meta store
> connection will be created in the {{MutationCoordinators}}. Additionally,
> partition paths will be resolved using
> {{org.apache.hadoop.hive.metastore.Warehouse.getPartitionPath(Path,
> LinkedHashMap<String, String>)}} which should be suitable so long as standard
> Hive partition layouts are followed.
> * If the user does choose to create partitions on demand then the system will
> operate as is does currently; using the meta store to both issue
> {{add_partition}} events and look up partition meta data.
> * The documentation will be updated to describe these behaviours and outline
> alternative approaches to collecting affected partition names and creating
> partitions in a less intensive manner.
> Side note for follow up: The parameter names {{tblName}} and {{dbName}} seem
> to be the wrong way around on the method
> {{org.apache.hadoop.hive.metastore.IMetaStoreClient.getPartition(String,
> String, List<String>)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
