[jira] [Commented] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
[ https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181633#comment-16181633 ] Sergey Shelukhin commented on HIVE-17585: - Well, we already have new instance per thread everywhere else... > Improve thread safety when loading dynamic partitions in parallel > - > > Key: HIVE-17585 > URL: https://issues.apache.org/jira/browse/HIVE-17585 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Fix For: 3.0.0 > > Attachments: HIVE-17585.1.patch, HIVE-17585.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
[ https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180289#comment-16180289 ] Tao Li commented on HIVE-17585: --- [~sershe] Thanks for the comments. I thought about that and was a little concerned with the memory overhead to have a new Hive instance per thread (especially the embedded metastore scenario), so I chose to go with the singleton approach. But I agree that having the thread local will give us the best safety. Regarding the latency, the major advantage of the loading partitions in parallel is making the HDFS calls in parallel, so synchronizing on the metastore client should not be a big concern. > Improve thread safety when loading dynamic partitions in parallel > - > > Key: HIVE-17585 > URL: https://issues.apache.org/jira/browse/HIVE-17585 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Fix For: 3.0.0 > > Attachments: HIVE-17585.1.patch, HIVE-17585.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
[ https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180102#comment-16180102 ] Sergey Shelukhin commented on HIVE-17585: - Hmm... wouldn't a simpler solution be to run Hive.get() instead of synchronizing a set of methods called by loadPartition, given that Hive object is not thread safe and so the original code uses it incorrectly by calling loadPartition from multiple threads? If someone changes what loadPartition calls, this will break again as far as I can tell. And it's not good to change every method to use synchronized MSC, that will just be a perf hit. Unless I'm missing something. > Improve thread safety when loading dynamic partitions in parallel > - > > Key: HIVE-17585 > URL: https://issues.apache.org/jira/browse/HIVE-17585 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Fix For: 3.0.0 > > Attachments: HIVE-17585.1.patch, HIVE-17585.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
[
https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178031#comment-16178031
]
Thejas M Nair commented on HIVE-17585:
--
For reference these are some exceptions seen -
{code}
WARN [load-dynamic-partitions-3]: metastore.RetryingMetaStoreClient
(RetryingMetaStoreClient.java:invoke(206)) - MetaStoreClient lost connection.
Attempting to reconnect (1 of 24) after 5s. fireListenerEvent
org.apache.thrift.TApplicationException: fire_listener_event failed: out of
sequence response
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:84)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_fire_listener_event(ThriftHiveMetastore.java:4302)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.fire_listener_event(ThriftHiveMetastore.java:4289)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.fireListenerEvent(HiveMetaStoreClient.java:2088)
at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:178)
at com.sun.proxy.$Proxy5.fireListenerEvent(Unknown Source)
at
org.apache.hadoop.hive.ql.metadata.Hive.fireInsertEvent(Hive.java:2248)
at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1528)
at org.apache.hadoop.hive.ql.metadata.Hive$2.call(Hive.java:1794)
at org.apache.hadoop.hive.ql.metadata.Hive$2.call(Hive.java:1785)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
WARN [load-dynamic-partitions-13]: metastore.RetryingMetaStoreClient
(RetryingMetaStoreClient.java:invoke(206)) - MetaStoreClient lost connection.
Attempting to reconnect (1 of 24) after 5s. getPartitionWithAuthInfo
org.apache.thrift.TApplicationException: get_partition_with_auth failed: out of
sequence response
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:84)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partition_with_auth(ThriftHiveMetastore.java:1977)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partition_with_auth(ThriftHiveMetastore.java:1960)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getPartitionWithAuthInfo(HiveMetaStoreClient.java:1273)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:178)
at com.sun.proxy.$Proxy5.getPartitionWithAuthInfo(Unknown Source)
at
org.apache.hadoop.hive.metastore.SynchronizedMetaStoreClient.getPartitionWithAuthInfo(SynchronizedMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:2112)
at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:2053)
at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1468)
at org.apache.hadoop.hive.ql.metadata.Hive$2.call(Hive.java:1794)
at org.apache.hadoop.hive.ql.metadata.Hive$2.call(Hive.java:1785)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
WARN [load-dynamic-partitions-3]: transport.TIOStreamTransport
(TIOStreamTransport.java:close(112)) - Error closing output stream.
java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
at
org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:566)
at
[jira] [Commented] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
[ https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178024#comment-16178024 ] Thejas M Nair commented on HIVE-17585: -- +1 > Improve thread safety when loading dynamic partitions in parallel > - > > Key: HIVE-17585 > URL: https://issues.apache.org/jira/browse/HIVE-17585 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-17585.1.patch, HIVE-17585.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
[ https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177893#comment-16177893 ] Tao Li commented on HIVE-17585: --- Test result looks good. cc [~thejas] > Improve thread safety when loading dynamic partitions in parallel > - > > Key: HIVE-17585 > URL: https://issues.apache.org/jira/browse/HIVE-17585 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-17585.1.patch, HIVE-17585.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
[
https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177880#comment-16177880
]
Hive QA commented on HIVE-17585:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12888625/HIVE-17585.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 11059 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=231)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=231)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_4] (batchId=6)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=170)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6961/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6961/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6961/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12888625 - PreCommit-HIVE-Build
> Improve thread safety when loading dynamic partitions in parallel
> -
>
> Key: HIVE-17585
> URL: https://issues.apache.org/jira/browse/HIVE-17585
> Project: Hive
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17585.1.patch, HIVE-17585.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
[
https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177602#comment-16177602
]
Hive QA commented on HIVE-17585:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12888625/HIVE-17585.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11059 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=231)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=231)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=170)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6954/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6954/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6954/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12888625 - PreCommit-HIVE-Build
> Improve thread safety when loading dynamic partitions in parallel
> -
>
> Key: HIVE-17585
> URL: https://issues.apache.org/jira/browse/HIVE-17585
> Project: Hive
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17585.1.patch, HIVE-17585.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
