[jira] [Updated] (HIVE-27062) Disable flaky test TestRpc#testClientTimeout
[ https://issues.apache.org/jira/browse/HIVE-27062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27062: -- Labels: pull-request-available (was: ) > Disable flaky test TestRpc#testClientTimeout > > > Key: HIVE-27062 > URL: https://issues.apache.org/jira/browse/HIVE-27062 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > TestRpc#testClientTimeout is flaky in branch-3. I don't see this test in the > main branch, so I think we should disable this test to make sure branch-3 is > green. The test works locally. > Failing run: > http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3989/6/tests/ > Fails with the stack trace: > java.lang.AssertionError: Client should have failed to connect to server. > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hive.spark.client.rpc.TestRpc.testClientTimeout(TestRpc.java:308) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413) > Passing run (on same commit): > http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3989/5/tests/ > In my opinion the test is not deterministic because it makes some timing > assumptions IIUC. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27062) Disable flaky test TestRpc#testClientTimeout
[ https://issues.apache.org/jira/browse/HIVE-27062?focusedWorklogId=846256&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846256 ] ASF GitHub Bot logged work on HIVE-27062: - Author: ASF GitHub Bot Created on: 17/Feb/23 20:05 Start Date: 17/Feb/23 20:05 Worklog Time Spent: 10m Work Description: vihangk1 opened a new pull request, #4072: URL: https://github.com/apache/hive/pull/4072 ### What changes were proposed in this pull request? This PR disables TestRpc because it is flaky. I ran the flaky test runner and it failed. ref: http://ci.hive.apache.org/job/hive-flaky-check/613/console ### Why are the changes needed? branch-3 stability ### Does this PR introduce _any_ user-facing change? NO ### How was this patch tested? Confirmed that the test skips locally. Issue Time Tracking --- Worklog Id: (was: 846256) Remaining Estimate: 0h Time Spent: 10m > Disable flaky test TestRpc#testClientTimeout > > > Key: HIVE-27062 > URL: https://issues.apache.org/jira/browse/HIVE-27062 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > TestRpc#testClientTimeout is flaky in branch-3. I don't see this test in the > main branch, so I think we should disable this test to make sure branch-3 is > green. The test works locally. > Failing run: > http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3989/6/tests/ > Fails with the stack trace: > java.lang.AssertionError: Client should have failed to connect to server. > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hive.spark.client.rpc.TestRpc.testClientTimeout(TestRpc.java:308) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413) > Passing run (on same commit): > http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3989/5/tests/ > In my opinion the test is not deterministic because it makes some timing > assumptions IIUC. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27092) Reenable flaky test TestRpc on branch-3
[ https://issues.apache.org/jira/browse/HIVE-27092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-27092: --- Summary: Reenable flaky test TestRpc on branch-3 (was: Reenable flaky test TestRpc) > Reenable flaky test TestRpc on branch-3 > --- > > Key: HIVE-27092 > URL: https://issues.apache.org/jira/browse/HIVE-27092 > Project: Hive > Issue Type: Test >Reporter: Vihang Karajgaonkar >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27062) Disable flaky test TestRpc#testClientTimeout
[ https://issues.apache.org/jira/browse/HIVE-27062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17690556#comment-17690556 ] Vihang Karajgaonkar commented on HIVE-27062: I submitted a flaky check job for this test to confirm and found that it fails in #try 74. http://ci.hive.apache.org/job/hive-flaky-check/613/console > Disable flaky test TestRpc#testClientTimeout > > > Key: HIVE-27062 > URL: https://issues.apache.org/jira/browse/HIVE-27062 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > TestRpc#testClientTimeout is flaky in branch-3. I don't see this test in the > main branch, so I think we should disable this test to make sure branch-3 is > green. The test works locally. > Failing run: > http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3989/6/tests/ > Fails with the stack trace: > java.lang.AssertionError: Client should have failed to connect to server. > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hive.spark.client.rpc.TestRpc.testClientTimeout(TestRpc.java:308) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413) > Passing run (on same commit): > http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3989/5/tests/ > In my opinion the test is not deterministic because it makes some timing > assumptions IIUC. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=846251&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846251 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 17/Feb/23 19:50 Start Date: 17/Feb/23 19:50 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1435160748 Thanks @yigress for the comments Issue Time Tracking --- Worklog Id: (was: 846251) Time Spent: 1h 20m (was: 1h 10m) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27090) Test fix for external_jdbc_auth.q
[ https://issues.apache.org/jira/browse/HIVE-27090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Raj resolved HIVE-27090. - Resolution: Fixed Resolving this issue. Thanks [~vihangk1] for the review and merge > Test fix for external_jdbc_auth.q > - > > Key: HIVE-27090 > URL: https://issues.apache.org/jira/browse/HIVE-27090 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > The external_jdbc_auth.q test fails due to the following errors : > java.lang.AssertionError: > Client Execution succeeded but contained differences (error code = 1) after > executing external_jdbc_auth.q > 220,221c220,221 > < 20 20 20.0 20.0 > < 20 20 20.0 20.0 > --- > > 20 20 20.0 20.0 20 > > 20 20 20.0 20.0 20 > > This test was newly added in https://issues.apache.org/jira/browse/HIVE-20651 > but the test has continuously failed on branch-3. Needs a fix. No other > change had been committed to this test after this ticket. Since the ticket > was merged in branch-3 without validating the test, fixing it now. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=846248&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846248 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 17/Feb/23 19:28 Start Date: 17/Feb/23 19:28 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1435135855 @zabetak @abstractdog Can you please review and merge this PR. Issue Time Tracking --- Worklog Id: (was: 846248) Time Spent: 1h 10m (was: 1h) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27087) Fix TestMiniSparkOnYarnCliDriver test failures on branch-3
[ https://issues.apache.org/jira/browse/HIVE-27087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17690525#comment-17690525 ] Vihang Karajgaonkar commented on HIVE-27087: [~chaosun] You probably are more familiar with HoS code than me. Do you have any thoughts or suggestions here? > Fix TestMiniSparkOnYarnCliDriver test failures on branch-3 > -- > > Key: HIVE-27087 > URL: https://issues.apache.org/jira/browse/HIVE-27087 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > TestMiniSparkOnYarnCliDriver are failing with the error below > [ERROR] 2023-02-16 14:13:08.991 [Driver] SparkContext - Error initializing > SparkContext. > java.lang.RuntimeException: java.lang.NoSuchFieldException: > DEFAULT_TINY_CACHE_SIZE > at > org.apache.spark.network.util.NettyUtils.getPrivateStaticField(NettyUtils.java:131) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.util.NettyUtils.createPooledByteBufAllocator(NettyUtils.java:118) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.init(TransportServer.java:94) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.(TransportServer.java:73) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.TransportContext.createServer(TransportContext.java:114) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.netty.NettyRpcEnv.startServer(NettyRpcEnv.scala:119) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:465) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:464) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2271) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160) > ~[scala-library-2.11.8.jar:?] > at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2263) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory.create(NettyRpcEnv.scala:469) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:57) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:249) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:256) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.(SparkContext.scala:423) > [spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:161) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:536) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_322] > The root cause of the problem is that we upgrade the netty library from > 4.1.17.Final to 4.1.69.Final. The upgraded library does not have > `DEFAULT_TINY_CACHE_SIZE` field > [here|https://github.com/netty/netty/blob/netty-4.1.51.Final/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L46] > which was removed in 4.1.52.Final -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27087) Fix TestMiniSparkOnYarnCliDriver test failures on branch-3
[ https://issues.apache.org/jira/browse/HIVE-27087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17690522#comment-17690522 ] Vihang Karajgaonkar commented on HIVE-27087: I understand what you are saying [~amanraj2520]but if netty upgrade has broken a feature we should also consider that. I looked into the possibility of if upgrading spark will solve the problem or not. Unfortunately, it looks like even if we upgrade spark to 2.4 it would still depend on 4.1.47 as seen [here|https://github.com/apache/spark/blob/branch-2.4/pom.xml#L634] so that is not a solution either. Is there is a way to have a dependency only for spark-client to limit the exposure to it. I don't see Hive-on-Spark on master branch so the upgrade doesn't affect in master branch in this context and hence the goal of having it closer to master branch doesn't make much sense. branch-3 and master branches are significantly different. > Fix TestMiniSparkOnYarnCliDriver test failures on branch-3 > -- > > Key: HIVE-27087 > URL: https://issues.apache.org/jira/browse/HIVE-27087 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > TestMiniSparkOnYarnCliDriver are failing with the error below > [ERROR] 2023-02-16 14:13:08.991 [Driver] SparkContext - Error initializing > SparkContext. > java.lang.RuntimeException: java.lang.NoSuchFieldException: > DEFAULT_TINY_CACHE_SIZE > at > org.apache.spark.network.util.NettyUtils.getPrivateStaticField(NettyUtils.java:131) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.util.NettyUtils.createPooledByteBufAllocator(NettyUtils.java:118) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.init(TransportServer.java:94) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.(TransportServer.java:73) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.TransportContext.createServer(TransportContext.java:114) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.netty.NettyRpcEnv.startServer(NettyRpcEnv.scala:119) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:465) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:464) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2271) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160) > ~[scala-library-2.11.8.jar:?] > at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2263) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory.create(NettyRpcEnv.scala:469) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:57) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:249) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:256) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.(SparkContext.scala:423) > [spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:161) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:536) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_322] > The root cause of the problem is that we upgrade the netty library from > 4.1.17.Final to 4.1.69.Final. The upgraded library does not have > `DEFAULT_TINY_CACHE_SIZE` field > [here|https://github.com/netty/netty/blob/netty-4.1.51.Final/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L46] > which was removed in 4.1.52.Final -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27090) Test fix for external_jdbc_auth.q
[ https://issues.apache.org/jira/browse/HIVE-27090?focusedWorklogId=846235&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846235 ] ASF GitHub Bot logged work on HIVE-27090: - Author: ASF GitHub Bot Created on: 17/Feb/23 18:09 Start Date: 17/Feb/23 18:09 Worklog Time Spent: 10m Work Description: vihangk1 merged PR #4069: URL: https://github.com/apache/hive/pull/4069 Issue Time Tracking --- Worklog Id: (was: 846235) Time Spent: 50m (was: 40m) > Test fix for external_jdbc_auth.q > - > > Key: HIVE-27090 > URL: https://issues.apache.org/jira/browse/HIVE-27090 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > The external_jdbc_auth.q test fails due to the following errors : > java.lang.AssertionError: > Client Execution succeeded but contained differences (error code = 1) after > executing external_jdbc_auth.q > 220,221c220,221 > < 20 20 20.0 20.0 > < 20 20 20.0 20.0 > --- > > 20 20 20.0 20.0 20 > > 20 20 20.0 20.0 20 > > This test was newly added in https://issues.apache.org/jira/browse/HIVE-20651 > but the test has continuously failed on branch-3. Needs a fix. No other > change had been committed to this test after this ticket. Since the ticket > was merged in branch-3 without validating the test, fixing it now. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27090) Test fix for external_jdbc_auth.q
[ https://issues.apache.org/jira/browse/HIVE-27090?focusedWorklogId=846234&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846234 ] ASF GitHub Bot logged work on HIVE-27090: - Author: ASF GitHub Bot Created on: 17/Feb/23 18:08 Start Date: 17/Feb/23 18:08 Worklog Time Spent: 10m Work Description: vihangk1 commented on PR #4069: URL: https://github.com/apache/hive/pull/4069#issuecomment-1435052738 The fix looks good to me. I can merge it in branch-3. Issue Time Tracking --- Worklog Id: (was: 846234) Time Spent: 40m (was: 0.5h) > Test fix for external_jdbc_auth.q > - > > Key: HIVE-27090 > URL: https://issues.apache.org/jira/browse/HIVE-27090 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > The external_jdbc_auth.q test fails due to the following errors : > java.lang.AssertionError: > Client Execution succeeded but contained differences (error code = 1) after > executing external_jdbc_auth.q > 220,221c220,221 > < 20 20 20.0 20.0 > < 20 20 20.0 20.0 > --- > > 20 20 20.0 20.0 20 > > 20 20 20.0 20.0 20 > > This test was newly added in https://issues.apache.org/jira/browse/HIVE-20651 > but the test has continuously failed on branch-3. Needs a fix. No other > change had been committed to this test after this ticket. Since the ticket > was merged in branch-3 without validating the test, fixing it now. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27016) Invoke optional output committer in TezProcessor
[ https://issues.apache.org/jira/browse/HIVE-27016?focusedWorklogId=846233&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846233 ] ASF GitHub Bot logged work on HIVE-27016: - Author: ASF GitHub Bot Created on: 17/Feb/23 18:07 Start Date: 17/Feb/23 18:07 Worklog Time Spent: 10m Work Description: yigress commented on PR #4046: URL: https://github.com/apache/hive/pull/4046#issuecomment-1435051139 @zabetak Thank you for working hard to bring branch-3 to release, users are looking forward to a new release in Hive. I completely agree that right now stabilizing the branch-3 tests is the first priority, as I see that due to tests not green, it is hard to mend tests as well. Rather than revert this commit, can we continue with test case fix in https://github.com/apache/hive/pull/4068#issuecomment-1435041562 ? The reason is that in branch-4 the test cases are fixed by HIVE-22589, maybe the test cases should have been fixed sooner. Issue Time Tracking --- Worklog Id: (was: 846233) Time Spent: 2h (was: 1h 50m) > Invoke optional output committer in TezProcessor > > > Key: HIVE-27016 > URL: https://issues.apache.org/jira/browse/HIVE-27016 > Project: Hive > Issue Type: Improvement > Components: Query Processor, StorageHandler >Affects Versions: 3.1.3 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 2h > Remaining Estimate: 0h > > This is backport HIVE-24629 and HIVE-24867, so StorageHandler with their own > OutputCommitter run in tez. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=846230&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846230 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 17/Feb/23 17:59 Start Date: 17/Feb/23 17:59 Worklog Time Spent: 10m Work Description: yigress commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1435041562 @zabetak @amanraj2520 I wonder if we should continue with this fix, rather than revert HIVE-27016? The reason is that I see in branch-4 the permissions were changed as proposed here in HIVE-22589, but it is not apparent to me that HIVE-22589 itself causes the permission changes, so it looks like the orc_merge* tests permissions should have been updated. I agree that stabilizing the tests is the first priority in branch-3 right now. Our users looking forward to the next branch-3 release as branch-4 seems a bit too far, this feature is very important for users who are staying on branch-3 with storagehandlers. We will watch branch-3 progress and not making addons until tests are stabilized. Issue Time Tracking --- Worklog Id: (was: 846230) Time Spent: 1h (was: 50m) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27090) Test fix for external_jdbc_auth.q
[ https://issues.apache.org/jira/browse/HIVE-27090?focusedWorklogId=846223&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846223 ] ASF GitHub Bot logged work on HIVE-27090: - Author: ASF GitHub Bot Created on: 17/Feb/23 17:03 Start Date: 17/Feb/23 17:03 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4069: URL: https://github.com/apache/hive/pull/4069#issuecomment-1434931974 @zabetak @abstractdog I could not find Daniel Dai @daijy in the list of contributors so I cannot add him for review (Contributor for HIVE-20651). Please review this and merge if it looks good. Issue Time Tracking --- Worklog Id: (was: 846223) Time Spent: 0.5h (was: 20m) > Test fix for external_jdbc_auth.q > - > > Key: HIVE-27090 > URL: https://issues.apache.org/jira/browse/HIVE-27090 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > The external_jdbc_auth.q test fails due to the following errors : > java.lang.AssertionError: > Client Execution succeeded but contained differences (error code = 1) after > executing external_jdbc_auth.q > 220,221c220,221 > < 20 20 20.0 20.0 > < 20 20 20.0 20.0 > --- > > 20 20 20.0 20.0 20 > > 20 20 20.0 20.0 20 > > This test was newly added in https://issues.apache.org/jira/browse/HIVE-20651 > but the test has continuously failed on branch-3. Needs a fix. No other > change had been committed to this test after this ticket. Since the ticket > was merged in branch-3 without validating the test, fixing it now. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=846222&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846222 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 17/Feb/23 16:58 Start Date: 17/Feb/23 16:58 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1434921476 @zabetak @abstractdog Please ignore my above message. I found the commit that caused this failure. I have reverted that commit and tested it. This is the commit that brought the failure - HIVE-27016 . Pull request - https://github.com/apache/hive/pull/4013. On reverting I can see -rw getting replaced by -r--  This has run in my local so its USER is different. @sunchao @yigress Is the fix I did an expected behaviour as a result of this commit or does there need to be a change in this https://github.com/apache/hive/pull/4013 Issue Time Tracking --- Worklog Id: (was: 846222) Time Spent: 50m (was: 40m) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27077) upgrade hive grammar to Antlr4
[ https://issues.apache.org/jira/browse/HIVE-27077?focusedWorklogId=846214&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846214 ] ASF GitHub Bot logged work on HIVE-27077: - Author: ASF GitHub Bot Created on: 17/Feb/23 16:03 Start Date: 17/Feb/23 16:03 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4058: URL: https://github.com/apache/hive/pull/4058#issuecomment-1434850270 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4058) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4058&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4058&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4058&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4058&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4058&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4058&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 846214) Time Spent: 40m (was: 0.5h) > upgrade hive grammar to Antlr4 > -- > > Key: HIVE-27077 > URL: https://issues.apache.org/jira/browse/HIVE-27077 > Project: Hive > Issue Type: Improvement > Components: Parser >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Introducing new module parser-v4 that hosts hive grammar defined using Antlr4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=846209&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846209 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 17/Feb/23 15:36 Start Date: 17/Feb/23 15:36 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1434816579 @zabetak I think it is either of this PR - https://github.com/apache/hive/pull/4046 or https://github.com/apache/hive/pull/4014. Issue Time Tracking --- Worklog Id: (was: 846209) Time Spent: 40m (was: 0.5h) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27027) Upgrade jettison to 1.5.3 to fix CVE-2022-45693
[ https://issues.apache.org/jira/browse/HIVE-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha resolved HIVE-27027. - Resolution: Fixed Merged to master. Thanks for the patch, [~ssand] !!! > Upgrade jettison to 1.5.3 to fix CVE-2022-45693 > --- > > Key: HIVE-27027 > URL: https://issues.apache.org/jira/browse/HIVE-27027 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Assignee: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27027) Upgrade jettison to 1.5.3 to fix CVE-2022-45693
[ https://issues.apache.org/jira/browse/HIVE-27027?focusedWorklogId=846192&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846192 ] ASF GitHub Bot logged work on HIVE-27027: - Author: ASF GitHub Bot Created on: 17/Feb/23 14:13 Start Date: 17/Feb/23 14:13 Worklog Time Spent: 10m Work Description: pkumarsinha merged PR #4053: URL: https://github.com/apache/hive/pull/4053 Issue Time Tracking --- Worklog Id: (was: 846192) Time Spent: 1h 10m (was: 1h) > Upgrade jettison to 1.5.3 to fix CVE-2022-45693 > --- > > Key: HIVE-27027 > URL: https://issues.apache.org/jira/browse/HIVE-27027 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Assignee: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27027) Upgrade jettison to 1.5.3 to fix CVE-2022-45693
[ https://issues.apache.org/jira/browse/HIVE-27027?focusedWorklogId=846191&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846191 ] ASF GitHub Bot logged work on HIVE-27027: - Author: ASF GitHub Bot Created on: 17/Feb/23 14:10 Start Date: 17/Feb/23 14:10 Worklog Time Spent: 10m Work Description: pkumarsinha commented on PR #4053: URL: https://github.com/apache/hive/pull/4053#issuecomment-1434703511 +1 LGTM Issue Time Tracking --- Worklog Id: (was: 846191) Time Spent: 1h (was: 50m) > Upgrade jettison to 1.5.3 to fix CVE-2022-45693 > --- > > Key: HIVE-27027 > URL: https://issues.apache.org/jira/browse/HIVE-27027 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Assignee: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27091) Add double quotes for tables in PartitionProjectionEvaluator
[
https://issues.apache.org/jira/browse/HIVE-27091?focusedWorklogId=846178&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846178
]
ASF GitHub Bot logged work on HIVE-27091:
-
Author: ASF GitHub Bot
Created on: 17/Feb/23 12:27
Start Date: 17/Feb/23 12:27
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4071:
URL: https://github.com/apache/hive/pull/4071#issuecomment-1434572956
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4071)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4071&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4071&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4071&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=CODE_SMELL)
[2 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4071&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4071&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4071&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 846178)
Time Spent: 20m (was: 10m)
> Add double quotes for tables in PartitionProjectionEvaluator
>
>
> Key: HIVE-27091
> URL: https://issues.apache.org/jira/browse/HIVE-27091
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When PartitionProjectionEvaluator requests partitions against PostgreSQL,
> there throws exception:
> {noformat}
> javax.jdo.JDODataStoreException: Error executing SQL query "select
> "SDS"."LOCATION","PARTITIONS"."CREATE_TIME","SDS"."SD_ID","PARTITIONS"."PART_ID"
> from PARTITIONS left outer join SDS on PARTITIONS."SD_ID" = SDS."SD_ID"
> left outer join SERDES on SDS."SERDE_ID" = SERDES."SERDE_ID" where "PART_ID"
> in (92731,92732,92733,92734,92735,92736) order by "PART_NAME" asc".
> …
> Caused by: org.postgresql.util.PSQLException: ERROR: relation "pa
[jira] [Work logged] (HIVE-27016) Invoke optional output committer in TezProcessor
[ https://issues.apache.org/jira/browse/HIVE-27016?focusedWorklogId=846169&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846169 ] ASF GitHub Bot logged work on HIVE-27016: - Author: ASF GitHub Bot Created on: 17/Feb/23 11:23 Start Date: 17/Feb/23 11:23 Worklog Time Spent: 10m Work Description: zabetak commented on PR #4046: URL: https://github.com/apache/hive/pull/4046#issuecomment-1434503405 Hey @sunchao and @yigress at this point the scope for branch-3 is only to address test failures. This PR along with other recent merges do not seem to address test failures. Please check https://lists.apache.org/thread/grwo0y7w6h4jfo3tld2s05onbp96moy1 and comment there if you have a different opinion. Thanks! Issue Time Tracking --- Worklog Id: (was: 846169) Time Spent: 1h 50m (was: 1h 40m) > Invoke optional output committer in TezProcessor > > > Key: HIVE-27016 > URL: https://issues.apache.org/jira/browse/HIVE-27016 > Project: Hive > Issue Type: Improvement > Components: Query Processor, StorageHandler >Affects Versions: 3.1.3 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > This is backport HIVE-24629 and HIVE-24867, so StorageHandler with their own > OutputCommitter run in tez. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=846164&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846164 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 17/Feb/23 11:13 Start Date: 17/Feb/23 11:13 Worklog Time Spent: 10m Work Description: zabetak commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1434493180 @amanraj2520 Thanks for the patch. Any ideas which commit broke these tests? Can you link the respective JIRA tickets? Issue Time Tracking --- Worklog Id: (was: 846164) Time Spent: 0.5h (was: 20m) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27027) Upgrade jettison to 1.5.3 to fix CVE-2022-45693
[ https://issues.apache.org/jira/browse/HIVE-27027?focusedWorklogId=846135&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846135 ] ASF GitHub Bot logged work on HIVE-27027: - Author: ASF GitHub Bot Created on: 17/Feb/23 10:19 Start Date: 17/Feb/23 10:19 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4053: URL: https://github.com/apache/hive/pull/4053#issuecomment-1434430411 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4053) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4053&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4053&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4053&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4053&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4053&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4053&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 846135) Time Spent: 50m (was: 40m) > Upgrade jettison to 1.5.3 to fix CVE-2022-45693 > --- > > Key: HIVE-27027 > URL: https://issues.apache.org/jira/browse/HIVE-27027 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Assignee: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27088) Using MergeJoin and using filters does not work
[
https://issues.apache.org/jira/browse/HIVE-27088?focusedWorklogId=846132&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846132
]
ASF GitHub Bot logged work on HIVE-27088:
-
Author: ASF GitHub Bot
Created on: 17/Feb/23 10:00
Start Date: 17/Feb/23 10:00
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4070:
URL: https://github.com/apache/hive/pull/4070#issuecomment-1434407876
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4070)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4070&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4070&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4070&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=CODE_SMELL)
[1 Code
Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4070&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4070&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4070&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 846132)
Time Spent: 20m (was: 10m)
> Using MergeJoin and using filters does not work
> ---
>
> Key: HIVE-27088
> URL: https://issues.apache.org/jira/browse/HIVE-27088
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When hive.merge.nway.joins is set to `true` and JOIN is performed with INNER
> and OUTER and a filter exists, normal results cannot be obtained.
> For example:
> {code:java}
> FROM a
> INNER JOIN b
> ON a.id = b.id
> AND (a.code = 1 or a.code = 2)
> LEFT OUTER JOIN c
> ON a.id = c.id{code}
> I think this is also related to the next ticket:
> https://issues.apache.org/jira/browse/HIVE-21322
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27091) Add double quotes for tables in PartitionProjectionEvaluator
[
https://issues.apache.org/jira/browse/HIVE-27091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27091:
--
Labels: pull-request-available (was: )
> Add double quotes for tables in PartitionProjectionEvaluator
>
>
> Key: HIVE-27091
> URL: https://issues.apache.org/jira/browse/HIVE-27091
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When PartitionProjectionEvaluator requests partitions against PostgreSQL,
> there throws exception:
> {noformat}
> javax.jdo.JDODataStoreException: Error executing SQL query "select
> "SDS"."LOCATION","PARTITIONS"."CREATE_TIME","SDS"."SD_ID","PARTITIONS"."PART_ID"
> from PARTITIONS left outer join SDS on PARTITIONS."SD_ID" = SDS."SD_ID"
> left outer join SERDES on SDS."SERDE_ID" = SERDES."SERDE_ID" where "PART_ID"
> in (92731,92732,92733,92734,92735,92736) order by "PART_NAME" asc".
> …
> Caused by: org.postgresql.util.PSQLException: ERROR: relation "partitions"
> does not exist{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27091) Add double quotes for tables in PartitionProjectionEvaluator
[
https://issues.apache.org/jira/browse/HIVE-27091?focusedWorklogId=846122&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846122
]
ASF GitHub Bot logged work on HIVE-27091:
-
Author: ASF GitHub Bot
Created on: 17/Feb/23 09:46
Start Date: 17/Feb/23 09:46
Worklog Time Spent: 10m
Work Description: dengzhhu653 opened a new pull request, #4071:
URL: https://github.com/apache/hive/pull/4071
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
Issue Time Tracking
---
Worklog Id: (was: 846122)
Remaining Estimate: 0h
Time Spent: 10m
> Add double quotes for tables in PartitionProjectionEvaluator
>
>
> Key: HIVE-27091
> URL: https://issues.apache.org/jira/browse/HIVE-27091
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When PartitionProjectionEvaluator requests partitions against PostgreSQL,
> there throws exception:
> {noformat}
> javax.jdo.JDODataStoreException: Error executing SQL query "select
> "SDS"."LOCATION","PARTITIONS"."CREATE_TIME","SDS"."SD_ID","PARTITIONS"."PART_ID"
> from PARTITIONS left outer join SDS on PARTITIONS."SD_ID" = SDS."SD_ID"
> left outer join SERDES on SDS."SERDE_ID" = SERDES."SERDE_ID" where "PART_ID"
> in (92731,92732,92733,92734,92735,92736) order by "PART_NAME" asc".
> …
> Caused by: org.postgresql.util.PSQLException: ERROR: relation "partitions"
> does not exist{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HIVE-27091) Add double quotes for tables in PartitionProjectionEvaluator
[
https://issues.apache.org/jira/browse/HIVE-27091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng reassigned HIVE-27091:
--
> Add double quotes for tables in PartitionProjectionEvaluator
>
>
> Key: HIVE-27091
> URL: https://issues.apache.org/jira/browse/HIVE-27091
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
>
> When PartitionProjectionEvaluator requests partitions against PostgreSQL,
> there throws exception:
> {noformat}
> javax.jdo.JDODataStoreException: Error executing SQL query "select
> "SDS"."LOCATION","PARTITIONS"."CREATE_TIME","SDS"."SD_ID","PARTITIONS"."PART_ID"
> from PARTITIONS left outer join SDS on PARTITIONS."SD_ID" = SDS."SD_ID"
> left outer join SERDES on SDS."SERDE_ID" = SERDES."SERDE_ID" where "PART_ID"
> in (92731,92732,92733,92734,92735,92736) order by "PART_NAME" asc".
> …
> Caused by: org.postgresql.util.PSQLException: ERROR: relation "partitions"
> does not exist{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27027) Upgrade jettison to 1.5.3 to fix CVE-2022-45693
[ https://issues.apache.org/jira/browse/HIVE-27027?focusedWorklogId=846109&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846109 ] ASF GitHub Bot logged work on HIVE-27027: - Author: ASF GitHub Bot Created on: 17/Feb/23 09:30 Start Date: 17/Feb/23 09:30 Worklog Time Spent: 10m Work Description: sss9700 commented on PR #4053: URL: https://github.com/apache/hive/pull/4053#issuecomment-1434373848 > @sss9700 https://nvd.nist.gov/vuln/detail/CVE-2022-40150 Actually in this, it states that the issue is reproducible in version <=1.4.0. It also says that this is under re-analysis. > > Should it be https://nvd.nist.gov/vuln/detail/CVE-2022-45693. ? If so, can you change the commit message accordingly? Updated the commit message. Issue Time Tracking --- Worklog Id: (was: 846109) Time Spent: 40m (was: 0.5h) > Upgrade jettison to 1.5.3 to fix CVE-2022-45693 > --- > > Key: HIVE-27027 > URL: https://issues.apache.org/jira/browse/HIVE-27027 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Assignee: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27027) Upgrade jettison to 1.5.3 to fix CVE-2022-45693
[ https://issues.apache.org/jira/browse/HIVE-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sand Shreeya updated HIVE-27027: Summary: Upgrade jettison to 1.5.3 to fix CVE-2022-45693 (was: Upgrade jettison to 1.5.3 to fix CVE-2022-40150) > Upgrade jettison to 1.5.3 to fix CVE-2022-45693 > --- > > Key: HIVE-27027 > URL: https://issues.apache.org/jira/browse/HIVE-27027 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Assignee: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27087) Fix TestMiniSparkOnYarnCliDriver test failures on branch-3
[ https://issues.apache.org/jira/browse/HIVE-27087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17690270#comment-17690270 ] Aman Raj commented on HIVE-27087: - I do not have an exact list of the CVEs. Netty was downgraded in branch-3.1 from where 3.1.3 release was done. We could have followed that way, but we decided to make netty closer to oss/master(4.1.77.Final) so this was kind of the first stepping stone to upgrade to 4.1.69.Final. As far I debugged there were quite a lot of changes in netty from 4.1.50- to 4.1.50+. Therefore once we have netty to 4.1.69.Final it won't take much effort to bump it up to 4.1.77.Final to match with oss/master. I am also trying different approaches Vihang. But I do not think we should downgrade netty as was discussed extensively on this thread - https://github.com/apache/hive/pull/3859 > Fix TestMiniSparkOnYarnCliDriver test failures on branch-3 > -- > > Key: HIVE-27087 > URL: https://issues.apache.org/jira/browse/HIVE-27087 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > TestMiniSparkOnYarnCliDriver are failing with the error below > [ERROR] 2023-02-16 14:13:08.991 [Driver] SparkContext - Error initializing > SparkContext. > java.lang.RuntimeException: java.lang.NoSuchFieldException: > DEFAULT_TINY_CACHE_SIZE > at > org.apache.spark.network.util.NettyUtils.getPrivateStaticField(NettyUtils.java:131) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.util.NettyUtils.createPooledByteBufAllocator(NettyUtils.java:118) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.init(TransportServer.java:94) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.(TransportServer.java:73) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.TransportContext.createServer(TransportContext.java:114) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.netty.NettyRpcEnv.startServer(NettyRpcEnv.scala:119) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:465) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:464) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2271) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160) > ~[scala-library-2.11.8.jar:?] > at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2263) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory.create(NettyRpcEnv.scala:469) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:57) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:249) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:256) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.(SparkContext.scala:423) > [spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:161) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:536) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_322] > The root cause of the problem is that we upgrade the netty library from > 4.1.17.Final to 4.1.69.Final. The upgraded library does not have > `DEFAULT_TINY_CACHE_SIZE` field > [here|https://github.com/netty/netty/blob/netty-4.1.51.Final/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L46] > which was removed in 4.1.52.Final -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=846097&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846097 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 17/Feb/23 08:53 Start Date: 17/Feb/23 08:53 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1434325920 @zabetak @abstractdog Can you please approve and merge this. All other failures are intermittent or existing issues. Issue Time Tracking --- Worklog Id: (was: 846097) Time Spent: 20m (was: 10m) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27088) Using MergeJoin and using filters does not work
[
https://issues.apache.org/jira/browse/HIVE-27088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-27088:
-
Assignee: Ryu Kobayashi
Status: Patch Available (was: Open)
> Using MergeJoin and using filters does not work
> ---
>
> Key: HIVE-27088
> URL: https://issues.apache.org/jira/browse/HIVE-27088
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When hive.merge.nway.joins is set to `true` and JOIN is performed with INNER
> and OUTER and a filter exists, normal results cannot be obtained.
> For example:
> {code:java}
> FROM a
> INNER JOIN b
> ON a.id = b.id
> AND (a.code = 1 or a.code = 2)
> LEFT OUTER JOIN c
> ON a.id = c.id{code}
> I think this is also related to the next ticket:
> https://issues.apache.org/jira/browse/HIVE-21322
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27088) Using MergeJoin and using filters does not work
[
https://issues.apache.org/jira/browse/HIVE-27088?focusedWorklogId=846092&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846092
]
ASF GitHub Bot logged work on HIVE-27088:
-
Author: ASF GitHub Bot
Created on: 17/Feb/23 08:47
Start Date: 17/Feb/23 08:47
Worklog Time Spent: 10m
Work Description: ryukobayashi opened a new pull request, #4070:
URL: https://github.com/apache/hive/pull/4070
### What changes were proposed in this pull request?
When hive.merge.nway.joins is set to `true` and JOIN is performed with INNER
and OUTER and a filter exists, normal results cannot be obtained.
For example:
```
FROM a
INNER JOIN b
ON a.id = b.id
AND (a.code = 1 or a.code = 2)
LEFT OUTER JOIN c
ON a.id = c.id
```
### Why are the changes needed?
In the case above, it doesn't return normal results.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Add pattern tests to itest for these to work correctly.
Issue Time Tracking
---
Worklog Id: (was: 846092)
Remaining Estimate: 0h
Time Spent: 10m
> Using MergeJoin and using filters does not work
> ---
>
> Key: HIVE-27088
> URL: https://issues.apache.org/jira/browse/HIVE-27088
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Ryu Kobayashi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When hive.merge.nway.joins is set to `true` and JOIN is performed with INNER
> and OUTER and a filter exists, normal results cannot be obtained.
> For example:
> {code:java}
> FROM a
> INNER JOIN b
> ON a.id = b.id
> AND (a.code = 1 or a.code = 2)
> LEFT OUTER JOIN c
> ON a.id = c.id{code}
> I think this is also related to the next ticket:
> https://issues.apache.org/jira/browse/HIVE-21322
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27088) Using MergeJoin and using filters does not work
[
https://issues.apache.org/jira/browse/HIVE-27088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27088:
--
Labels: pull-request-available (was: )
> Using MergeJoin and using filters does not work
> ---
>
> Key: HIVE-27088
> URL: https://issues.apache.org/jira/browse/HIVE-27088
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Ryu Kobayashi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When hive.merge.nway.joins is set to `true` and JOIN is performed with INNER
> and OUTER and a filter exists, normal results cannot be obtained.
> For example:
> {code:java}
> FROM a
> INNER JOIN b
> ON a.id = b.id
> AND (a.code = 1 or a.code = 2)
> LEFT OUTER JOIN c
> ON a.id = c.id{code}
> I think this is also related to the next ticket:
> https://issues.apache.org/jira/browse/HIVE-21322
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27027) Upgrade jettison to 1.5.3 to fix CVE-2022-40150
[ https://issues.apache.org/jira/browse/HIVE-27027?focusedWorklogId=846086&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846086 ] ASF GitHub Bot logged work on HIVE-27027: - Author: ASF GitHub Bot Created on: 17/Feb/23 08:17 Start Date: 17/Feb/23 08:17 Worklog Time Spent: 10m Work Description: pkumarsinha commented on PR #4053: URL: https://github.com/apache/hive/pull/4053#issuecomment-1434281448 @sss9700 https://nvd.nist.gov/vuln/detail/CVE-2022-40150 Actually in this, it states that the issue is reproducible in version <=1.4.0. It also says that this is under re-analysis. Should it be https://nvd.nist.gov/vuln/detail/CVE-2022-45693. ? If so, can you change the commit message accordingly? Issue Time Tracking --- Worklog Id: (was: 846086) Time Spent: 0.5h (was: 20m) > Upgrade jettison to 1.5.3 to fix CVE-2022-40150 > --- > > Key: HIVE-27027 > URL: https://issues.apache.org/jira/browse/HIVE-27027 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Assignee: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
