[
https://issues.apache.org/jira/browse/HIVE-27848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis updated HIVE-27848:
---------------------------------------
Description:
The work started initially to fix the
TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing.
However, while changing the code to address the failure, the inheritance based
design for the Initator that was chosen in HIVE-27598 revealed some weaknesses
briefly outlined below.
Due to inheritance the InitiatorBase class becomes a Thread something that
doesn't really make sense and it comes with additional overhead every time we
instantiate it. Moreover, the only class that currently extends InitiatorBase
is the Initiator and it's difficult to imagine how we can make other extensions
from InitiatorBase; the code becomes complex and any subtle change in
InitiatorBase may have unpredictable effects on Initiator. Having a "Base"
class that is not really meant to be extended and no instructions on how to do
so is problematic.
For the reasons above the focus of the work changed from just re-enabling the
test to improving and addressing the shortcomings of the inheritance based
design of Initiator.
{noformat}
cd itest/hive-unit
mvn test
-Dtest=TestCrudCompactorOnTez#secondCompactionShouldBeRefusedBeforeEnqueueing
{noformat}
{noformat}
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 39.548
s <<< FAILURE! - in
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
[ERROR]
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing
Time elapsed: 39.444 s <<< FAILURE!
org.junit.ComparisonFailure: expected:<...ault.compaction_test[] is refused,
details...> but was:<...ault.compaction_test[CompactionId: -1] is refused,
details...>
at org.junit.Assert.assertEquals(Assert.java:117)
at org.junit.Assert.assertEquals(Assert.java:146)
at
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing(TestCrudCompactorOnTez.java:638)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
at
org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
{noformat}
The problem was discovered while trying to re-enable the tests under hive-unit
module (HIVE-27846).
was:
{noformat}
cd itest/hive-unit
mvn test
-Dtest=TestCrudCompactorOnTez#secondCompactionShouldBeRefusedBeforeEnqueueing
{noformat}
{noformat}
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 39.548
s <<< FAILURE! - in
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
[ERROR]
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing
Time elapsed: 39.444 s <<< FAILURE!
org.junit.ComparisonFailure: expected:<...ault.compaction_test[] is refused,
details...> but was:<...ault.compaction_test[CompactionId: -1] is refused,
details...>
at org.junit.Assert.assertEquals(Assert.java:117)
at org.junit.Assert.assertEquals(Assert.java:146)
at
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing(TestCrudCompactorOnTez.java:638)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
at
org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
{noformat}
The problem was discovered while trying to re-enable the tests under hive-unit
module (HIVE-27846).
> Refactor Initiator hierarchy into CompactorUtil and fix failure in
> TestCrudCompactorOnTez
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-27848
> URL: https://issues.apache.org/jira/browse/HIVE-27848
> Project: Hive
> Issue Type: Bug
> Reporter: Stamatis Zampetakis
> Assignee: Taraka Rama Rao Lethavadla
> Priority: Major
> Labels: pull-request-available
>
> The work started initially to fix the
> TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing.
> However, while changing the code to address the failure, the inheritance
> based design for the Initator that was chosen in HIVE-27598 revealed some
> weaknesses briefly outlined below.
> Due to inheritance the InitiatorBase class becomes a Thread something that
> doesn't really make sense and it comes with additional overhead every time we
> instantiate it. Moreover, the only class that currently extends InitiatorBase
> is the Initiator and it's difficult to imagine how we can make other
> extensions from InitiatorBase; the code becomes complex and any subtle change
> in InitiatorBase may have unpredictable effects on Initiator. Having a "Base"
> class that is not really meant to be extended and no instructions on how to
> do so is problematic.
> For the reasons above the focus of the work changed from just re-enabling the
> test to improving and addressing the shortcomings of the inheritance based
> design of Initiator.
> {noformat}
> cd itest/hive-unit
> mvn test
> -Dtest=TestCrudCompactorOnTez#secondCompactionShouldBeRefusedBeforeEnqueueing
> {noformat}
> {noformat}
> [INFO] -------------------------------------------------------
> [INFO] Running org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 39.548 s <<< FAILURE! - in
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez
> [ERROR]
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing
> Time elapsed: 39.444 s <<< FAILURE!
> org.junit.ComparisonFailure: expected:<...ault.compaction_test[] is refused,
> details...> but was:<...ault.compaction_test[CompactionId: -1] is refused,
> details...>
> at org.junit.Assert.assertEquals(Assert.java:117)
> at org.junit.Assert.assertEquals(Assert.java:146)
> at
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.secondCompactionShouldBeRefusedBeforeEnqueueing(TestCrudCompactorOnTez.java:638)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
> at
> org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
> {noformat}
> The problem was discovered while trying to re-enable the tests under
> hive-unit module (HIVE-27846).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
