[
https://issues.apache.org/jira/browse/FLINK-20975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379174#comment-17379174
]
Rui Li commented on FLINK-20975:
--------------------------------
I tried the test locally dozens of times but didn't reproduce it. According to
test logs, the test failed because of:
1. Direct SQL failed due to:
{noformat}
Caused by: org.apache.derby.iapi.error.StandardException: Invalid character
string format for type DECIMAL.
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source) ~[derby-10.10.2.0.jar:?]
at org.apache.derby.iapi.types.DataType.invalidFormat(Unknown Source)
~[derby-10.10.2.0.jar:?]
at org.apache.derby.iapi.types.DataType.setValue(Unknown Source)
~[derby-10.10.2.0.jar:?]
at
org.apache.derby.exe.ac1d3a42f2x017ax8d9bxd637x0000338c7ac889.e4(Unknown
Source) ~[?:?]
at org.apache.derby.impl.services.reflect.DirectCall.invoke(Unknown
Source) ~[derby-10.10.2.0.jar:?]
at
org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
Source) ~[derby-10.10.2.0.jar:?]
at
org.apache.derby.impl.sql.execute.NestedLoopJoinResultSet.getNextRowCore(Unknown
Source) ~[derby-10.10.2.0.jar:?]
at
org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
Source) ~[derby-10.10.2.0.jar:?]
at
org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown
Source) ~[derby-10.10.2.0.jar:?]
at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown
Source) ~[derby-10.10.2.0.jar:?]
at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source)
~[derby-10.10.2.0.jar:?]
at
org.datanucleus.store.rdbms.query.ForwardQueryResult.initialise(ForwardQueryResult.java:99)
~[datanucleus-rdbms-4.1.19.jar:?]
at
org.datanucleus.store.rdbms.query.SQLQuery.performExecute(SQLQuery.java:676)
~[datanucleus-rdbms-4.1.19.jar:?]
at org.datanucleus.store.query.Query.executeQuery(Query.java:1855)
~[datanucleus-core-4.1.17.jar:?]
at
org.datanucleus.store.rdbms.query.SQLQuery.executeWithArray(SQLQuery.java:807)
~[datanucleus-rdbms-4.1.19.jar:?]
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:368)
~[datanucleus-api-jdo-4.2.4.jar:?]
... 121 more
{noformat}
Since the partition column type is INT and should be safe to cast to decimal,
this might be a derby bug.
2. Then hive tried to fallback to JDO method to pushdown the filter. But failed
again because by default "Filtering is supported only on partition keys of type
string".
3. Finally the pushdown is aborted and we fallback to listing all partitions in
metastore, and thus the test failure.
I guess we can enable filter pushdown on integral partition column types so
that we'll have a 2nd chance in step #2
> HiveTableSourceITCase.testPartitionFilter fails on AZP
> ------------------------------------------------------
>
> Key: FLINK-20975
> URL: https://issues.apache.org/jira/browse/FLINK-20975
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
> Affects Versions: 1.13.0
> Reporter: Till Rohrmann
> Priority: Major
> Labels: auto-deprioritized-critical, auto-unassigned,
> test-stability
> Fix For: 1.13.2
>
>
> The test {{HiveTableSourceITCase.testPartitionFilter}} fails on AZP with the
> following exception:
> {code}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at org.junit.Assert.assertFalse(Assert.java:74)
> at
> org.apache.flink.connectors.hive.HiveTableSourceITCase.testPartitionFilter(HiveTableSourceITCase.java:278)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
