[
https://issues.apache.org/jira/browse/CALCITE-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616688#comment-16616688
]
Vladimir Sitnikov commented on CALCITE-1894:
--------------------------------------------
The issue is not a timeout.
The issue is CsvTest.testCsvStream does not perform what it is intended to do.
1) CsvStreamReader constructor has Thread.sleep. I don't think it is
acceptable:
https://github.com/apache/calcite/blob/c12e37d41a9d81b1303a345bad5067e1518d6c7d/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvStreamReader.java#L88
2) Linq4j.EnumeratorIterator calls {{moveNext}} in the constructor, which
effectively fetches the first element.
It causes early fetch of the results.
https://github.com/apache/calcite/blob/d59b639d27da704f00eff616324a2c04aa06f84c/linq4j/src/main/java/org/apache/calcite/linq4j/Linq4j.java#L669-L676
I suggest to use something like
{code:java} Boolean hasNext;
EnumeratorIterator(Enumerator<T> enumerator) {
this.enumerator = enumerator;
}
public boolean hasNext() {
if (hasNext == null) {
hasNext = enumerator.moveNext();
}
return hasNext;
}
public T next() {
if (hasNext == null) {
enumerator.moveNext();
}
T t = enumerator.current();
hasNext = enumerator.moveNext();
return t;
}
{code}
If I correct the above issues, the following exception appears:
{noformat}java.lang.NumberFormatException: For input string: "DEPTNO:int"
at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.base/java.lang.Integer.parseInt(Integer.java:652)
at java.base/java.lang.Integer.parseInt(Integer.java:770)
at
org.apache.calcite.adapter.csv.CsvEnumerator$RowConverter.convert(CsvEnumerator.java:273)
at
org.apache.calcite.adapter.csv.CsvEnumerator$ArrayRowConverter.convertStreamRow(CsvEnumerator.java:367)
at
org.apache.calcite.adapter.csv.CsvEnumerator$ArrayRowConverter.convertRow(CsvEnumerator.java:347)
at
org.apache.calcite.adapter.csv.CsvEnumerator$ArrayRowConverter.convertRow(CsvEnumerator.java:327)
at
org.apache.calcite.adapter.csv.CsvEnumerator.moveNext(CsvEnumerator.java:214)
at
org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1875)
at
org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1875)
at
org.apache.calcite.linq4j.TransformedEnumerator.moveNext(TransformedEnumerator.java:35)
at
org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.hasNext(Linq4j.java:680)
at
org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:45)
at
org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217)
at org.apache.calcite.test.CsvTest.testCsvStream(CsvTest.java:903)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:844)
{noformat}
It looks like it does not detect CSV header, and it tries to parse the first
row as data.
I've no idea how CSV identifies if it should throw-away the first line or not,
however the above exception does look like a true issue.
> CsvTest.testCsvStream failing often
> -----------------------------------
>
> Key: CALCITE-1894
> URL: https://issues.apache.org/jira/browse/CALCITE-1894
> Project: Calcite
> Issue Type: Bug
> Reporter: Rajeshbabu Chintaguntla
> Assignee: Julian Hyde
> Priority: Major
>
> CsvTest.testCsvStream is timing out and failing often.
> This is the build failed because of this test.
> https://travis-ci.org/apache/calcite/jobs/253794426
> {noformat}
> <<< FAILURE! - in org.apache.calcite.test.CsvTest
> testCsvStream(org.apache.calcite.test.CsvTest) Time elapsed: 10.044 sec <<<
> ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 10000
> milliseconds
> at org.apache.calcite.test.CsvTest.testCsvStream(CsvTest.java:729)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
