I've been mostly lurking for awhile, but I would like to start picking off some
bugs in the Java implementation.In the process of slogging through the build,
I've bumped into various issues. I'm happy to document them in java/README.md
or make any other changes that might be helpful to others. I'm pretty
experienced with Java and Maven, so I think these are not super-obvious, but
let me know if I'm missing something.A lot of these may be Windows-specific. I
normally prefer Linux but just got a new laptop and haven't set it up, but this
experience is giving me a lot of incentive to run screaming back to Linux ;-)
Environment details:- Windows 10- Java 8:openjdk version "1.8.0_282"OpenJDK
Runtime Environment (AdoptOpenJDK)(build 1.8.0_282-b08)OpenJDK 64-Bit Server VM
(AdoptOpenJDK)(build 25.282-b08, mixed mode)- Cygwin environment- Maven 3.6.2
Issues encountered thus far:- Build does require Java 8, not "8 or later" as
stated in java/README.md There's a reference to sun.misc.Unsafe in
memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java
which of course went away in JDK 9. I can update the build instructions.-
The build won't work on Windows because the java/format POM downloads a binary
flatc executables; when I looked, there was no version for Windows, just Linux
and OSX. I wound up downloading Visual Studio and building the flatbuffers
project.- I bumped into what looks like a spurious checkstyle error: it reports
memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java having no
linefeed at the end when it definitely does. I set up Git not to do Windows
conversions, and I checked with various editors and binary dump utilities. One
source says that this because I'm running on Windows, checkstyle actually
expects a CR-LF and throws an error if it doesn't find it! I've worked around
this by disabling the check.- The one thing that I'm stuck on now is failures
on jdbc:
[INFO] Results:[INFO][ERROR] Failures:[ERROR]
JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:209
expected:<45935000> but was:<74735000>[ERROR]
JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:213
expected:<1518439535000> but was:<1518468335000>[ERROR]
JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:205
expected:<-365> but was:<-364>[ERROR]
JdbcToArrowNullTest.testJdbcToArrowValues:123->testDataSets:165->testAllVectorValues:209
expected:<17574> but was:<17573>[ERROR]
JdbcToArrowTest.testJdbcToArrowValues:138->testDataSets:206 expected:<17574>
but was:<17573>[ERROR]
JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit:107->validate:199->assertDateDayVectorValues:277
expected:<17574> but was:<17573>[ERROR]
JdbcToArrowVectorIteratorTest.testJdbcToArrowValues:95->validate:199->assertDateDayVectorValues:277
expected:<17574> but was:<17573>[INFO][ERROR] Tests run: 93, Failures: 7,
Errors: 0, Skipped: 0
I attached the full build output.Looking more closely at these errors, they
seem to be due to the timezone difference; for example, the difference between
74735000 (actual value) and 45935000 (expected) is 2880000, or 8 hours in
milliseconds, which is the PST timezone offset. I see in the pom.xml that
user.timezone is set to UTC. I have seen these types of errors in tests before;
I know there are ways to insulate the test from the user's current timezone but
maybe someone knows what's going on.
Thanks for any input!
[[1;34mINFO[m] Scanning for projects...
[[1;34mINFO[m]
------------------------------------------------------------------------
[[1;34mINFO[m] Detecting the operating system and CPU architecture
[[1;34mINFO[m]
------------------------------------------------------------------------
[[1;34mINFO[m] os.detected.name: windows
[[1;34mINFO[m] os.detected.arch: x86_64
[[1;34mINFO[m] os.detected.version: 10.0
[[1;34mINFO[m] os.detected.version.major: 10
[[1;34mINFO[m] os.detected.version.minor: 0
[[1;34mINFO[m] os.detected.classifier: windows-x86_64
[[1;34mINFO[m]
[1m------------------------------------------------------------------------[m
[[1;34mINFO[m] [1mReactor Build Order:[m
[[1;34mINFO[m]
[[1;34mINFO[m] Arrow JDBC Adapter
[jar]
[[1;34mINFO[m] Arrow Plasma Client
[jar]
[[1;34mINFO[m] Arrow Flight Core
[jar]
[[1;34mINFO[m] Arrow Flight GRPC
[jar]
[[1;34mINFO[m] Arrow AVRO Adapter
[jar]
[[1;34mINFO[m] Arrow Algorithms
[jar]
[[1;34mINFO[m] Arrow Performance Benchmarks
[jar]
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--------------------<
[0;36morg.apache.arrow:arrow-jdbc[0;1m >---------------------[m
[[1;34mINFO[m] [1mBuilding Arrow JDBC Adapter 4.0.0-SNAPSHOT
[1/7][m
[[1;34mINFO[m] [1m--------------------------------[ jar
]---------------------------------[m
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-clean-plugin:3.0.0:clean[m
[1m(default-clean)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m] Deleting C:\projects\arrow\java\adapter\jdbc\target
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mapache-rat-plugin:0.13:check[m
[1m(rat-checks)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m] Enabled default license matchers.
[[1;34mINFO[m] Will parse SCM ignores for exclusions...
[[1;34mINFO[m] Finished adding exclusions from SCM ignore files.
[[1;34mINFO[m] 62 implicit excludes (use -debug for more details).
[[1;34mINFO[m] 36 explicit excludes (use -debug for more details).
[[1;34mINFO[m] 77 resources included (use -debug for more details)
[[1;34mINFO[m] Rat check: Summary over all files. Unapproved: 0, unknown: 0,
generated: 0, approved: 77 licenses.
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-checkstyle-plugin:3.1.0:check[m
[1m(validate)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m] Starting audit...
Audit done.
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mgit-commit-id-plugin:2.2.2:revision[m
[1m(for-jars)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-remote-resources-plugin:1.5:process[m
[1m(process-resource-bundles)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-resources-plugin:2.6:resources[m
[1m(default-resources)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m] Using 'UTF-8' encoding to copy filtered resources.
[[1;34mINFO[m] skip non existing resourceDirectory
C:\projects\arrow\java\adapter\jdbc\src\main\resources
[[1;34mINFO[m] Copying 3 resources
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-compiler-plugin:3.6.2:compile[m
[1m(default-compile)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m] Compiling 28 source files to
C:\projects\arrow\java\adapter\jdbc\target\classes
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-resources-plugin:2.6:testResources[m
[1m(default-testResources)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m] Using 'UTF-8' encoding to copy filtered resources.
[[1;34mINFO[m] Copying 36 resources
[[1;34mINFO[m] Copying 3 resources
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-compiler-plugin:3.6.2:testCompile[m
[1m(default-testCompile)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m] Compiling 12 source files to
C:\projects\arrow\java\adapter\jdbc\target\test-classes
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-surefire-plugin:3.0.0-M3:test[m
[1m(default-test)[m @ [36marrow-jdbc[0;1m ---[m
[[1;34mINFO[m]
[[1;34mINFO[m] -------------------------------------------------------
[[1;34mINFO[m] T E S T S
[[1;34mINFO[m] -------------------------------------------------------
[[1;34mINFO[m] Running
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowCharSetTest[m
[[1;34mINFO[m] Running
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowArrayTest[m
15:25:45.305 [main] INFO o.apache.arrow.memory.BaseAllocator - Debug mode
enabled.
15:25:45.309 [main] INFO o.a.a.m.DefaultAllocationManagerOption - allocation
manager type not specified, using netty as the default type
15:25:45.311 [main] INFO o.apache.arrow.memory.CheckAllocator - Using
DefaultAllocationManager at
memory-netty/4.0.0-SNAPSHOT/arrow-memory-netty-4.0.0-SNAPSHOT.jar!/org/apache/arrow/memory/DefaultAllocationManagerFactory.class
15:25:45.365 [main] INFO o.apache.arrow.memory.BaseAllocator - Debug mode
enabled.
15:25:45.370 [main] INFO o.a.a.m.DefaultAllocationManagerOption - allocation
manager type not specified, using netty as the default type
15:25:45.372 [main] INFO o.apache.arrow.memory.CheckAllocator - Using
DefaultAllocationManager at
memory-netty/4.0.0-SNAPSHOT/arrow-memory-netty-4.0.0-SNAPSHOT.jar!/org/apache/arrow/memory/DefaultAllocationManagerFactory.class
[[1;34mINFO[m] [1;32mTests run: [0;1;32m3[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 0.532 s - in
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowArrayTest[m
[[1;34mINFO[m] Running
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowDataTypesTest[m
[[1;34mINFO[m] [1;32mTests run: [0;1;32m8[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 0.7 s - in
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowCharSetTest[m
[[1;34mINFO[m] Running
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowTimeZoneTest[m
[[1;34mINFO[m] [1;32mTests run: [0;1;32m18[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 0.141 s - in
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowTimeZoneTest[m
[[1;34mINFO[m] Running org.apache.arrow.adapter.jdbc.[1mJdbcFieldInfoTest[m
[[1;34mINFO[m] [1;32mTests run: [0;1;32m2[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 0 s - in
org.apache.arrow.adapter.jdbc.[1mJdbcFieldInfoTest[m
[[1;34mINFO[m] Running
org.apache.arrow.adapter.jdbc.[1mJdbcToArrowConfigTest[m
[[1;34mINFO[m] [1;32mTests run: [0;1;32m11[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 0.001 s - in
org.apache.arrow.adapter.jdbc.[1mJdbcToArrowConfigTest[m
[[1;31mERROR[m] [1;31mTests [0;1mrun: [0;1m36[m, [1;31mFailures:
[0;1;31m3[m, Errors: 0, Skipped: 0, Time elapsed: 0.383 s[1;31m <<<
FAILURE![m - in
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowDataTypesTest[m
[[1;31mERROR[m] testJdbcToArrowValues[7] Time elapsed: 0.012 s <<< FAILURE!
java.lang.AssertionError: expected:<-365> but was:<-364>
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowDataTypesTest.testDataSets(JdbcToArrowDataTypesTest.java:205)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowDataTypesTest.testJdbcToArrowValues(JdbcToArrowDataTypesTest.java:146)
[[1;31mERROR[m] testJdbcToArrowValues[13] Time elapsed: 0.007 s <<< FAILURE!
java.lang.AssertionError: expected:<45935000> but was:<74735000>
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowDataTypesTest.testDataSets(JdbcToArrowDataTypesTest.java:209)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowDataTypesTest.testJdbcToArrowValues(JdbcToArrowDataTypesTest.java:146)
[[1;31mERROR[m] testJdbcToArrowValues[14] Time elapsed: 0.007 s <<< FAILURE!
java.lang.AssertionError: expected:<1518439535000> but was:<1518468335000>
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowDataTypesTest.testDataSets(JdbcToArrowDataTypesTest.java:213)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowDataTypesTest.testJdbcToArrowValues(JdbcToArrowDataTypesTest.java:146)
[[1;34mINFO[m] Running
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowNullTest[m
[[1;31mERROR[m] [1;31mTests [0;1mrun: [0;1m6[m, [1;31mFailures:
[0;1;31m1[m, Errors: 0, Skipped: 0, Time elapsed: 0.224 s[1;31m <<<
FAILURE![m - in org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowNullTest[m
[[1;31mERROR[m] testJdbcToArrowValues[2] Time elapsed: 0.019 s <<< FAILURE!
java.lang.AssertionError: expected:<17574> but was:<17573>
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowNullTest.testAllVectorValues(JdbcToArrowNullTest.java:209)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowNullTest.testDataSets(JdbcToArrowNullTest.java:165)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowNullTest.testJdbcToArrowValues(JdbcToArrowNullTest.java:123)
[[1;34mINFO[m] Running org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowTest[m
[[1;31mERROR[m] [1;31mTests [0;1mrun: [0;1m3[m, [1;31mFailures:
[0;1;31m1[m, Errors: 0, Skipped: 0, Time elapsed: 3.13 s[1;31m <<<
FAILURE![m - in org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowTest[m
[[1;31mERROR[m] testJdbcToArrowValues[0] Time elapsed: 0.02 s <<< FAILURE!
java.lang.AssertionError: expected:<17574> but was:<17573>
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowTest.testDataSets(JdbcToArrowTest.java:206)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowTest.testJdbcToArrowValues(JdbcToArrowTest.java:138)
[[1;34mINFO[m] Running
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowVectorIteratorTest[m
[[1;31mERROR[m] [1;31mTests [0;1mrun: [0;1m6[m, [1;31mFailures:
[0;1;31m2[m, Errors: 0, Skipped: 0, Time elapsed: 3.069 s[1;31m <<<
FAILURE![m - in
org.apache.arrow.adapter.jdbc.h2.[1mJdbcToArrowVectorIteratorTest[m
[[1;31mERROR[m] testJdbcToArrowValuesNoLimit[0] Time elapsed: 0.018 s <<<
FAILURE!
java.lang.AssertionError: expected:<17574> but was:<17573>
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowVectorIteratorTest.assertDateDayVectorValues(JdbcToArrowVectorIteratorTest.java:277)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowVectorIteratorTest.validate(JdbcToArrowVectorIteratorTest.java:199)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit(JdbcToArrowVectorIteratorTest.java:107)
[[1;31mERROR[m] testJdbcToArrowValues[0] Time elapsed: 0.067 s <<< FAILURE!
java.lang.AssertionError: expected:<17574> but was:<17573>
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowVectorIteratorTest.assertDateDayVectorValues(JdbcToArrowVectorIteratorTest.java:277)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowVectorIteratorTest.validate(JdbcToArrowVectorIteratorTest.java:199)
at
org.apache.arrow.adapter.jdbc.h2.JdbcToArrowVectorIteratorTest.testJdbcToArrowValues(JdbcToArrowVectorIteratorTest.java:95)
[[1;34mINFO[m]
[[1;34mINFO[m] Results:
[[1;34mINFO[m]
[[1;31mERROR[m] [1;31mFailures: [m
[[1;31mERROR[m] [1;31m
JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:209
expected:<45935000> but was:<74735000>[m
[[1;31mERROR[m] [1;31m
JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:213
expected:<1518439535000> but was:<1518468335000>[m
[[1;31mERROR[m] [1;31m
JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:205
expected:<-365> but was:<-364>[m
[[1;31mERROR[m] [1;31m
JdbcToArrowNullTest.testJdbcToArrowValues:123->testDataSets:165->testAllVectorValues:209
expected:<17574> but was:<17573>[m
[[1;31mERROR[m] [1;31m
JdbcToArrowTest.testJdbcToArrowValues:138->testDataSets:206 expected:<17574>
but was:<17573>[m
[[1;31mERROR[m] [1;31m
JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit:107->validate:199->assertDateDayVectorValues:277
expected:<17574> but was:<17573>[m
[[1;31mERROR[m] [1;31m
JdbcToArrowVectorIteratorTest.testJdbcToArrowValues:95->validate:199->assertDateDayVectorValues:277
expected:<17574> but was:<17573>[m
[[1;34mINFO[m]
[[1;31mERROR[m] [1;31mTests run: 93, Failures: 7, Errors: 0, Skipped: 0[m
[[1;34mINFO[m]
[[1;34mINFO[m]
[1m------------------------------------------------------------------------[m
[[1;34mINFO[m] [1mReactor Summary for Arrow JDBC Adapter 4.0.0-SNAPSHOT:[m
[[1;34mINFO[m]
[[1;34mINFO[m] Arrow JDBC Adapter .................................
[1;31mFAILURE[m [ 21.395 s]
[[1;34mINFO[m] Arrow Plasma Client ................................
[1;33mSKIPPED[m
[[1;34mINFO[m] Arrow Flight Core ..................................
[1;33mSKIPPED[m
[[1;34mINFO[m] Arrow Flight GRPC ..................................
[1;33mSKIPPED[m
[[1;34mINFO[m] Arrow AVRO Adapter .................................
[1;33mSKIPPED[m
[[1;34mINFO[m] Arrow Algorithms ...................................
[1;33mSKIPPED[m
[[1;34mINFO[m] Arrow Performance Benchmarks .......................
[1;33mSKIPPED[m
[[1;34mINFO[m]
[1m------------------------------------------------------------------------[m
[[1;34mINFO[m] [1;31mBUILD FAILURE[m
[[1;34mINFO[m]
[1m------------------------------------------------------------------------[m
[[1;34mINFO[m] Total time: 21.751 s
[[1;34mINFO[m] Finished at: 2021-03-11T15:25:53-08:00
[[1;34mINFO[m]
[1m------------------------------------------------------------------------[m
[[1;31mERROR[m] Failed to execute goal
[32morg.apache.maven.plugins:maven-surefire-plugin:3.0.0-M3:test[m
[1m(default-test)[m on project [36marrow-jdbc[m: [1;31mThere are test
failures.[m
[[1;31mERROR[m] [1;31m[m
[[1;31mERROR[m] [1;31mPlease refer to
C:\projects\arrow\java\adapter\jdbc\target\surefire-reports for the individual
test results.[m
[[1;31mERROR[m] [1;31mPlease refer to dump files (if any exist) [date].dump,
[date]-jvmRun[N].dump and [date].dumpstream.[m
[[1;31mERROR[m] -> [1m[Help 1][m
[[1;31mERROR[m]
[[1;31mERROR[m] To see the full stack trace of the errors, re-run Maven with
the [1m-e[m switch.
[[1;31mERROR[m] Re-run Maven using the [1m-X[m switch to enable full debug
logging.
[[1;31mERROR[m]
[[1;31mERROR[m] For more information about the errors and possible solutions,
please read the following articles:
[[1;31mERROR[m] [1m[Help 1][m
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
