Hi Bob, Thanks for reporting the issues. I remember encountering the same problems with the JDBC tests (over one year ago).
Maybe it is not just related to the time zone, it is also related to the machine locale. I think we can open an issue to track it. Best, Liya Fan On Fri, Mar 12, 2021 at 12:09 PM Micah Kornfield <emkornfi...@gmail.com> wrote: > Hi Bob, > Thanks for some feedback, I don't think a lot of people are developing on > windows. Some answers in line: > > * Build does require Java 8, not "8 or later" as stated in java/README.md > > There's a reference to sun.misc.Unsafe > > in > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java > > which of course went away in JDK 9. > > Is this the case even using "-Dio.netty.tryReflectionSetAccessible=true" as > mentioned in the README? > > > * The build won't work on Windows because the java/format POM downloads a > > binary flatc executable, but there's no artifact for Windows, just Linux > > and OSX. I wound up downloading Visual Studio and building the > flatbuffers > > project. > > This unfortunately sounds familiar, I think this indicate the popularity on > windows. I also think the hosting of the mac and linux are not exactly > official (they are hosted by a former contributor). Updating the Readme > might be a good first step with instructions on how to do this. > > I see in the pom.xml that user.timezone is set to UTC. I have seen these > > types of errors in tests before; I know there are ways to insulate the > test > > from the user's current timezone but maybe someone knows what's going on > > This is somewhat surprising. I would thought we had the user.timezone set > for exactly this reason. There might have been a regression, this might > make a good second contribution if you wanted to look into fixing it. > > * I bumped into what looks like a spurious checkstyle error: it reports > > memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java having no > > linefeed at the end when it definitely does. I set up Git not to do > Windows > > conversions, and I checked with various editors and binary dump > utilities. > > One source says that this because I'm running on Windows, checkstyle > > actually expects a CR-LF and throws an error if it doesn't find it! I've > > worked around this by disabling the check. > > It looks like we can force the checker to assume linux line feeds: > > https://stackoverflow.com/questions/997021/how-to-get-rid-of-checkstyle-message-file-does-not-end-with-a-newline > (second answer). This would also make a good contribution for someone new. > > Cheers, > Micah > > > > > > > > > > > On Thu, Mar 11, 2021 at 7:14 PM bobtins <bobti...@gmail.com> wrote: > > > My mail client took out all the linefeeds, so let me reformat; sorry > about > > that! > > > > > > In the process of slogging through the build, I've bumped into various > > issues. I'm happy to document them in java/README.md or make any other > > changes that might be helpful to others. > > > > I'm pretty experienced with Java and Maven, so I think these are not > > beginner's mistakes, but let me know if I'm missing something. > > > > A lot of these may be Windows-specific. I normally prefer Linux but just > > got a new laptop and haven't set it up, but this experience is giving me > a > > lot of incentive to run screaming back to Linux ;-) > > > > Environment details: > > * Windows 10 > > * Java 8 > > here's the output of java -version: > > openjdk version "1.8.0_282" > > OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_282-b08) > > OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.282-b08, mixed mode) > > > > * Cygwin environment > > * Maven 3.6.2 > > > > > > Issues encountered thus far: > > > > * Build does require Java 8, not "8 or later" as stated in java/README.md > > There's a reference to sun.misc.Unsafe > > in > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java > > which of course went away in JDK 9. > > * The build won't work on Windows because the java/format POM downloads a > > binary flatc executable, but there's no artifact for Windows, just Linux > > and OSX. I wound up downloading Visual Studio and building the > flatbuffers > > project. > > > > * I bumped into what looks like a spurious checkstyle error: it reports > > memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java having no > > linefeed at the end when it definitely does. I set up Git not to do > Windows > > conversions, and I checked with various editors and binary dump > utilities. > > One source says that this because I'm running on Windows, checkstyle > > actually expects a CR-LF and throws an error if it doesn't find it! I've > > worked around this by disabling the check. > > > > * The one thing that I'm stuck on now is failures on the jdbc module: > > [INFO] > > [INFO] Results: > > [INFO] > > [ERROR] Failures: > > [ERROR] > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:209 > > expected:<45935000> but was:<74735000> > > [ERROR] > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:213 > > expected:<1518439535000> but was:<1518468335000> > > [ERROR] > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:205 > > expected:<-365> but was:<-364> > > [ERROR] > > > JdbcToArrowNullTest.testJdbcToArrowValues:123->testDataSets:165->testAllVectorValues:209 > > expected:<17574> but was:<17573> > > [ERROR] JdbcToArrowTest.testJdbcToArrowValues:138->testDataSets:206 > > expected:<17574> but was:<17573> > > [ERROR] > > > JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit:107->validate:199->assertDateDayVectorValues:277 > > expected:<17574> but was:<17573> > > [ERROR] > > > JdbcToArrowVectorIteratorTest.testJdbcToArrowValues:95->validate:199->assertDateDayVectorValues:277 > > expected:<17574> but was:<17573> > > [INFO] > > [ERROR] Tests run: 93, Failures: 7, Errors: 0, Skipped: 0 > > > > I attached the full build output. > > Looking more closely at these errors, they seem to be due to the timezone > > difference; for example, the difference between 74735000 (actual value) > and > > 45935000 (expected) is 2880000, or 8 hours in milliseconds, which is the > > PST timezone offset. > > > > I see in the pom.xml that user.timezone is set to UTC. I have seen these > > types of errors in tests before; I know there are ways to insulate the > test > > from the user's current timezone but maybe someone knows what's going on. > > > > Thanks for any input! > > > > > > On 2021/03/11 23:49:37, Bob Tinsman <bobt...@pacbell.net> wrote: > > > I've been mostly lurking for awhile, but I would like to start picking > > off some bugs in the Java implementation.In the process of slogging > through > > the build, I've bumped into various issues. I'm happy to document them in > > java/README.md or make any other changes that might be helpful to others. > > I'm pretty experienced with Java and Maven, so I think these are not > > super-obvious, but let me know if I'm missing something.A lot of these > may > > be Windows-specific. I normally prefer Linux but just got a new laptop > and > > haven't set it up, but this experience is giving me a lot of incentive to > > run screaming back to Linux ;-) > > > Environment details:- Windows 10- Java 8:openjdk version > > "1.8.0_282"OpenJDK Runtime Environment (AdoptOpenJDK)(build > > 1.8.0_282-b08)OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.282-b08, > > mixed mode)- Cygwin environment- Maven 3.6.2 > > > Issues encountered thus far:- Build does require Java 8, not "8 or > > later" as stated in java/README.md There's a reference to > > sun.misc.Unsafe > > in > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java > > which of course went away in JDK 9. I can update the build > > instructions.- The build won't work on Windows because the java/format > POM > > downloads a binary flatc executables; when I looked, there was no version > > for Windows, just Linux and OSX. I wound up downloading Visual Studio and > > building the flatbuffers project.- I bumped into what looks like a > spurious > > checkstyle error: it > > reports memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java > > having no linefeed at the end when it definitely does. I set up Git not > to > > do Windows conversions, and I checked with various editors and binary > dump > > utilities. One source says that this because I'm running on Windows, > > checkstyle actually expects a CR-LF and throws an error if it doesn't > find > > it! I've worked > > around this by disabling the check.- The one thing that I'm stuck on > now > > is failures on jdbc: > > > [INFO] Results:[INFO][ERROR] Failures:[ERROR] > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:209 > > expected:<45935000> but was:<74735000>[ERROR] > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:213 > > expected:<1518439535000> but was:<1518468335000>[ERROR] > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:205 > > expected:<-365> but was:<-364>[ERROR] > > > JdbcToArrowNullTest.testJdbcToArrowValues:123->testDataSets:165->testAllVectorValues:209 > > expected:<17574> but was:<17573>[ERROR] > > JdbcToArrowTest.testJdbcToArrowValues:138->testDataSets:206 > > expected:<17574> but was:<17573>[ERROR] > > > JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit:107->validate:199->assertDateDayVectorValues:277 > > expected:<17574> but was:<17573>[ERROR] > > > JdbcToArrowVectorIteratorTest.testJdbcToArrowValues:95->validate:199->assertDateDayVectorValues:277 > > expected:<17574> but was:<17573>[INFO][ERROR] Tests run: 93, Failures: 7, > > Errors: 0, Skipped: 0 > > > I attached the full build output.Looking more closely at these errors, > > they seem to be due to the timezone difference; for example, the > difference > > between 74735000 (actual value) and 45935000 (expected) is 2880000, or 8 > > hours in milliseconds, which is the PST timezone offset. I see in the > > pom.xml that user.timezone is set to UTC. I have seen these types of > errors > > in tests before; I know there are ways to insulate the test from the > user's > > current timezone but maybe someone knows what's going on. > > > Thanks for any input! > > > > > >
