[
https://issues.apache.org/jira/browse/ARROW-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki updated ARROW-3476:
------------------------------------
Description:
Apache Arrow is becoming commonplace to exchange data among important emerging
analytics frameworks such as Pandas, numpy, and Spark.
[IBM Z|https://en.wikipedia.org/wiki/IBM_Z] is one of platforms to process
critical transactions such as bank or credit card. Users of IBM Z want to
extract insights from these transactions using these emerging analytics systems
on IBM Z Linux. These analytics pipelines can be also fast and effective on IBM
Z Linux by using Apache Arrow on memory.
Since IBM Z Linux uses big-endian data format, it is not possible to use Apache
Arrow in this pipeline. If Apache Arrow could support big-endian, the use case
would be expanded.
When I ran test case of Apache arrow on a big-endian platform (ppc64be), {{mvn
test}} in memory causes a failure due to an assertion.
In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during
an allocation of a {{RootAllocator}} class.
{code}
$ uname -a
Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC
2016 ppc64 ppc64 ppc64 GNU/Linux
$ arch
ppc64
$ cd java/memory
$ mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Arrow Memory 0.12.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
...
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 s
- in org.apache.arrow.memory.TestAccountant
[INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 s
- in org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Running org.apache.arrow.memory.TestBaseAllocator
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746 s
<<< FAILURE! - in org.apache.arrow.memory.TestEndianess
[ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess) Time elapsed:
0.313 s <<< ERROR!
java.lang.ExceptionInInitializerError
at
org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian
systems.
at
org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
[ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed: 0.055
s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
...
{code}
was:
On a big-endian platform, {{mvn test}} in memory causes a failure due to an
assertion.
In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during
an allocation of a {{RootAllocator}} class.
{code}
$ uname -a
Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC
2016 ppc64 ppc64 ppc64 GNU/Linux
$ arch
ppc64
$ cd java/memory
$ mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Arrow Memory 0.12.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
...
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 s
- in org.apache.arrow.memory.TestAccountant
[INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 s
- in org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Running org.apache.arrow.memory.TestBaseAllocator
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746 s
<<< FAILURE! - in org.apache.arrow.memory.TestEndianess
[ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess) Time elapsed:
0.313 s <<< ERROR!
java.lang.ExceptionInInitializerError
at
org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian
systems.
at
org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
[ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed: 0.055
s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
...
{code}
> [Java] mvn test in memory fails on a big-endian platform
> --------------------------------------------------------
>
> Key: ARROW-3476
> URL: https://issues.apache.org/jira/browse/ARROW-3476
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
> Reporter: Kazuaki Ishizaki
> Priority: Major
>
> Apache Arrow is becoming commonplace to exchange data among important
> emerging analytics frameworks such as Pandas, numpy, and Spark.
> [IBM Z|https://en.wikipedia.org/wiki/IBM_Z] is one of platforms to process
> critical transactions such as bank or credit card. Users of IBM Z want to
> extract insights from these transactions using these emerging analytics
> systems on IBM Z Linux. These analytics pipelines can be also fast and
> effective on IBM Z Linux by using Apache Arrow on memory.
> Since IBM Z Linux uses big-endian data format, it is not possible to use
> Apache Arrow in this pipeline. If Apache Arrow could support big-endian, the
> use case would be expanded.
> When I ran test case of Apache arrow on a big-endian platform (ppc64be),
> {{mvn test}} in memory causes a failure due to an assertion.
> In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during
> an allocation of a {{RootAllocator}} class.
> {code}
> $ uname -a
> Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC
> 2016 ppc64 ppc64 ppc64 GNU/Linux
> $ arch
> ppc64
> $ cd java/memory
> $ mvn test
> [INFO] Scanning for projects...
> [INFO]
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Building Arrow Memory 0.12.0-SNAPSHOT
> [INFO]
> ------------------------------------------------------------------------
> [INFO]
> ...
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082
> s - in org.apache.arrow.memory.TestAccountant
> [INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012
> s - in org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Running org.apache.arrow.memory.TestBaseAllocator
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746
> s <<< FAILURE! - in org.apache.arrow.memory.TestEndianess
> [ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess) Time
> elapsed: 0.313 s <<< ERROR!
> java.lang.ExceptionInInitializerError
> at
> org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian
> systems.
> at
> org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> [ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed:
> 0.055 s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
