Github user michaelkamprath commented on the issue:
https://github.com/apache/spark/pull/16403
@srowen To answer the use case question, it is primarily academic for
learning and testing. Students and researchers build clusters of Raspberry PI,
ODROID, or other SBCs to have a cost effective access to a multi-node hardware
cluster. [Here](http://likemagicappears.com/projects/raspberry-pi-cluster/)
[are](http://coen.boisestate.edu/ece/research-areas/raspberry-pi/)
[some](https://www.raspberrypi.org/magpi/pi-spark-supercomputer/)
[examples](http://hackaday.com/2016/05/09/designing-a-high-performance-parallel-personal-cluster/)
[of](http://katie.atomicburn.com/2016/06/12/2016-school-gt-exhibition-raspberry-piodroid-c2-supercomputer/)
[projects](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4803722/). There is
even [a commercial vendor](https://www.picocluster.com/collections/) selling
these SBC clusters. [In my own
case](http://diybigdata.net/odroid-xu4-cluster/), its being used to
economically learn how to deal problems of efficiency (it's easier to spot and
work through patter
ns of inefficiency on constrained systems than full powered systems).
I am personally not aware if there are currently any server-class CPUs that
requires double alignment. Double alignment SPARC processors used to be the
bane of my existence in the early 2000's, but that was over a decade ago. My
understanding is that today x86 supports unaligned double access with [a
theoretical performance
hit](https://developers.redhat.com/blog/2016/06/01/how-to-avoid-wasting-megabytes-of-memory-a-few-bytes-at-a-time/)
that [in practice is rarely
seen](http://lemire.me/blog/2012/05/31/data-alignment-for-speed-myth-or-reality/).
Typically you never concern yourself with alignment in Java because the JVM
takes care of it for you, but here we are delving into the world of Unsafe,
which bypasses the protections the JVM provides. Admittedly, it took me a long
while to even figure out that my problem was related to alignment because as
indicated, I haven't dealt with such issues in over a decade.
With all that said, maybe a better approach here is to create a patch that
users can use to create a spark build when they want to run Spark on a system
that requires double alignment, which to the best of my knowledge are currently
just ARM 32-bit CPUs. That would even let to be more concise, without needing
to determine at runtime which method to use. And if ever should arise a
server-class CPU with alignment requirements, we know what to do.
Given that
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
