Github user a-roberts commented on the issue:
https://github.com/apache/spark/pull/13589
Still getting seg faults on Power and with Intel on OpenJDK with this
change when we use spark-submit with two worker instances.
Git diff
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
@@ -490,6 +490,7 @@ class CodegenContext {
addNewFunction(compareFunc, funcCode)
s"this.$compareFunc($c1, $c2)"
case schema: StructType =>
+ INPUT_ROW = "i"
val comparisons = GenerateOrdering.genComparisons(this, schema)
val compareFunc = freshName("compareStruct")
val funcCode: String =
segv occurs four times
aroberts@jtcspark03:~/09JuneClone/Spark-DK$ cat
work/app-20160610093117-0000/*/stderr | grep -C 2 "Segmentation error"
16/06/10 09:34:27 INFO CodeGenerator: Code generated in 115.554506 ms
Unhandled exception
Type=Segmentation error vmState=0x00000000
J9Generic_Signal_Number=00000004 Signal_Number=0000000b
Error_Value=00000000 Signal_Code=00000001
Handler1=00003FFF7B487AA0 Handler2=00003FFF7B312AC0
**offHeap is NOT ENABLED**
spark-env.sh has
export SPARK_WORKER_CORES=2
export SPARK_WORKER_INSTANCES=2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
