[
https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227196#comment-17227196
]
huangtianhua commented on SPARK-32691:
--------------------------------------
We have found the problem, it is take long time to replicate remote over the
default timeout 120 seconds, so it try again to another executor, but in fact
the replication is complete, so there are 3 replications total. Then we found
the progress hang in CryptoRandomFactory.getCryptoRandom(properties), we found
the jar commons-crypto v1.0.0 doesn't support aarch64, after we change to use
v1.1.0 then the tests pass and the time is short.
So I plan to propose a PR to change to use commons-crypto v1.1.0 which support
aarch64: http://commons.apache.org/proper/commons-crypto/changes-report.html
https://issues.apache.org/jira/browse/CRYPTO-139
> Test org.apache.spark.DistributedSuite failed on arm64 jenkins
> --------------------------------------------------------------
>
> Key: SPARK-32691
> URL: https://issues.apache.org/jira/browse/SPARK-32691
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Tests
> Affects Versions: 3.1.0
> Environment: ARM64
> Reporter: huangtianhua
> Assignee: zhengruifeng
> Priority: Major
> Attachments: Screen Shot 2020-09-28 at 8.49.04 AM.png, failure.log,
> success.log
>
>
> Tests of org.apache.spark.DistributedSuite are failed on arm64 jenkins:
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-arm/
> - caching in memory and disk, replicated (encryption = on) (with
> replication as stream) *** FAILED ***
> 3 did not equal 2; got 3 replicas instead of 2 (DistributedSuite.scala:191)
> - caching in memory and disk, serialized, replicated (encryption = on)
> (with replication as stream) *** FAILED ***
> 3 did not equal 2; got 3 replicas instead of 2 (DistributedSuite.scala:191)
> - caching in memory, serialized, replicated (encryption = on) (with
> replication as stream) *** FAILED ***
> 3 did not equal 2; got 3 replicas instead of 2 (DistributedSuite.scala:191)
> .......
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
