[
https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17181637#comment-17181637
]
Dongjoon Hyun commented on SPARK-32517:
---------------------------------------
So, the failure situation looks like this. For the following cases,
{code}
"caching in memory, serialized, replicated" -> StorageLevel.MEMORY_ONLY_SER_2,
{code}
The number of replicas should be 2 and it does in Intel CPU. But, in ARM
testing environment, it became 3.
{code}
- caching in memory, serialized, replicated (encryption = off)
- caching in memory, serialized, replicated (encryption = off) (with
replication as stream)
- caching in memory, serialized, replicated (encryption = on)
- caching in memory, serialized, replicated (encryption = on) (with replication
as stream) *** FAILED ***
3 did not equal 2; got 3 replicas instead of 2 (DistributedSuite.scala:191)
{code}
{code}
- caching in memory and disk, serialized, replicated (encryption = off)
- caching in memory and disk, serialized, replicated (encryption = off) (with
replication as stream)
- caching in memory and disk, serialized, replicated (encryption = on)
- caching in memory and disk, serialized, replicated (encryption = on) (with
replication as stream) *** FAILED ***
3 did not equal 2; got 3 replicas instead of 2 (DistributedSuite.scala:191)
{code}
> Add StorageLevel.DISK_ONLY_3
> ----------------------------
>
> Key: SPARK-32517
> URL: https://issues.apache.org/jira/browse/SPARK-32517
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.1.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Fix For: 3.1.0
>
>
> This issue aims to add `StorageLevel.DISK_ONLY_3` as a built-in StorageLevel.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
