LuciferYang commented on code in PR #37853:
URL: https://github.com/apache/spark/pull/37853#discussion_r972570934
##########
docs/running-on-yarn.md:
##########
@@ -852,8 +852,8 @@ The following extra configuration options are available
when the shuffle service
<td><code>spark.shuffle.service.db.backend</code></td>
<td>LEVELDB</td>
<td>
- To specify the kind of disk-base store used in shuffle service state
store, supports `LEVELDB` and `ROCKSDB` now
- and `LEVELDB` as default value.
+ When work-preserving restart is enabled in YARN, this is used to specify
the disk-base store used
+ in shuffle service state store, supports `LEVELDB` and `ROCKSDB` with
`LEVELDB` as default value.
The original data store in `LevelDB/RocksDB` will not be automatically
convert to another kind of storage now.
Review Comment:
> @dongjoon-hyun @mridulm Automatic data format conversion may be a useful
feature. I think it is more friendly for migrating stock users to use new
features. I have filed a Jira
[SPARK-40464](https://issues.apache.org/jira/browse/SPARK-40464) and will
promote its completion if necessary.
also cc @panbingkun , what I discussed with you offline yesterday
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
