Thanks for your information, Sean and Min! On Tue, Aug 10, 2021 at 9:59 PM Min Shen <ms...@apache.org> wrote:
> Hi Gengliang, > > SPARK-36378 (Switch to using RPCResponse to communicate common block push > failures to the client) should be another one. > This introduces a slight protocol change to push-based shuffle to improve > code robustness and performance, and is almost ready to be committed. > Because of the protocol change, it’s best to include it with 3.2.0 > release. > > Best, > Min > > On Tue, Aug 10, 2021 at 01:13 Gengliang Wang <ltn...@gmail.com> wrote: > >> Hi all, >> >> As of now, there are still some open/in-progress blockers for Spark 3.2.0 >> release: >> >> - Prohibit update mode in native support of session window ( >> SPARK-36463 <https://issues.apache.org/jira/browse/SPARK-36463>) >> - Avoid inlining non-deterministic With-CTEs(SPARK-36447 >> <https://issues.apache.org/jira/browse/SPARK-36447>) >> - Data Source V2: Remove read specific distributions(SPARK-33807 >> <https://issues.apache.org/jira/browse/SPARK-33807>) >> - Support fetching shuffle blocks in batch with i/o encryption( >> SPARK-34827 <https://issues.apache.org/jira/browse/SPARK-34827>) >> - Add a new Maven profile "no-shaded-client" for older Hadoop 3.x >> versions(SPARK-35959 >> <https://issues.apache.org/jira/browse/SPARK-35959>) >> - Review and fix issues in API docs(SPARK-34185 >> <https://issues.apache.org/jira/browse/SPARK-34185>) >> - Introduce the RocksDBStateStoreProvider in the programming guide( >> SPARK-36041 <https://issues.apache.org/jira/browse/SPARK-36041>) >> - Push-based shuffle documentation(SPARK-36374 >> <https://issues.apache.org/jira/browse/SPARK-36374>) >> >> Thus, I propose to cut RC1 next week after all the blockers are resolved. >> If there are any other blockers, please reply to this email. >> >> Thanks >> Gengliang >> >
