[jira] [Updated] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Matt Cheah (Jira)


 [ 
https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Cheah updated SPARK-31798:
---
Description: 
Part of the overall sub-push for shuffle metadata management as proposed in 
[this 
document|https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit].

The shuffle plugin system needs a means for storing custom metadata for shuffle 
readers to eventually use. This task tracks the API changes required for the 
map output writer to return some such metadata back to the driver.

  was:
Part of the overall sub-push for shuffle metadata management as proposed in 
[this 
document|[https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit]]

The shuffle plugin system needs a means for storing custom metadata for shuffle 
readers to eventually use. This task tracks the API changes required for the 
map output writer to return some such metadata back to the driver.


> Return map output metadata from shuffle writers
> ---
>
> Key: SPARK-31798
> URL: https://issues.apache.org/jira/browse/SPARK-31798
> Project: Spark
>  Issue Type: Sub-task
>  Components: Shuffle
>Affects Versions: 3.0.0
>Reporter: Matt Cheah
>Priority: Major
>
> Part of the overall sub-push for shuffle metadata management as proposed in 
> [this 
> document|https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit].
> The shuffle plugin system needs a means for storing custom metadata for 
> shuffle readers to eventually use. This task tracks the API changes required 
> for the map output writer to return some such metadata back to the driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Updated] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Matt Cheah (Jira)


 [ 
https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Cheah updated SPARK-31798:
---
Description: 
Part of the overall sub-push for shuffle metadata management as proposed in 
[this 
document|[https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit]]

The shuffle plugin system needs a means for storing custom metadata for shuffle 
readers to eventually use. This task tracks the API changes required for the 
map output writer to return some such metadata back to the driver.

  was:
Part of the overall sub-push for shuffle metadata management as proposed in 
[this 
document|[https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit]].

The shuffle plugin system needs a means for storing custom metadata for shuffle 
readers to eventually use. This task tracks the API changes required for the 
map output writer to return some such metadata back to the driver.


> Return map output metadata from shuffle writers
> ---
>
> Key: SPARK-31798
> URL: https://issues.apache.org/jira/browse/SPARK-31798
> Project: Spark
>  Issue Type: Sub-task
>  Components: Shuffle
>Affects Versions: 3.0.0
>Reporter: Matt Cheah
>Priority: Major
>
> Part of the overall sub-push for shuffle metadata management as proposed in 
> [this 
> document|[https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit]]
> The shuffle plugin system needs a means for storing custom metadata for 
> shuffle readers to eventually use. This task tracks the API changes required 
> for the map output writer to return some such metadata back to the driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Updated] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Matt Cheah (Jira)


 [ 
https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Cheah updated SPARK-31798:
---
Description: 
Part of the overall sub-push for shuffle metadata management as proposed in 
[this 
document|[https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit]].

The shuffle plugin system needs a means for storing custom metadata for shuffle 
readers to eventually use. This task tracks the API changes required for the 
map output writer to return some such metadata back to the driver.

  was:
Part of the overall sub-push for shuffle metadata management as proposed in[ 
this 
document|[https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit]].

The shuffle plugin system needs a means for storing custom metadata for shuffle 
readers to eventually use. This task tracks the API changes required for the 
map output writer to return some such metadata back to the driver.


> Return map output metadata from shuffle writers
> ---
>
> Key: SPARK-31798
> URL: https://issues.apache.org/jira/browse/SPARK-31798
> Project: Spark
>  Issue Type: Sub-task
>  Components: Shuffle
>Affects Versions: 3.0.0
>Reporter: Matt Cheah
>Priority: Major
>
> Part of the overall sub-push for shuffle metadata management as proposed in 
> [this 
> document|[https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit]].
> The shuffle plugin system needs a means for storing custom metadata for 
> shuffle readers to eventually use. This task tracks the API changes required 
> for the map output writer to return some such metadata back to the driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org