[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442620#comment-16442620
]
Apache Spark commented on SPARK-23989:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442568#comment-16442568
]
Wenchen Fan commented on SPARK-23989:
-
OK now I see the problem,
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442514#comment-16442514
]
Wenchen Fan commented on SPARK-23989:
-
it goes to `SortShuffleWriter` and then? We get the correct
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442176#comment-16442176
]
liuxian commented on SPARK-23989:
-
test({color:#6a8759}"groupBy"{color}) {
{color:#808080}
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441997#comment-16441997
]
Wenchen Fan commented on SPARK-23989:
-
You have to provide an end-to-end use case to convince us this
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441982#comment-16441982
]
liuxian commented on SPARK-23989:
-
We assume that: numPartitions >
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441980#comment-16441980
]
liuxian commented on SPARK-23989:
-
{color:#9876aa}I think '{color:#33}SortShuffleWriter{color}'
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441957#comment-16441957
]
Saisai Shao commented on SPARK-23989:
-
I've no idea what are you trying to express.
What specific
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441952#comment-16441952
]
liuxian commented on SPARK-23989:
-
1. Make 'BypassMergeSortShuffleHandle' and 'SerializedShuffleHandle'
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441906#comment-16441906
]
Saisai Shao commented on SPARK-23989:
-
Please provide a reproducible case.
Did you reuse the object
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440250#comment-16440250
]
liuxian commented on SPARK-23989:
-
If we make 'BypassMergeSortShuffleHandle' and
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439384#comment-16439384
]
Wenchen Fan commented on SPARK-23989:
-
do you have an end-to-end case to show this bug? IIRC we
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439231#comment-16439231
]
liuxian commented on SPARK-23989:
-
For {color:#33}`SortShuffleWriter`{color}, `records:
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439205#comment-16439205
]
Wenchen Fan commented on SPARK-23989:
-
Can you be more specific about what the problem is?
> When
[
https://issues.apache.org/jira/browse/SPARK-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439148#comment-16439148
]
liuxian commented on SPARK-23989:
-
[~joshrosen]
> When using `SortShuffleWriter`, the data will be
15 matches
Mail list logo
