XiDuo You created SPARK-35786:
-
Summary: Support adaptive repartition
Key: SPARK-35786
URL: https://issues.apache.org/jira/browse/SPARK-35786
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-35786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35786:
--
Summary: Support optimize repartition by expression in AQE (was: Support
adaptive repartition by
[
https://issues.apache.org/jira/browse/SPARK-35786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35786:
--
Summary: Support adaptive repartition with repartition by expression (was:
Support adaptive
[
https://issues.apache.org/jira/browse/SPARK-35786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35786:
--
Summary: Support adaptive repartition by expression (was: Support adaptive
repartition with
XiDuo You created SPARK-35687:
-
Summary: PythonUDFSuite move assume into its methods
Key: SPARK-35687
URL: https://issues.apache.org/jira/browse/SPARK-35687
Project: Spark
Issue Type:
XiDuo You created SPARK-35675:
-
Summary: EnsureRequirements remove shuffle should respect
PartitioningCollection
Key: SPARK-35675
URL: https://issues.apache.org/jira/browse/SPARK-35675
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-33832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359190#comment-17359190
]
XiDuo You commented on SPARK-33832:
---
hi [~ekoifman] I created a PR for this issue, do you mind taking
[
https://issues.apache.org/jira/browse/SPARK-35675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35675:
--
Description:
Currently EnsureRequirements only check if child has semantic equal
HashPartitioning
XiDuo You created SPARK-35813:
-
Summary: Add new adaptive config into sql-performance-tuning docs
Key: SPARK-35813
URL: https://issues.apache.org/jira/browse/SPARK-35813
Project: Spark
Issue
XiDuo You created SPARK-35853:
-
Summary: Remark the shuffle origin to ENSURE_REQUIREMENTS as far
as possible
Key: SPARK-35853
URL: https://issues.apache.org/jira/browse/SPARK-35853
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-35853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35853:
--
Description:
In some queries, we might repartition by some columns with a large partition
number
XiDuo You created SPARK-35888:
-
Summary: Add dataSize field in CoalescedPartitionSpec
Key: SPARK-35888
URL: https://issues.apache.org/jira/browse/SPARK-35888
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-35888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35888:
--
Description:
Currently, all test suite about `CoalescedPartitionSpec` do not check the data
size due
[
https://issues.apache.org/jira/browse/SPARK-35888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35888:
--
Description:
Currently, all test suite about `CoalescedPartitionSpec` do not check the data
size due
[
https://issues.apache.org/jira/browse/SPARK-35786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35786:
--
Description:
Add a new hint to distingush if we can optimize it safely.
was:Currently, we only
[
https://issues.apache.org/jira/browse/SPARK-35786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35786:
--
Summary: Add a new operator to distingush if AQE can optimize safely (was:
Add a new hint to
[
https://issues.apache.org/jira/browse/SPARK-35786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35786:
--
Summary: Add a new hint to distingush if AQE can optimize safely (was:
Support optimize repartition
XiDuo You created SPARK-35725:
-
Summary: Support repartition expand partitions
Key: SPARK-35725
URL: https://issues.apache.org/jira/browse/SPARK-35725
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-35725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35725:
--
Parent: SPARK-33828
Issue Type: Sub-task (was: Improvement)
> Support repartition expand
[
https://issues.apache.org/jira/browse/SPARK-35725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35725:
--
Summary: Support repartition expand partitions in AQE (was: Support
repartition expand partitions)
XiDuo You created SPARK-35376:
-
Summary: Fallback config should override defaultValue
Key: SPARK-35376
URL: https://issues.apache.org/jira/browse/SPARK-35376
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-35332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344501#comment-17344501
]
XiDuo You commented on SPARK-35332:
---
[~luxianghao] Now you can `set
[
https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35442:
--
Description:
If Aggregate and Join have the same output partitioning, the plan will look
like:
[
https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35442:
--
Description:
If Aggregate and Join have the same output partitioning, the plan will look
like:
[
https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35442:
--
Description:
If Aggregate and Join have the same output partitioning, the plan will look
like:
[
https://issues.apache.org/jira/browse/SPARK-35332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341107#comment-17341107
]
XiDuo You commented on SPARK-35332:
---
The reason is Spark force disable the AQE during executing the
XiDuo You created SPARK-35455:
-
Summary: Enhance EliminateUnnecessaryJoin
Key: SPARK-35455
URL: https://issues.apache.org/jira/browse/SPARK-35455
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-35332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341244#comment-17341244
]
XiDuo You commented on SPARK-35332:
---
Adding a new cache-specific option in a CACHE statement seems a
[
https://issues.apache.org/jira/browse/SPARK-35455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35455:
--
Parent: SPARK-33828
Issue Type: Sub-task (was: Improvement)
> Enhance
[
https://issues.apache.org/jira/browse/SPARK-35455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35455:
--
Priority: Major (was: Minor)
> Enhance EliminateUnnecessaryJoin
>
>
[
https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You resolved SPARK-35442.
---
Resolution: Duplicate
> Eliminate unnecessary join through Aggregate
>
XiDuo You created SPARK-35608:
-
Summary: Support AQE optimizer side transformUpWithPruning
Key: SPARK-35608
URL: https://issues.apache.org/jira/browse/SPARK-35608
Project: Spark
Issue Type:
XiDuo You created SPARK-35585:
-
Summary: Support propagate empty relation through project/filter
Key: SPARK-35585
URL: https://issues.apache.org/jira/browse/SPARK-35585
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You reopened SPARK-35442:
---
> Eliminate unnecessary join through Aggregate
>
>
>
XiDuo You created SPARK-35552:
-
Summary: Make query stage materialized more readable
Key: SPARK-35552
URL: https://issues.apache.org/jira/browse/SPARK-35552
Project: Spark
Issue Type:
XiDuo You created SPARK-35629:
-
Summary: Drop database should check if exists
Key: SPARK-35629
URL: https://issues.apache.org/jira/browse/SPARK-35629
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-35282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35282:
--
Description:
Use AQE runtime statistics to decide if we can use shuffled hash join instead
of sort
XiDuo You created SPARK-35540:
-
Summary: Make config maxShuffledHashJoinLocalMapThreshold fallback
to advisoryPartitionSizeInBytes
Key: SPARK-35540
URL: https://issues.apache.org/jira/browse/SPARK-35540
[
https://issues.apache.org/jira/browse/SPARK-36014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36014:
--
Description:
Currently, spark on kubernetes with client mode would use `"spark-application-"
+
XiDuo You created SPARK-36014:
-
Summary: Use uuid as app id in kubernetes client mode
Key: SPARK-36014
URL: https://issues.apache.org/jira/browse/SPARK-36014
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35989:
--
Description: The shuffle origin is `REPARTITION_BY_NUM` if user specify an
exact partition number
[
https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35989:
--
Environment: (was: The shuffle origin is `REPARTITION_BY_NUM` if user
specify an exact partition
XiDuo You created SPARK-35989:
-
Summary: Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled
Key: SPARK-35989
URL: https://issues.apache.org/jira/browse/SPARK-35989
Project: Spark
Issue
XiDuo You created SPARK-35961:
-
Summary: Only use local shuffle reader for
REBALANCE_PARTITIONS_BY_NONE without CustomShuffleReaderExec
Key: SPARK-35961
URL: https://issues.apache.org/jira/browse/SPARK-35961
[
https://issues.apache.org/jira/browse/SPARK-35961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35961:
--
Parent: SPARK-35793
Issue Type: Sub-task (was: Improvement)
> Only use local shuffle reader
[
https://issues.apache.org/jira/browse/SPARK-35961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35961:
--
Description:
After [SPARK-35725](https://issues.apache.org/jira/browse/SPARK-35725), we
might expand
[
https://issues.apache.org/jira/browse/SPARK-35961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-35961:
--
Summary: Only use local shuffle reader when REBALANCE_PARTITIONS_BY_NONE
without
XiDuo You created SPARK-35923:
-
Summary: Coalesce empty partition with mixed
CoalescedPartitionSpec and PartialReducerPartitionSpec
Key: SPARK-35923
URL: https://issues.apache.org/jira/browse/SPARK-35923
XiDuo You created SPARK-36085:
-
Summary: Make broadcast query stage executionContext isolation
from AQE
Key: SPARK-36085
URL: https://issues.apache.org/jira/browse/SPARK-36085
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-36085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You resolved SPARK-36085.
---
Resolution: Won't Fix
> Make broadcast query stage executionContext isolation from AQE
>
[
https://issues.apache.org/jira/browse/SPARK-36085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36085:
--
Parent: (was: SPARK-33828)
Issue Type: Improvement (was: Sub-task)
> Make broadcast
XiDuo You created SPARK-36032:
-
Summary: RemoveRedundantSorts should be applied after reOptimize
in AQE
Key: SPARK-36032
URL: https://issues.apache.org/jira/browse/SPARK-36032
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-36032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36032:
--
Summary: Use inputPlan instead of currentPhysicalPlan to initialize logical
link (was:
[
https://issues.apache.org/jira/browse/SPARK-36032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36032:
--
Description:
At {{initialPlan}} we may remove some Spark Plan with
{{queryStagePreparationRules}},
XiDuo You created SPARK-35442:
-
Summary: Eliminate unnecessary join through Aggregate
Key: SPARK-35442
URL: https://issues.apache.org/jira/browse/SPARK-35442
Project: Spark
Issue Type:
XiDuo You created SPARK-36424:
-
Summary: Support eliminate limits in AQE Optimizer
Key: SPARK-36424
URL: https://issues.apache.org/jira/browse/SPARK-36424
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-36993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36993:
--
Affects Version/s: 3.0.3
> Fix json_tupe throw NPE if fields exist no foldable null value
>
XiDuo You created SPARK-36993:
-
Summary: Fix json_tupe throw NPE if fields exist no foldable null
column
Key: SPARK-36993
URL: https://issues.apache.org/jira/browse/SPARK-36993
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-36993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36993:
--
Description:
If json_tuple exists no foldable null field, Spark would throw NPE during eval
[
https://issues.apache.org/jira/browse/SPARK-36993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36993:
--
Summary: Fix json_tupe throw NPE if fields exist no foldable null field
(was: Fix json_tupe throw
[
https://issues.apache.org/jira/browse/SPARK-36993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36993:
--
Summary: Fix json_tupe throw NPE if fields exist no foldable null value
(was: Fix json_tupe throw
XiDuo You created SPARK-36992:
-
Summary: Improve byte array sort perf by unify getPrefix function
of UTF8String and ByteArray
Key: SPARK-36992
URL: https://issues.apache.org/jira/browse/SPARK-36992
XiDuo You created SPARK-36979:
-
Summary: Add RewriteLateralSubquery rule into nonExcludableRules
Key: SPARK-36979
URL: https://issues.apache.org/jira/browse/SPARK-36979
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-37080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37080:
--
Summary: Add benchmark tool guide in pull request template (was: Add
benchmark guide in pull request
XiDuo You created SPARK-37080:
-
Summary: Add benchmark guide in pull request template
Key: SPARK-37080
URL: https://issues.apache.org/jira/browse/SPARK-37080
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-37063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430838#comment-17430838
]
XiDuo You commented on SPARK-37063:
---
thank you [~dongjoon] for creating this umbrella !
> SQL
XiDuo You created SPARK-37064:
-
Summary: Fix outer join return the wrong max rows if other side is
empty
Key: SPARK-37064
URL: https://issues.apache.org/jira/browse/SPARK-37064
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-37037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37037:
--
Description:
BinaryType use `TypeUtils.compareBinary` to compare two byte array, however
it's slow
[
https://issues.apache.org/jira/browse/SPARK-37037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37037:
--
Description:
BinaryType use `TypeUtils.compareBinary` to compare two byte array, however
it's slow
XiDuo You created SPARK-37037:
-
Summary: Improve byte array sort by unify compareTo function of
UTF8String and ByteArray
Key: SPARK-37037
URL: https://issues.apache.org/jira/browse/SPARK-37037
Project:
XiDuo You created SPARK-37043:
-
Summary: Cancel all running job after AQE plan finished
Key: SPARK-37043
URL: https://issues.apache.org/jira/browse/SPARK-37043
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-37043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37043:
--
Description:
We see stage was still running after AQE plan finished. This is because the
plan which
[
https://issues.apache.org/jira/browse/SPARK-36424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36424:
--
Parent: SPARK-33828
Issue Type: Sub-task (was: Improvement)
> Support eliminate limits in
XiDuo You created SPARK-36822:
-
Summary: BroadcastNestedLoopJoinExec should use all condition
instead of non-equi condition
Key: SPARK-36822
URL: https://issues.apache.org/jira/browse/SPARK-36822
XiDuo You created SPARK-36823:
-
Summary: Support broadcast nested loop join hint for equi-join
Key: SPARK-36823
URL: https://issues.apache.org/jira/browse/SPARK-36823
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-36823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-36823:
--
Description:
For the join if one side is small and other side is large, the shuffle overhead
is also
XiDuo You created SPARK-37098:
-
Summary: Alter table properties should invalidate cache
Key: SPARK-37098
URL: https://issues.apache.org/jira/browse/SPARK-37098
Project: Spark
Issue Type: Bug
XiDuo You created SPARK-37559:
-
Summary: ShuffledRowRDD get preferred locations order by reduce
size
Key: SPARK-37559
URL: https://issues.apache.org/jira/browse/SPARK-37559
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-37796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37796:
--
Description:
The method `arrayEquals` in `ByteArrayMethods` is critical function which is
used in
[
https://issues.apache.org/jira/browse/SPARK-37796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37796:
--
Summary: ByteArrayMethods arrayEquals should fast skip the check of
aligning with unaligned platform
XiDuo You created SPARK-37796:
-
Summary: ByteArrayMethods arrayEquals should fast skip the
checking of aligned in unaligned platform
Key: SPARK-37796
URL: https://issues.apache.org/jira/browse/SPARK-37796
XiDuo You created SPARK-37357:
-
Summary: Add merged last partition factor for rebalance
Key: SPARK-37357
URL: https://issues.apache.org/jira/browse/SPARK-37357
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Description:
`Rebalance` provide a functionality that split the large reduce partition into
smalls.
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Description:
`Rebalance` provide a functionality that split the large reduce partition into
smalls.
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Description:
`Rebalance` provide a functionality that split the large reduce partition into
smalls.
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Summary: Add merged last partition factor for split skew partition (was:
Add merged last partition
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Description:
For example `Rebalance` provide a functionality that split the large reduce
partition
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Summary: Add merged last partition factor for rebalance (was: Add merged
last partition factor for
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Description:
`Rebalance` provide a functionality that split the large reduce partition into
smalls.
XiDuo You created SPARK-37267:
-
Summary: OptimizeSkewInRebalancePartitions support optimize
non-root node
Key: SPARK-37267
URL: https://issues.apache.org/jira/browse/SPARK-37267
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37287:
--
Description:
FileFormatWriter.write now is used by all V1 write which includes datasource
and hive
[
https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37287:
--
Description:
`FileFormatWriter.write` now is used by all V1 write which includes datasource
and hive
[
https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37287:
--
Description:
FileFormatWriter.write now is used by all V1 write which includes datasource
and hive
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Summary: Add small partition factor for rebalance partitions (was: Create
skew partition specs
XiDuo You created SPARK-37333:
-
Summary: Specify the required distribution at V1Write
Key: SPARK-37333
URL: https://issues.apache.org/jira/browse/SPARK-37333
Project: Spark
Issue Type:
XiDuo You created SPARK-37287:
-
Summary: Pull out dynamic partition and bucket sort from
FileFormatWriter
Key: SPARK-37287
URL: https://issues.apache.org/jira/browse/SPARK-37287
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Description:
`Rebalance` provide a functionality that split the large reduce partition into
smalls.
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Description:
For example `Rebalance` provide a functionality that split the large reduce
partition
[
https://issues.apache.org/jira/browse/SPARK-37357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-37357:
--
Summary: Create skew partition specs should respect min partition size
(was: Add merged last
XiDuo You created SPARK-37194:
-
Summary: Avoid unnecessary sort in FileFormatWriter if it's not
dynamic partition
Key: SPARK-37194
URL: https://issues.apache.org/jira/browse/SPARK-37194
Project: Spark
1 - 100 of 407 matches
Mail list logo
