Jonathan Vexler created HUDI-9535:
-------------------------------------
Summary: Prevent Broadcast of full Partitioner during
mapPartitionsAsRDD
Key: HUDI-9535
URL: https://issues.apache.org/jira/browse/HUDI-9535
Project: Apache Hudi
Issue Type: Improvement
Components: spark, writer-core
Reporter: Jonathan Vexler
Assignee: Jonathan Vexler
Fix For: 1.1.0
Partitioner is used on the executor which can result in a large amount of data
being serialized to the executors for large tables. Most of the data is unused
so we can pull what we need into a lighter weight class.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
