[
https://issues.apache.org/jira/browse/HUDI-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-9019:
--------------------------------------
Remaining Estimate: 24h
Original Estimate: 24h
> Support writes using spark dataframe end to end
> ------------------------------------------------
>
> Key: HUDI-9019
> URL: https://issues.apache.org/jira/browse/HUDI-9019
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Critical
> Labels: story
> Fix For: 1.0.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> We wanted to support writes using spark end to end using dataframe w/o
> converting them to avro record.
>
> This opens up lot of opportunities for Hudi
> * This will place Hudi close to direct parquet writes for straight forward
> immutable use-cases. Also for mutable use-cases, it will increase
> * For mutable use-cases, we are anticipating 10 to 20% improvement over rdd
> based write client impl.
> * We can leverage spark optimizations which can kick in only with dataframe.
> * Rapids, vectorized reading etc can speed up writes with Hudi once we move
> to end to end data frame writes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
