[
https://issues.apache.org/jira/browse/HUDI-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lokesh Jain updated HUDI-5858:
------------------------------
Description:
This is a follow up Jira to review comments in
[https://github.com/apache/hudi/pull/7982/files]
The config names for error table can be generalised to accomadate for
datasource and spark sql as well. Currently they are tied to DeltaStreamer.
([https://github.com/apache/hudi/pull/7982/files#r1118341097|https://github.com/apache/hudi/pull/7982/files#r1118341097)])
The API should throw exception so that any error cases can be handled.
([https://github.com/apache/hudi/pull/7982/files#r1118372238|https://github.com/apache/hudi/pull/7982/files#r1118372238)])
The constructor for error table writer can be generalised to handle DS and
spark sql use cases as well.
([https://github.com/apache/hudi/pull/7982/files#r1118345162|https://github.com/apache/hudi/pull/7982/files#r1118345162)])
Add API for starting a commit. Handle concurrent error events or concurrent
writes to the error table.
([https://github.com/apache/hudi/pull/7982/files#r1118350161|https://github.com/apache/hudi/pull/7982/files#r1118350161)])
Handle error records from writeStatusRDD
([https://github.com/apache/hudi/pull/7982/files#r1118380323])
was:
The config names for error table can be generalised to accomadate for
datasource and spark sql as well. Currently they are tied to DeltaStreamer.
> Improve API and configs for error Table
> ---------------------------------------
>
> Key: HUDI-5858
> URL: https://issues.apache.org/jira/browse/HUDI-5858
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Lokesh Jain
> Assignee: Lokesh Jain
> Priority: Major
>
> This is a follow up Jira to review comments in
> [https://github.com/apache/hudi/pull/7982/files]
> The config names for error table can be generalised to accomadate for
> datasource and spark sql as well. Currently they are tied to DeltaStreamer.
> ([https://github.com/apache/hudi/pull/7982/files#r1118341097|https://github.com/apache/hudi/pull/7982/files#r1118341097)])
>
> The API should throw exception so that any error cases can be handled.
> ([https://github.com/apache/hudi/pull/7982/files#r1118372238|https://github.com/apache/hudi/pull/7982/files#r1118372238)])
>
> The constructor for error table writer can be generalised to handle DS and
> spark sql use cases as well.
> ([https://github.com/apache/hudi/pull/7982/files#r1118345162|https://github.com/apache/hudi/pull/7982/files#r1118345162)])
> Add API for starting a commit. Handle concurrent error events or concurrent
> writes to the error table.
> ([https://github.com/apache/hudi/pull/7982/files#r1118350161|https://github.com/apache/hudi/pull/7982/files#r1118350161)])
> Handle error records from writeStatusRDD
> ([https://github.com/apache/hudi/pull/7982/files#r1118380323])
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
