[
https://issues.apache.org/jira/browse/HUDI-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17345613#comment-17345613
]
Nishith Agarwal commented on HUDI-1881:
---------------------------------------
Looking into the HoodieMultiTableDeltaStreamer is not designed to run in
continuous mode as can be seen, the execution is done serially and in
continuous mode, it's stuck at the first one.
{code:java}
for (TableExecutionContext context : tableExecutionContexts) {
try {
new HoodieDeltaStreamer(context.getConfig(), jssc,
Option.ofNullable(context.getProperties())).sync();
successTables.add(Helpers.getTableWithDatabase(context));
} catch (Exception e) {
logger.error("error while running MultiTableDeltaStreamer for table: " +
context.getTableName(), e);
failedTables.add(Helpers.getTableWithDatabase(context));
}
}
{code}
> HoodieMultiTableDeltaStreamer does not ingest from all topics when using
> continuous mode
> ----------------------------------------------------------------------------------------
>
> Key: HUDI-1881
> URL: https://issues.apache.org/jira/browse/HUDI-1881
> Project: Apache Hudi
> Issue Type: Bug
> Components: DeltaStreamer
> Reporter: Nishith Agarwal
> Assignee: Nishith Agarwal
> Priority: Major
> Labels: sev:critical
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
