[
https://issues.apache.org/jira/browse/HUDI-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984771#comment-16984771
]
Pratyaksh Sharma commented on HUDI-288:
---------------------------------------
Hi [~xleesf],
> is it reasonable to _ `<base_path_prefix>/<topic_name>`_ ?
I guess as a user, one should be able to understand the database and table for
which data is kept in the concerned target path just by looking at the path. So
keeping <topic name> in the target path looks a bit skeptical to me because a
topic name might not necessarily include table name. [~vinoth] WDYT?
> would we get rid of oozie as introducing it to hudi might be not very
>reasonable?
Oozie is not a part of the wrapper's code. We are scheduling the wrapper using
oozie externally. Sorry for the confusion.
> And is there any other considerations not supporting continous mode currently?
Since the wrapper runs in a loop, picks up tableConfig objects one by one (or
to say it runs for every table one by one) and creates a new
HoodieDeltaStreamer instance in every iteration, if we submit the job in
continuous mode, then it will keep on running for the first table itself and
will never pick up the next table. This is something which we have not taken
care of right now. (To fix this, may be we can try to use ExecutorService and
run each job with a separate thread, if feasible.) We need to decide if we want
to support continuous mode for this wrapper in the first place.
> Also, the wrapper seem to be able to replace the current DeltaStreamer?
I did not get this question.
> Add support for ingesting multiple kafka streams in a single DeltaStreamer
> deployment
> -------------------------------------------------------------------------------------
>
> Key: HUDI-288
> URL: https://issues.apache.org/jira/browse/HUDI-288
> Project: Apache Hudi (incubating)
> Issue Type: Improvement
> Components: deltastreamer
> Reporter: Vinoth Chandar
> Assignee: leesf
> Priority: Major
>
> https://lists.apache.org/thread.html/3a69934657c48b1c0d85cba223d69cb18e18cd8aaa4817c9fd72cef6@<dev.hudi.apache.org>
> has all the context
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
