[
https://issues.apache.org/jira/browse/HUDI-7416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817965#comment-17817965
]
Vinish Reddy commented on HUDI-7416:
------------------------------------
No change in public API's, Option has been used to define the new field in the
constructors and previous constructors are backwards compatible.
> Add interface for StreamProfile to be used in StreamSync for reading and
> writing data.
> ---------------------------------------------------------------------------------------
>
> Key: HUDI-7416
> URL: https://issues.apache.org/jira/browse/HUDI-7416
> Project: Apache Hudi
> Issue Type: Improvement
> Components: deltastreamer
> Reporter: Vinish Reddy
> Assignee: Vinish Reddy
> Priority: Major
> Labels: pull-request-available
>
> Introducing a new class known as {{StreamProfile}} which contains details
> about how the next sync round in StreamSync should be consumed and written.
> For eg:
> {{KafkaStreamProfile}} contains number of events to consume in this sync
> round.
> {{S3StreamProfile}} contains the list of files to consume in this sync round
> {{HudiIncrementalStreamProfile}} contains the beginInstant and endInstant
> commit times to consume in this sync round.
> In future we can add the method for choosing the writeOperationType and
> indexType as well, for now {{streamProfile.getSourceSpecificContext() }}will
> be used to consume the data from the source.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
