> >> user
> >>>> sql.
> >>>> In Switched-start-Position, this is the same question as `how dynamic
> >>>> switched start timestamp works` from @Martijn above. We offer
> >>>> SupportsGetEndTimestamp interface to extract first source spl
Hello Ran Tao,
Thanks for this FLIP.
I have a comment about the handover of switching context between sources.
You have proposed to define interfaces named around timestamps,
SupportsGetEndTimestamp
and SupportsSwitchedStartTimestamp.
These work well with KafkaSource as the downstream child
```
>>>> create table hybrid_source(
>>>> f0 varchar,
>>>> f1 varchar,
>>>> f2 bigint
>>>> ) with(
>>>> 'connector'='hybrid',
>>>> 'source-identifiers'='historical01,historical02,realtime',
>>>> 'historical
'realtime.connector'='kafka',
> > > 'realtime.topic' = 'test',
> > > 'realtime.properties.bootstrap.servers' = 'localhost:9092',
> > > 'realtime.properties.group.id' = 'testGroup',
> > > 'realtime.scan.startup.mode' = 'earliest-offset',
> > > 'realtime.f
s can be found at [1] & [2].
> > Looking forward to your more concerns and opinions.
> >
> > 1.
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235836225
> > 2.https://github.com/apache/flink/pull/21841
> >
> > Best Regards,
> &g
t; support table & sql api for a long time.
>>
>> I have wrote a discussed FLIP.
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235836225
>>
>> Sorry for my unclear subject of previous email, so here i have copied
>> the respond from the Timo
836225
>
> Sorry for my unclear subject of previous email, so here i have copied the
> respond from the Timo and sent this email. look forward to your comments.
>
> ```
> Hi Ran,
>
> Thanks for proposing a FLIP. Btw according to the process, the subject
> of this email shou
Hi, all. i have updated flip-278[1]. I think all problems or comments has
been addressed.
1.about option prefix, we use identifiers.
2.table api implementation and demo
3.about switched dynamic position (hybrid source use it auto switch from
previous to next source)
More details can be found at
Hi, Martijn. i have updated the flip about table api & switched start
timestamp.
thanks.
Martijn Visser 于2022年12月16日周五 16:59写道：
> Hi Ran,
>
> For completeness, this is a new thread that was already previously started
> at https://lists.apache.org/thread/xptn2ddzj34q9f5vtbfb62lsybmvcwjq. I'm
>
a mistake,
childSources.get(sourceIndex).setStartTimetamp(switchedTimestamp);
Ran Tao 于2022年12月19日周一 16:10写道：
> Hi, John. thanks for your comments.
> About question-2 the "handoff" is using for switching next source
> seamlessly. but it's an option. Not every hybrid source job need to using
>
Hi, John. thanks for your comments.
About question-2 the "handoff" is using for switching next source
seamlessly. but it's an option. Not every hybrid source job need to using
this mode.
The hybrid source sql or table need to implement two ways like DataStream
api below. One for fixed position,
Hello all,
Thanks for the FLIP, Ran!
The HybridSource is a really cool feature, and I was glad to see a proposal to
expose it in the Table and SQL APIs.
My main question is also about the switching control (question 2). It seems
like the existing Kafka connector has all the options you'd want
Hi, Martijn, thanks for your comments.
Using identifier as child source prefix may be a good way instead of index.
i will update the flip to illustrate how we can read from hybrid schema to
generate child schemas for the question1.
question2 is start position for the next kafka source. But
Hi Ran,
For completeness, this is a new thread that was already previously started
at https://lists.apache.org/thread/xptn2ddzj34q9f5vtbfb62lsybmvcwjq. I'm
linking them because I think Timo's comments are relevant to be kept with
this discussion thread.
I agree with Timo's comments from there
Fyi.
This flip using index as child source option prefix because we may use the
same connector as hybrid child sources.
e.g.
create table hybrid_source(
f0 varchar,
f1 varchar,
f2 bigint
) with(
'connector'='hybrid',
'sources'='filesystem,filesystem',
'0.path' = '/tmp/a.csv',
'0.format' =
pied the
respond from the Timo and sent this email. look forward to your comments.
```
Hi Ran,
Thanks for proposing a FLIP. Btw according to the process, the subject
of this email should be `[DISCUSS] FLIP-278: Hybrid Source Connector` so
that people can identify this discussion as a FLIP discus
16 matches
Mail list logo
