Thanks Fabian,
I was looking forward to use the unified Source interface in my use case.
The implementation was very intuitive with this new design.
I will try with TableFunction then.
Best.
Krzysztof Chmielewski
pt., 5 lis 2021 o 14:20 Fabian Paul napisał(a):
> Hi Krzysztof,
>
> The blog post
Hi Krzysztof,
The blog post is not building a lookup source but only a scan source. For scan
sources you can choose between the old RichSourceFunction
or the new unified Source interface.
For lookup sources you need to implement either a TableFunction or a
AsyncTableFunction there are current
Ok,
I think there is some misunderstanding here.
As it is presented in [1] for implementing Custom Source Connector for
Table API and SQL:
*"You first need to have a source connector which can be used in Flink’s
runtime system (...)*
*For complex connectors, you may want to implement the Source
I think neither Source nor RichSourceFunction are correct in this case. You can
have a look at the Jdbc lookup source[1][2]. Your function needs to
implement TableFunction.
Best,
Fabian
[1]
https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-jdbc/src/main/java/org/apac
Thank you Fabian,
what if I would rewrite my custom Source to use old RichSourchFunction
instead unified Source Interface?
Would it work then as Lookup?
Best,
Krzysztof
pt., 5 lis 2021 o 11:18 Fabian Paul napisał(a):
> Hi Krzysztof,
>
> The unified Source is meant to be used for the DataStream
Hi Krzysztof,
The unified Source is meant to be used for the DataStream API and Table API.
Currently, we do not have definition of look up sources in the
DataStream API therefore the new source do not work as lookups and only as scan
sources.
Maybe in the future we also want to define look ups
BTW,
@Ingo Burk
You wrote that "*the new, unified Source interface can only work as a scan
source.*"
Is there any special design reason behind it or its just simply not yet
implemented?
Thanks,
Krzysztof Chmielewski
czw., 4 lis 2021 o 16:27 Krzysztof Chmielewski <
krzysiek.chmielew...@gmail.co
Sure,
I have a connector that will uses HTTP rest call to 3rd party system to get
some data based on URL and parameters.
Idea is to make it available to Flink SQL in order to use it like
SELECT * FROM T where t.id = 123
I would like to have two streams, one would be from T, and the second one
wou
Hi Krzysztof,
the new, unified Source interface can only work as a scan source. Could you
maybe elaborate a bit on the connector implementation you have and how you
intend to have it work as a lookup source?
Best
Ingo
On Thu, Nov 4, 2021 at 4:11 PM Krzysztof Chmielewski <
krzysiek.chmielew...@g
Thanks Fabian and Ingo,
yes I forgot to add the refrence links, so here they are:
[1] https://flink.apache.org/2021/09/07/connector-table-sql-api-part1.html
[2] https://flink.apache.org/2021/09/07/connector-table-sql-api-part2
[3]
https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/dat
Hi Krzystof,
instead of SourceFunctionProvider you need to use SourceProvider. If you
look at the filesystem connector you can see an example for that, too
(FileSystemTableSource#createSourceProvider).
Best
Ingo
On Thu, Nov 4, 2021 at 3:48 PM Krzysztof Chmielewski <
krzysiek.chmielew...@gmail.c
Hi Krzysztof,
It is great to hear that you have implemented your source with the unified
Source interface. To integrate you source in the Table API you can use the
SourceProvider. You can take a look at how our FileSource does is[1]
Btw I think you forgot to add your references ;)
Best,
Fabi
Hi,
I was wondering if it is possible to implement a Source Table connector
like it is described in [1][2] with custom source that implements a new
Source interface [3] and not a SourceFunction.
I already have my custom source but when I'm trying to implement a Table
Source from LookupTableSource
13 matches
Mail list logo
