Hi Gyula,
if I understand the discussion correctly, you want to use a PTF without
table arguments to return a table (read from savepoint metadata)? If
this is the case, you don't need a PTF for it. A regular table function
can also do the job. IIRC we support TVF with constant args.
Cheers,
Timo
On 28.03.25 08:10, Gyula Fóra wrote:
Hi Timo!
Thanks for the answers.
Just to give some context here is this thread:
https://lists.apache.org/thread/08jwrocqyk1q82lnfdldhnyb79m496lp
We were considering a PTF like state_metadata("checkpointpath") to create a
table with the available state metadata instead of creating a custom
connector for reading the metadata. Our thinking was this could completely
replace the need for a new connector.
But this would only make sense if state_metadata("checkpointpath") could
work as a proper table, such as we can make batch operations on it as well.
Cheers,
Gyula
On Fri, Mar 28, 2025 at 7:39 AM Timo Walther <twal...@apache.org> wrote:
Hi Gabor,
great that you already try out PTFs. I'm in the process of writing
documentation for it. Including a list of limitations.
Please note that PTF won't be support in batch mode in the first phase.
For stateful PTFs we would need to use a batch state backend and also
other code paths around time need to be adjusted.
Cheers,
Timo
On 28.03.25 03:31, Shengkai Fang wrote:
I think it is by design. You can read the FLIP, it says:
*Time Semantics*:
-
PTFs support event-time semantics only.
-
Processing-time doesn’t go well with batch mode and thus a unified
API
should built on event-time.
The proposed onWatermark timers allow for making processing
nevertheless
and key-independent. An onWatermark should cover most processing
time use
cases.
But I think if the PTF doesn't implement the `onTime` method, it means
the
function doesn't care about the time. In this case, we can just
convert directly in batch mode.
Best,
Shengkai
Gabor Somogyi <gabor.g.somo...@gmail.com> 于2025年3月28日周五 00:25写道：
Hi All,
Seems like the process table function scan operation is not supported in
batch mode.
Steps to repro [1] which gives the following exception:
Caused by: org.apache.flink.table.api.TableException: Unsupported
function
for scan:PROCESS_TABLE
Is this something which is planned?
[1]
https://github.com/gaborgsomogyi/flink/commit/494b297082de718eae16e4e555ed58cefa404676
BR,
G
