Hey all,
Thank you for the feedback! Since it sounds like the runtime warnings wouldn't
be too onerous, then I think it'd be good to emit them for deprecated class
initialisation and function use.
> Actually there exists this kind of mechanism, see the following as an
> example which is learned from the practices of other Python projects.
> However, since Flink 2.0, we have removed most deprecated APIs[1] and
> so it seems missing.
Aha, that would explain it - thank you for the context! Yeah, I think wrapping
the sphinx doc and the `warnings.warn` behavior in a Deprecated decorator would
be useful in this case for when we have to deprecate in the future.
Thanks all again for the feedback. I'll create a ticket for this and work on an
initial implementation.
Kind regards,
Mika
On Thu, 20 Feb 2025, at 2:17 PM, Dian Fu wrote:
> Hi Mika,
>
> Thanks a lot for bringing up this discussion.
>
> > Currently, the Java APIs use annotations like @PublicEvolving and
> > @Experimental to clearly indicate the stability status of API components.
> > This is super helpful for understanding which parts of the API are stable
> > for production use and which might change in future releases.
>
> This is indeed missing in the Python API. +1 for this. It should be a
> good improvement.
>
> > It's possible to add a `deprecated` directive to the class/function
> > docstrings to produce a deprecated warning in the API documentation, but
> > it's not widely used.
>
> Actually there exists this kind of mechanism, see the following as an
> example which is learned from the practices of other Python projects.
> However, since Flink 2.0, we have removed most deprecated APIs[1] and
> so it seems missing.
>
> ```
> def register_table(self, name: str, table: Table):
> """
> .. note:: Deprecated in 1.10. Use :func:`create_temporary_view` instead.
> """
> warnings.warn("Deprecated in 1.10. Use create_temporary_view
> instead.", DeprecationWarning)
> self._j_tenv.registerTable(name, table._j_table)
> ```
>
> Another problem is that sometimes the deprecation is not always
> consistent between the Java and Python API. When we deprecate an API
> in Java, the author may forget to deprecate it in Python sometimes.
>
> For the improvement to make it an annotation, +1 for it, this may be a
> good improvement.
>
> > A warning could be emitted at runtime on function use, for example, but I'm
> > not entirely sure how nice that would be.
>
> I agree with Timo on this. There is no compiling phase in Python and
> so I guess we could not do too much on this.
>
>
> Regards,
> Dian
>
> [1] https://github.com/apache/flink/pull/25398
>
> On Thu, Feb 20, 2025 at 6:30 PM Timo Walther <twal...@apache.org> wrote:
> >
> > Hi Mika,
> >
> > thanks for bringing this up! I was actually quite surprised that we
> > don't use any kind of marking in our interfaces in the Python API yet.
> >
> > Your proposal makes total sense. We should reuse the same annotation
> > categories that we use and agreed on in Java. Since the Python Table API
> > is a thin layer around the Java one, we should use the same stability
> > annotations corresponding to each method.
> >
> > > The only real wrinkle is that the @Deprecated annotation in Java
> > > outputs a warning at compile time if it is used, whereas Python
> > > doesn't have a compile time. A warning could be emitted at runtime on
> > > function use, for example, but I'm not entirely sure how nice that
> > > would be.
> >
> > I think printing a warning at runtime is fine as long as the "runtime"
> > is not the Flink runtime but Flink's pre-flight / pre-cluster phase. So
> > maybe we should avoid deprecations within UDFs. But deprecating
> > functions that construct the Table API program, and are not on the hot
> > (i.e. per record) path, should be fine.
> >
> > +1 to this effort. Personally, I don't think we need a FLIP. We
> > basically make the Python API consistent with previous FLIPs around API
> > stability. Also, we bring consistency between Java and Python.
> >
> > What do others think?
> >
> > Thanks,
> > Timo
> >
> >
> > On 19.02.25 15:31, Mika Naylor wrote:
> > > Hi everyone,
> > >
> > > I'd like to start a discussion about adding API stability annotations to
> > > PyFlink, similar to those currently used in the Java APIs
> > > (@PublicEvolving, @Experimental, etc.).
> > >
> > > Currently, the Java APIs use annotations like @PublicEvolving and
> > > @Experimental to clearly indicate the stability status of API components.
> > > This is super helpful for understanding which parts of the API are stable
> > > for production use and which might change in future releases.
> > >
> > > The Python APIs lack this kind of explicit stability indication. It's
> > > possible to add a `deprecated` directive to the class/function docstrings
> > > to produce a deprecated warning in the API documentation, but it's not
> > > widely used.
> > >
> > > I would like to port the API stability annotations from the Java project
> > > into Python equivalents as decorators for Python classes/functions, and
> > > to align the stability annotations between the Java and Python APIs.
> > > These decorators would enrich the docstrings of the underlying
> > > classes/functions to output consistent sections in the API documentation
> > > describing the stability of the class/function as PublicEvolving,
> > > Experimental, Deprecated, etc. For example:
> > >
> > > Example usage:
> > > ```python
> > >
> > > @PublicEvolving()
> > > class Table(object):
> > > ...
> > >
> > > @Deprecated(since="2.1.0", detail="Use
> > > :func:`Table.get_resolved_schema` instead.")
> > > def get_schema(self) -> TableSchema:
> > > ...
> > > ```
> > >
> > > Would output in the API documentation a notice that the Table class'
> > > interface is public but evolving, and that `get_schema` is deprecated
> > > since 2.1.0 in favour of `get_resolved_schema`.
> > >
> > > The only real wrinkle is that the @Deprecated annotation in Java outputs
> > > a warning at compile time if it is used, whereas Python doesn't have a
> > > compile time. A warning could be emitted at runtime on function use, for
> > > example, but I'm not entirely sure how nice that would be.
> > >
> > > Thanks for reading! I would love any opinions/feedback on whether this
> > > would be useful to developers currently using the Python APIs.
> > >
> > > Thanks,
> > > Mika
> >
>
