Re: [DISCUSS] Deprecation and removal of the legacy SQL planner
Thanks for the feedback everyone. You can track the progress of these efforts in: https://issues.apache.org/jira/browse/FLINK-21709 and https://issues.apache.org/jira/browse/FLINK-14437 Regards, Timo On 04.03.21 16:08, Jark Wu wrote: big +1 from my side. Best, Jark On Thu, 4 Mar 2021 at 20:59, Leonard Xu wrote: +1 for the roadmap. Thanks Timo for driving this. Best, Leonard 在 2021年3月4日，20:40，Timo Walther 写道： Last call for feedback on this topic. It seems everyone agrees to finally complete FLIP-32. Since FLIP-32 has been accepted for a very long time, I think we don't need another voting thread for executing the last implementation step. Please let me know if you think differently. I will start deprecating the affected classes and interfaces beginning of next week. Regards, Timo On 26.02.21 15:46, Seth Wiesman wrote: Strong +1 Having two planners is confusing to users and the diverging semantics make it difficult to provide useful learning material. It is time to rip the bandage off. Seth On Fri, Feb 26, 2021 at 12:54 AM Kurt Young wrote: breaking change.> Hi Timo, First of all I want to thank you for introducing this planner design back in 1.9, this is a great work that allows lots of blink features to be merged to Flink in a reasonably short time. It greatly accelerates the evolution speed of Table & SQL. Everything comes with a cost, as you said, right now we are facing the overhead of maintaining two planners and it causes bugs and also increases imbalance between these two planners. As a developer and also for the good of all Table & SQL users, I also think it's better for us to be more focused on a single planner. Your proposed roadmap looks good to me, +1 from my side and thanks again for all your efforts! Best, Kurt On Thu, Feb 25, 2021 at 5:01 PM Timo Walther wrote: Hi everyone, since Flink 1.9 we have supported two SQL planners. Most of the original plan of FLIP-32 [1] has been implemented. The Blink code merge has been completed and many additional features have been added exclusively to the new planner. The new planner is now in a much better shape than the legacy one. In order to avoid user confusion, reduce duplicate code, and improve maintainability and testing times of the Flink project as a whole we would like to propose the following steps to complete FLIP-32: In Flink 1.13: - Deprecate the `flink-table-planner` module - Deprecate `BatchTableEnvironment` for both Java, Scala, and Python In Flink 1.14: - Drop `flink-table-planner` early - Drop many deprecated interfaces and API on demand - Rename `flink-table-planner-blink` to `flink-table-planner` - Rename `flink-table-runtime-blink` to `flink-table-runtime` - Remove references of "Blink" in the code base This will have an impact on users that still use DataSet API together with Table API. With this change we will not support converting between DataSet API and Table API anymore. We hope to compensate the missing functionality in the new unified TableEnvironment and/or the batch mode in DataStream API during 1.14 and 1.15. For this, we are looking for further feedback which features are required in Table API/DataStream API to have a smooth migration path. Looking forward to your feedback. Regards, Timo [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions
Re: [DISCUSS] Deprecation and removal of the legacy SQL planner
big +1 from my side. Best, Jark On Thu, 4 Mar 2021 at 20:59, Leonard Xu wrote: > +1 for the roadmap. > > Thanks Timo for driving this. > > Best, > Leonard > > > 在 2021年3月4日，20:40，Timo Walther 写道： > > > > Last call for feedback on this topic. > > > > It seems everyone agrees to finally complete FLIP-32. Since FLIP-32 has > been accepted for a very long time, I think we don't need another voting > thread for executing the last implementation step. Please let me know if > you think differently. > > > > I will start deprecating the affected classes and interfaces beginning > of next week. > > > > Regards, > > Timo > > > > > > On 26.02.21 15:46, Seth Wiesman wrote: > >> Strong +1 > >> Having two planners is confusing to users and the diverging semantics > make > >> it difficult to provide useful learning material. It is time to rip the > >> bandage off. > >> Seth > >> On Fri, Feb 26, 2021 at 12:54 AM Kurt Young wrote: > >>> breaking > >>> change.> > >>> > >>> Hi Timo, > >>> > >>> First of all I want to thank you for introducing this planner design > back > >>> in 1.9, this is a great work > >>> that allows lots of blink features to be merged to Flink in a > reasonably > >>> short time. It greatly > >>> accelerates the evolution speed of Table & SQL. > >>> > >>> Everything comes with a cost, as you said, right now we are facing the > >>> overhead of maintaining > >>> two planners and it causes bugs and also increases imbalance between > these > >>> two planners. As > >>> a developer and also for the good of all Table & SQL users, I also > think > >>> it's better for us to be more > >>> focused on a single planner. > >>> > >>> Your proposed roadmap looks good to me, +1 from my side and thanks > >>> again for all your efforts! > >>> > >>> Best, > >>> Kurt > >>> > >>> > >>> On Thu, Feb 25, 2021 at 5:01 PM Timo Walther > wrote: > >>> > Hi everyone, > > since Flink 1.9 we have supported two SQL planners. Most of the > original > plan of FLIP-32 [1] has been implemented. The Blink code merge has > been > completed and many additional features have been added exclusively to > the new planner. The new planner is now in a much better shape than > the > legacy one. > > In order to avoid user confusion, reduce duplicate code, and improve > maintainability and testing times of the Flink project as a whole we > would like to propose the following steps to complete FLIP-32: > > In Flink 1.13: > - Deprecate the `flink-table-planner` module > - Deprecate `BatchTableEnvironment` for both Java, Scala, and Python > > In Flink 1.14: > - Drop `flink-table-planner` early > - Drop many deprecated interfaces and API on demand > - Rename `flink-table-planner-blink` to `flink-table-planner` > - Rename `flink-table-runtime-blink` to `flink-table-runtime` > - Remove references of "Blink" in the code base > > This will have an impact on users that still use DataSet API together > with Table API. With this change we will not support converting > between > DataSet API and Table API anymore. We hope to compensate the missing > functionality in the new unified TableEnvironment and/or the batch > mode > in DataStream API during 1.14 and 1.15. For this, we are looking for > further feedback which features are required in Table API/DataStream > API > to have a smooth migration path. > > Looking forward to your feedback. > > Regards, > Timo > > [1] > > > >>> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions > > >>> > > > >
Re: [DISCUSS] Deprecation and removal of the legacy SQL planner
+1 for the roadmap. Thanks Timo for driving this. Best, Leonard > 在 2021年3月4日，20:40，Timo Walther 写道： > > Last call for feedback on this topic. > > It seems everyone agrees to finally complete FLIP-32. Since FLIP-32 has been > accepted for a very long time, I think we don't need another voting thread > for executing the last implementation step. Please let me know if you think > differently. > > I will start deprecating the affected classes and interfaces beginning of > next week. > > Regards, > Timo > > > On 26.02.21 15:46, Seth Wiesman wrote: >> Strong +1 >> Having two planners is confusing to users and the diverging semantics make >> it difficult to provide useful learning material. It is time to rip the >> bandage off. >> Seth >> On Fri, Feb 26, 2021 at 12:54 AM Kurt Young wrote: >>> >> change.> >>> >>> Hi Timo, >>> >>> First of all I want to thank you for introducing this planner design back >>> in 1.9, this is a great work >>> that allows lots of blink features to be merged to Flink in a reasonably >>> short time. It greatly >>> accelerates the evolution speed of Table & SQL. >>> >>> Everything comes with a cost, as you said, right now we are facing the >>> overhead of maintaining >>> two planners and it causes bugs and also increases imbalance between these >>> two planners. As >>> a developer and also for the good of all Table & SQL users, I also think >>> it's better for us to be more >>> focused on a single planner. >>> >>> Your proposed roadmap looks good to me, +1 from my side and thanks >>> again for all your efforts! >>> >>> Best, >>> Kurt >>> >>> >>> On Thu, Feb 25, 2021 at 5:01 PM Timo Walther wrote: >>> Hi everyone, since Flink 1.9 we have supported two SQL planners. Most of the original plan of FLIP-32 [1] has been implemented. The Blink code merge has been completed and many additional features have been added exclusively to the new planner. The new planner is now in a much better shape than the legacy one. In order to avoid user confusion, reduce duplicate code, and improve maintainability and testing times of the Flink project as a whole we would like to propose the following steps to complete FLIP-32: In Flink 1.13: - Deprecate the `flink-table-planner` module - Deprecate `BatchTableEnvironment` for both Java, Scala, and Python In Flink 1.14: - Drop `flink-table-planner` early - Drop many deprecated interfaces and API on demand - Rename `flink-table-planner-blink` to `flink-table-planner` - Rename `flink-table-runtime-blink` to `flink-table-runtime` - Remove references of "Blink" in the code base This will have an impact on users that still use DataSet API together with Table API. With this change we will not support converting between DataSet API and Table API anymore. We hope to compensate the missing functionality in the new unified TableEnvironment and/or the batch mode in DataStream API during 1.14 and 1.15. For this, we are looking for further feedback which features are required in Table API/DataStream API to have a smooth migration path. Looking forward to your feedback. Regards, Timo [1] >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions >>> >
Re: [DISCUSS] Deprecation and removal of the legacy SQL planner
Last call for feedback on this topic. It seems everyone agrees to finally complete FLIP-32. Since FLIP-32 has been accepted for a very long time, I think we don't need another voting thread for executing the last implementation step. Please let me know if you think differently. I will start deprecating the affected classes and interfaces beginning of next week. Regards, Timo On 26.02.21 15:46, Seth Wiesman wrote: Strong +1 Having two planners is confusing to users and the diverging semantics make it difficult to provide useful learning material. It is time to rip the bandage off. Seth On Fri, Feb 26, 2021 at 12:54 AM Kurt Young wrote: Hi Timo, First of all I want to thank you for introducing this planner design back in 1.9, this is a great work that allows lots of blink features to be merged to Flink in a reasonably short time. It greatly accelerates the evolution speed of Table & SQL. Everything comes with a cost, as you said, right now we are facing the overhead of maintaining two planners and it causes bugs and also increases imbalance between these two planners. As a developer and also for the good of all Table & SQL users, I also think it's better for us to be more focused on a single planner. Your proposed roadmap looks good to me, +1 from my side and thanks again for all your efforts! Best, Kurt On Thu, Feb 25, 2021 at 5:01 PM Timo Walther wrote: Hi everyone, since Flink 1.9 we have supported two SQL planners. Most of the original plan of FLIP-32 [1] has been implemented. The Blink code merge has been completed and many additional features have been added exclusively to the new planner. The new planner is now in a much better shape than the legacy one. In order to avoid user confusion, reduce duplicate code, and improve maintainability and testing times of the Flink project as a whole we would like to propose the following steps to complete FLIP-32: In Flink 1.13: - Deprecate the `flink-table-planner` module - Deprecate `BatchTableEnvironment` for both Java, Scala, and Python In Flink 1.14: - Drop `flink-table-planner` early - Drop many deprecated interfaces and API on demand - Rename `flink-table-planner-blink` to `flink-table-planner` - Rename `flink-table-runtime-blink` to `flink-table-runtime` - Remove references of "Blink" in the code base This will have an impact on users that still use DataSet API together with Table API. With this change we will not support converting between DataSet API and Table API anymore. We hope to compensate the missing functionality in the new unified TableEnvironment and/or the batch mode in DataStream API during 1.14 and 1.15. For this, we are looking for further feedback which features are required in Table API/DataStream API to have a smooth migration path. Looking forward to your feedback. Regards, Timo [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions
Re: [DISCUSS] Deprecation and removal of the legacy SQL planner
Strong +1 Having two planners is confusing to users and the diverging semantics make it difficult to provide useful learning material. It is time to rip the bandage off. Seth On Fri, Feb 26, 2021 at 12:54 AM Kurt Young wrote: > change.> > > Hi Timo, > > First of all I want to thank you for introducing this planner design back > in 1.9, this is a great work > that allows lots of blink features to be merged to Flink in a reasonably > short time. It greatly > accelerates the evolution speed of Table & SQL. > > Everything comes with a cost, as you said, right now we are facing the > overhead of maintaining > two planners and it causes bugs and also increases imbalance between these > two planners. As > a developer and also for the good of all Table & SQL users, I also think > it's better for us to be more > focused on a single planner. > > Your proposed roadmap looks good to me, +1 from my side and thanks > again for all your efforts! > > Best, > Kurt > > > On Thu, Feb 25, 2021 at 5:01 PM Timo Walther wrote: > > > Hi everyone, > > > > since Flink 1.9 we have supported two SQL planners. Most of the original > > plan of FLIP-32 [1] has been implemented. The Blink code merge has been > > completed and many additional features have been added exclusively to > > the new planner. The new planner is now in a much better shape than the > > legacy one. > > > > In order to avoid user confusion, reduce duplicate code, and improve > > maintainability and testing times of the Flink project as a whole we > > would like to propose the following steps to complete FLIP-32: > > > > In Flink 1.13: > > - Deprecate the `flink-table-planner` module > > - Deprecate `BatchTableEnvironment` for both Java, Scala, and Python > > > > In Flink 1.14: > > - Drop `flink-table-planner` early > > - Drop many deprecated interfaces and API on demand > > - Rename `flink-table-planner-blink` to `flink-table-planner` > > - Rename `flink-table-runtime-blink` to `flink-table-runtime` > > - Remove references of "Blink" in the code base > > > > This will have an impact on users that still use DataSet API together > > with Table API. With this change we will not support converting between > > DataSet API and Table API anymore. We hope to compensate the missing > > functionality in the new unified TableEnvironment and/or the batch mode > > in DataStream API during 1.14 and 1.15. For this, we are looking for > > further feedback which features are required in Table API/DataStream API > > to have a smooth migration path. > > > > Looking forward to your feedback. > > > > Regards, > > Timo > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions > > >
Re: [DISCUSS] Deprecation and removal of the legacy SQL planner
Hi Timo, First of all I want to thank you for introducing this planner design back in 1.9, this is a great work that allows lots of blink features to be merged to Flink in a reasonably short time. It greatly accelerates the evolution speed of Table & SQL. Everything comes with a cost, as you said, right now we are facing the overhead of maintaining two planners and it causes bugs and also increases imbalance between these two planners. As a developer and also for the good of all Table & SQL users, I also think it's better for us to be more focused on a single planner. Your proposed roadmap looks good to me, +1 from my side and thanks again for all your efforts! Best, Kurt On Thu, Feb 25, 2021 at 5:01 PM Timo Walther wrote: > Hi everyone, > > since Flink 1.9 we have supported two SQL planners. Most of the original > plan of FLIP-32 [1] has been implemented. The Blink code merge has been > completed and many additional features have been added exclusively to > the new planner. The new planner is now in a much better shape than the > legacy one. > > In order to avoid user confusion, reduce duplicate code, and improve > maintainability and testing times of the Flink project as a whole we > would like to propose the following steps to complete FLIP-32: > > In Flink 1.13: > - Deprecate the `flink-table-planner` module > - Deprecate `BatchTableEnvironment` for both Java, Scala, and Python > > In Flink 1.14: > - Drop `flink-table-planner` early > - Drop many deprecated interfaces and API on demand > - Rename `flink-table-planner-blink` to `flink-table-planner` > - Rename `flink-table-runtime-blink` to `flink-table-runtime` > - Remove references of "Blink" in the code base > > This will have an impact on users that still use DataSet API together > with Table API. With this change we will not support converting between > DataSet API and Table API anymore. We hope to compensate the missing > functionality in the new unified TableEnvironment and/or the batch mode > in DataStream API during 1.14 and 1.15. For this, we are looking for > further feedback which features are required in Table API/DataStream API > to have a smooth migration path. > > Looking forward to your feedback. > > Regards, > Timo > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions >
[DISCUSS] Deprecation and removal of the legacy SQL planner
Hi everyone, since Flink 1.9 we have supported two SQL planners. Most of the original plan of FLIP-32 [1] has been implemented. The Blink code merge has been completed and many additional features have been added exclusively to the new planner. The new planner is now in a much better shape than the legacy one. In order to avoid user confusion, reduce duplicate code, and improve maintainability and testing times of the Flink project as a whole we would like to propose the following steps to complete FLIP-32: In Flink 1.13: - Deprecate the `flink-table-planner` module - Deprecate `BatchTableEnvironment` for both Java, Scala, and Python In Flink 1.14: - Drop `flink-table-planner` early - Drop many deprecated interfaces and API on demand - Rename `flink-table-planner-blink` to `flink-table-planner` - Rename `flink-table-runtime-blink` to `flink-table-runtime` - Remove references of "Blink" in the code base This will have an impact on users that still use DataSet API together with Table API. With this change we will not support converting between DataSet API and Table API anymore. We hope to compensate the missing functionality in the new unified TableEnvironment and/or the batch mode in DataStream API during 1.14 and 1.15. For this, we are looking for further feedback which features are required in Table API/DataStream API to have a smooth migration path. Looking forward to your feedback. Regards, Timo [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions
