Thank you all for helping with the roadmap documentation. I have merged the roadmap pull request.
Cheers, Jark On Wed, 23 Aug 2023 at 15:26, Jing Ge <j...@ververica.com.invalid> wrote: > Thanks Jark, +1 for the OLAP :-) > > Best regards, > Jing > > On Sun, Aug 20, 2023 at 5:04 PM Jark Wu <imj...@gmail.com> wrote: > > > Hi all, > > > > I have addressed another bunch of comments on the Google doc (mainly > about > > the OLAP roadmap). > > And I have opened a pull request for the website: > > https://github.com/apache/flink-web/pull/672 > > > > Please help to review it and continue the discussion on the pull request, > > thanks a lot! > > > > Best, > > Jark > > > > On Tue, 15 Aug 2023 at 12:15, Xintong Song <tonysong...@gmail.com> > wrote: > > > > > Thanks for driving this, Jark. > > > > > > The current draft looks good to me. I think it is good to open a PR > with > > > it. And if there are other comments, we can discuss them during the PR > > > review. > > > > > > I also added a few minor comments in the draft regarding the feature > > radar. > > > Those can also be discussed on the PR. > > > > > > Best, > > > > > > Xintong > > > > > > > > > > > > On Tue, Aug 15, 2023 at 11:15 AM Shammon FY <zjur...@gmail.com> wrote: > > > > > > > Hi Jark, > > > > > > > > Sounds good and I would love to, thanks! I will involve you and > > Xingtong > > > > on the document after updating. > > > > > > > > Best, > > > > Shammon FY > > > > > > > > > > > > On Mon, Aug 14, 2023 at 10:39 PM Jark Wu <imj...@gmail.com> wrote: > > > > > > > >> Hi Shammon, > > > >> > > > >> Sure, could you help to draft a subsection about this in the google > > doc? > > > >> > > > >> Best, > > > >> Jark > > > >> > > > >> 2023年8月14日 20:30，Shammon FY <zjur...@gmail.com> 写道： > > > >> > > > >> Thanks @Jark for driving the Flink Roadmap. > > > >> > > > >> As we discussed olap in the thread [1] and according to the > > suggestions > > > >> from @Xingtong Song, could we add a subsection in `Towards Streaming > > > >> Warehouses` or `Performance` that the short-lived query in Flink > > Session > > > >> Cluster is one of the future directions for Flink? > > > >> > > > >> Best, > > > >> Shammon FY > > > >> > > > >> On Mon, Aug 14, 2023 at 8:03 PM Jark Wu <imj...@gmail.com> wrote: > > > >> > > > >>> Thank you everyone for helping polish the roadmap [1]. > > > >>> > > > >>> I think I have addressed all the comments and we have included all > > > >>> ongoing > > > >>> parts of Flink. > > > >>> Please feel free to take a last look. I'm going to prepare the pull > > > >>> request > > > >>> if there are no more concerns. > > > >>> > > > >>> Best, > > > >>> Jark > > > >>> > > > >>> [1]: > > > >>> > > > >>> > > > > > > https://docs.google.com/document/d/12BDiVKEsY-f7HI3suO_IxwzCmR04QcVqLarXgyJAb7c/edit > > > >>> > > > >>> On Sun, 13 Aug 2023 at 13:04, Yuan Mei <yuanmei.w...@gmail.com> > > wrote: > > > >>> > > > >>> > Sorry for taking so long > > > >>> > > > > >>> > I've added a section about Flink Disaggregated State Management > > > >>> Evolution > > > >>> > in the attached doc. > > > >>> > > > > >>> > I found some of the contents might be overlapped with the > > > "large-scale > > > >>> > streaming jobs". So that part might need some changes as well. > > > >>> > > > > >>> > Please let me know what you think. > > > >>> > > > > >>> > Best > > > >>> > Yuan > > > >>> > > > > >>> > On Mon, Jul 24, 2023 at 12:07 PM Yuan Mei < > yuanmei.w...@gmail.com> > > > >>> wrote: > > > >>> > > > > >>> > > Sorry have missed this email and respond a bit late. > > > >>> > > > > > >>> > > I will put a draft for the long-term vision for the state as > well > > > as > > > >>> > > large-scale state support into the roadmap. > > > >>> > > > > > >>> > > Best > > > >>> > > Yuan > > > >>> > > > > > >>> > > On Mon, Jul 17, 2023 at 10:34 AM Jark Wu <imj...@gmail.com> > > wrote: > > > >>> > > > > > >>> > >> Hi Jiabao, > > > >>> > >> > > > >>> > >> Thank you for your suggestions. I have added them to the > "Going > > > >>> Beyond a > > > >>> > >> SQL Stream/Batch Processing Engine" and "Large-Scale State > Jobs" > > > >>> > sections. > > > >>> > >> > > > >>> > >> Best, > > > >>> > >> Jark > > > >>> > >> > > > >>> > >> On Thu, 13 Jul 2023 at 16:06, Jiabao Sun < > > jiabao....@xtransfer.cn > > > >>> > >> .invalid> > > > >>> > >> wrote: > > > >>> > >> > > > >>> > >> > Thanks Jark and Martijn for driving this. > > > >>> > >> > > > > >>> > >> > There are two suggestions about the Table API: > > > >>> > >> > > > > >>> > >> > - Add the JSON type to adapt to the no sql database type. > > > >>> > >> > - Remove changelog normalize operator for upsert stream. > > > >>> > >> > > > > >>> > >> > > > > >>> > >> > Best, > > > >>> > >> > Jiabao > > > >>> > >> > > > > >>> > >> > > > > >>> > >> > > 2023年7月13日 下午3:49，Jark Wu <imj...@gmail.com> 写道： > > > >>> > >> > > > > > >>> > >> > > Hi all, > > > >>> > >> > > > > > >>> > >> > > Sorry for taking so long back here. > > > >>> > >> > > > > > >>> > >> > > Martijn and I have drafted the first version of the > updated > > > >>> roadmap, > > > >>> > >> > > including the updated feature radar reflecting the current > > > >>> state of > > > >>> > >> > > different components. > > > >>> > >> > > > > > >>> > >> > > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://docs.google.com/document/d/12BDiVKEsY-f7HI3suO_IxwzCmR04QcVqLarXgyJAb7c/edit > > > >>> > >> > > > > > >>> > >> > > Feel free to leave comments in the thread or the document. > > > >>> > >> > > We may miss mentioning something important, so your help > in > > > >>> > enriching > > > >>> > >> > > the content is greatly appreciated. > > > >>> > >> > > > > > >>> > >> > > Best, > > > >>> > >> > > Jark & Martijn > > > >>> > >> > > > > > >>> > >> > > > > > >>> > >> > > On Fri, 2 Jun 2023 at 00:50, Jing Ge > > > <j...@ververica.com.invalid > > > >>> > > > > >>> > >> wrote: > > > >>> > >> > > > > > >>> > >> > >> Hi Jark, > > > >>> > >> > >> > > > >>> > >> > >> Fair enough. Let's do it like you suggested. Thanks! > > > >>> > >> > >> > > > >>> > >> > >> Best regards, > > > >>> > >> > >> Jing > > > >>> > >> > >> > > > >>> > >> > >> On Thu, Jun 1, 2023 at 6:00 PM Jark Wu <imj...@gmail.com > > > > > >>> wrote: > > > >>> > >> > >> > > > >>> > >> > >>> Hi Jing, > > > >>> > >> > >>> > > > >>> > >> > >>> This thread is for discussing the roadmap for versions > > 1.18, > > > >>> 2.0, > > > >>> > >> and > > > >>> > >> > >> even > > > >>> > >> > >>> more. > > > >>> > >> > >>> One of the outcomes of this discussion will be an > updated > > > >>> version > > > >>> > of > > > >>> > >> > the > > > >>> > >> > >>> current roadmap. > > > >>> > >> > >>> Let's work together on refining the roadmap in this > > thread. > > > >>> > >> > >>> > > > >>> > >> > >>> Best, > > > >>> > >> > >>> Jark > > > >>> > >> > >>> > > > >>> > >> > >>> On Thu, 1 Jun 2023 at 23:25, Jing Ge > > > >>> <j...@ververica.com.invalid> > > > >>> > >> > wrote: > > > >>> > >> > >>> > > > >>> > >> > >>>> Hi Jark, > > > >>> > >> > >>>> > > > >>> > >> > >>>> Thanks for driving it! For point 2, since we are > > developing > > > >>> 1.18 > > > >>> > >> now, > > > >>> > >> > >>>> does it make sense to update the roadmap this time > while > > we > > > >>> are > > > >>> > >> > >> releasing > > > >>> > >> > >>>> 1.18? This discussion thread will be focusing on the > > Flink > > > >>> 2.0 > > > >>> > >> > roadmap, > > > >>> > >> > >>> as > > > >>> > >> > >>>> you mentioned previously. WDYT? > > > >>> > >> > >>>> > > > >>> > >> > >>>> Best regards, > > > >>> > >> > >>>> Jing > > > >>> > >> > >>>> > > > >>> > >> > >>>> On Thu, Jun 1, 2023 at 3:31 PM Jark Wu < > imj...@gmail.com > > > > > > >>> wrote: > > > >>> > >> > >>>> > > > >>> > >> > >>>>> Hi all, > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> Martijn and I would like to initiate a discussion on > the > > > >>> Flink > > > >>> > >> > >> roadmap, > > > >>> > >> > >>>>> which should cover the project's long-term roadmap and > > the > > > >>> > regular > > > >>> > >> > >>> update > > > >>> > >> > >>>>> mechanism. > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> Xintong has already started a discussion about Flink > 2.0 > > > >>> > planning. > > > >>> > >> > >> One > > > >>> > >> > >>> of > > > >>> > >> > >>>>> the points raised in that discussion is that we should > > > have > > > >>> a > > > >>> > >> > >>> high-level > > > >>> > >> > >>>>> discussion of the roadmap to present where the project > > is > > > >>> > heading > > > >>> > >> > >>> (which > > > >>> > >> > >>>>> doesn't necessarily need to block the Flink 2.0 > > planning). > > > >>> > >> Moreover, > > > >>> > >> > >>> the > > > >>> > >> > >>>>> roadmap on the Flink website [1] hasn't been updated > for > > > >>> half a > > > >>> > >> year, > > > >>> > >> > >>> and > > > >>> > >> > >>>>> the last update was for the feature radar for the 1.15 > > > >>> release. > > > >>> > It > > > >>> > >> > >> has > > > >>> > >> > >>>> been > > > >>> > >> > >>>>> 2 years since the community discussed Flink's overall > > > >>> roadmap. > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> I would like to raise two topics for discussion: > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> 1. The new roadmap. This should be an updated version > of > > > the > > > >>> > >> current > > > >>> > >> > >>>>> roadmap[1]. > > > >>> > >> > >>>>> 2. A mechanism to regularly discuss and update the > > > roadmap. > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> To make the first topic discussion more efficient, > > Martijn > > > >>> and I > > > >>> > >> > >>>> volunteer > > > >>> > >> > >>>>> to summarize the ongoing big things of different > > > components > > > >>> and > > > >>> > >> > >>> present a > > > >>> > >> > >>>>> roadmap draft to the community in the next few weeks. > > This > > > >>> > should > > > >>> > >> be > > > >>> > >> > >> a > > > >>> > >> > >>>> good > > > >>> > >> > >>>>> starting point for a more detailed discussion. > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> Regarding the regular update mechanism, there was a > > > >>> proposal in > > > >>> > a > > > >>> > >> > >>> thread > > > >>> > >> > >>>>> [2] three years ago to make the release manager > > > responsible > > > >>> for > > > >>> > >> > >>> updating > > > >>> > >> > >>>>> the roadmap. However, it appears that this was not > > > >>> documented > > > >>> > as a > > > >>> > >> > >>>> release > > > >>> > >> > >>>>> management task [3], and the roadmap update wasn't > > > >>> performed for > > > >>> > >> > >>> releases > > > >>> > >> > >>>>> 1.16 and 1.17. > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> In my opinion, making release managers responsible for > > > >>> keeping > > > >>> > the > > > >>> > >> > >>>> roadmap > > > >>> > >> > >>>>> up to date is a good idea. Specifically, release > > managers > > > of > > > >>> > >> release > > > >>> > >> > >> X > > > >>> > >> > >>>> can > > > >>> > >> > >>>>> kick off the roadmap update at the beginning of > release > > X, > > > >>> which > > > >>> > >> can > > > >>> > >> > >>> be a > > > >>> > >> > >>>>> joint task with collecting a feature list [4]. > > > Additionally, > > > >>> > >> release > > > >>> > >> > >>>>> managers of release X-1 can help verify and remove the > > > >>> > >> accomplished > > > >>> > >> > >>> items > > > >>> > >> > >>>>> from the roadmap and update the feature radar. > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> What do you think? Do you have other ideas? > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> Best, > > > >>> > >> > >>>>> Jark & Martijn > > > >>> > >> > >>>>> > > > >>> > >> > >>>>> [1]: https://flink.apache.org/roadmap.html > > > >>> > >> > >>>>> [2]: > > > >>> > >> > >> > > > >>> https://lists.apache.org/thread/o0l3cg6yphxwrww0k7215jgtw3yfoybv > > > >>> > >> > >>>>> [3]: > > > >>> > >> > >>>>> > > > >>> > >> > >>>> > > > >>> > >> > >>> > > > >>> > >> > >> > > > >>> > >> > > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Flink+Release+Management > > > >>> > >> > >>>>> [4]: > > > >>> > >> > https://cwiki.apache.org/confluence/display/FLINK/1.18+Release > > > >>> > >> > >>>>> > > > >>> > >> > >>>> > > > >>> > >> > >>> > > > >>> > >> > >> > > > >>> > >> > > > > >>> > >> > > > > >>> > >> > > > >>> > > > > > >>> > > > > >>> > > > >> > > > >> > > > > > >
