Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-25 Thread Yanfei Lei
Hi, Thanks for Lorenzo'reply and Zakelly's supplement. @Lorenzo > I think you got my question, and I did not realize that is not even allowed > to modify some externally scoped variable in a lambda. I guess the point is > that it is possible, but the user would really need to be willing to do

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-24 Thread Zakelly Lan
Hi Lorenzo, Thanks for your comments! I think you got my question, and I did not realize that is not even allowed > to modify some externally scoped variable in a lambda. > I guess the point is that it is possible, but the user would really need > to be willing to do it and "shoot him/herself in

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-22 Thread lorenzo . affetti
Thank you Yanfei for addressing all the questions! > I'm not sure if I understand your question. In Java, this case(modifying the local local variable) is not allowed[1], but there are ways to get around the limitation of lambda. In this case, the modification to x may be concurrent, which needs

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-21 Thread Yanfei Lei
Thanks for your reading and valuable comments! > 1) About locking VS reference counting: I would like to clear out which > mechanism prevents what: The `KeyAccountingUnit` implements locking behavior on keys and ensures 2 state requests on the same key happen in order. Double-locking the same

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-21 Thread lorenzo . affetti
Thank you everybody for the questions and answers (especially Yanfei Lei), it was very instructive to go over the discussion. I am gonna add some questions on top of what happened and add some thoughts as well below. 1) About locking VS reference counting: I would like to clear out which

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-19 Thread Yanfei Lei
Hi everyone, Thanks for your valuable discussion and feedback! Our discussions have been going on for a while and there have been no new comments for several days. So I would like to start a vote after 72 hours. Please let me know if you have any concerns, thanks! Yanfei Lei 于2024年3月13日周三

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-12 Thread Yanfei Lei
Hi Jing, Thanks for the reply and follow up. > What is the benefit for users to build a chain of mails instead of just one > mail(it is still async)? Just to make sure we're on the same page, I try to paraphrase your question: A `then()` call will be encapsulated as a callback mail. Your

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-12 Thread Jing Ge
Hi Yanfei, Thanks for your clarification! Now I got a much clear picture and I am still trying to understand your thoughts for some of those questions: > How many mails are encapsulated depends on how the user writes the > code. The statements in a `then()` will be wrapped into a mail. >

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-11 Thread Yanfei Lei
Hi Jane, Thanks for the perspective. We plan to adapt SQL operators in Milestone 2, so this part is not fully described. > Would you provide more details on how to adapt to the new API? Two interfaces similar to `setCurrentKey` will be provided: let's say preProcessKey()/postProcessKey()

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-11 Thread Jane Chan
Hi Yanfei, Thanks for driving the discussion! I have a few questions from the SQL perspective (and may have some follow-ups). 1. Currently, there are some mini-batch optimizations[1][2] in SQL that update the state during the prepare snapshot phase, which will explicitly call setCurrentKey in

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-11 Thread Yanfei Lei
Hi Gyula, Thanks for your comments. "Record Context" is a concept similar to the current "Key Context"[1]. which is mutable. > If yes, does this mean that the context itself becomes "immutable" or the > context is switched in the background? "Record Context" is switched in the background, its

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-10 Thread Gyula Fóra
Hey! I may have missed this in the FLIP but did not find it spelled out explicitly. Can the user use the entire record processing context in their async execution? In other words can they access all functionality from the callback? For example can they simply reference the keyed context and

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-10 Thread Yanfei Lei
Hi Jing, Thanks for your thoughtful feedback! > does it mean that there will be three mails for Read, Update, and Output ? With this example, there are two mails. The Read is processed by `mailboxDefaultAction`[1], and the Update and Output are encapsulated as mail. > does it make sense to

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-10 Thread Jing Ge
Hi Yanfei, Thanks for your proposal! The FLIP contains a lot of great new ideas. I'd like to ask some questions to make sure we are on the same page. > For the asynchronous interface, Record A should run with Read, Update and Output, while Record B should stay at the Blocking buffer. 1. With

[DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-07 Thread Yanfei Lei
Hi devs, I'd like to start a discussion on FLIP-425: Asynchronous Execution Model[1], which is a sub-FLIP of FLIP-423: Disaggregated State Storage and Management[2]. FLIP-425 introduces a non-blocking execution model leveraging the asynchronous APIs introduced in FLIP-424[3]. For the whole story