Hi, While in theory, not all data within CallContext or RealmContext is required to execute the task, t he current TaskExecutor implementation does rely on these objects during execution. For instance, consider the code in TaskExecutorImpl <https://github.com/apache/polaris/blob/main/runtime/service/src/main/java/org/apache/polaris/service/task/TaskExecutorImpl.java#L153> .
If we aim to pass only the essential information to the executor, the CallContext appears to be the necessary data needed for the executor code. However, if we believe the full CallContext or RealmContext is not needed, a more a robust solution would be to refactor the implementation to depend only on the specific data it requires—instead of relying on these entire context objects. The approach—passing selected data as parameters and then reconstructing the context—does not seem scalable, especially if CallContext evolves or if we need to include additional information in the future. If we say we want to pass the necessary information to the executor, then the CallContext seems the necessary information needed to be passed. If we think we do not need the whole CallContext or RealmContex, I think a more A robust way is to refactor the implementation code to not rely on CallContex or RealmContext, just the information it needed. The current approach passes information as parameter, and then intended to reconstruct the object doesn't sounds very scalable if future refactoring happens to CallContext, or if we intended to add more information to the object. A cleaner and more maintainable solution would be to leverage CDI to properly propagate CallContext and RealmContext t o the background thread. This would avoid manual reconstruction and promote a more consistent execution environment. Best Regards, Yun On Wed, Aug 13, 2025 at 1:01 PM Dmitri Bourlatchkov <di...@apache.org> wrote: > Hi Yufei, > > I would oppose serializing CallContext (especially across server nodes). > > Execution context is established based on a request. Executing a Task is > one of possible requests. As such it has well defined parameters and > begin/end boundaries. Any context that a task needs should naturally flow > from its parameters (which define the job to perform). > > Ultimately, I believe Polaris should leverage CDI for communicating context > information within a particular execution environment. > > The CallContext class is merely a runtime object that serves to represent > some set of parameters in some cases at this time. I do not think it is > defined well enough to act as a medium for transferring execution context > information across nodes. > > How a task's parameters are communicated to other nodes is probably beyond > the scope of this email thread, but I'd expect that aspect to be define in > the tasks proposal(s). My point is that it is a matter specific to tasks > and does not directly relate to CallContext. > > Cheers, > Dmitri. > > On Wed, Aug 13, 2025 at 2:46 PM Yufei Gu <flyrain...@gmail.com> wrote: > > > If we want tasks to run on any node, we can’t avoid serializing the call > > context, or part of it like the realm context. The refactor, as written, > > could NOT support tasks on any node as well. As long as we add a field > > other than the realm ID to the realm context, there is no way to > construct > > a realm context from another node just by realm id. > > > > Yufei > > > > > > On Wed, Aug 13, 2025 at 10:20 AM Eric Maynard <eric.w.mayn...@gmail.com> > > wrote: > > > > > I agree that we shouldn't need the entire CallContext in each & every > > task > > > -- tasks should include whatever information is necessary to execute > > them, > > > and in the end CallContext.copy() shouldn't be needed. > > > > > > However, at least in Will's proposal, we would keep the existing > > framework > > > around (with its unfortunate CallContext.copy()) in parallel with the > new > > > framework for some time. I don't see a big impetus to refactor this > just > > > yet. > > > > > > --EM > > > > > > On Wed, Aug 13, 2025 at 12:18 AM Robert Stupp <sn...@snazy.de> wrote: > > > > > > > Thanks for the reply Yufei, but the intent of all tasks-proposals is > > > > to be able to execute tasks on _any_ node. Are you suggesting making > > > > (Polaris)CallContext serializable? > > > > > > > > On Wed, Aug 13, 2025 at 3:12 AM Yufei Gu <flyrain...@gmail.com> > wrote: > > > > > > > > > > > To still let TaskExecutorImpl making "safe clones", a > functionality > > > to > > > > > get (fresh) instances of RealmContext is required. To enable this, > > the > > > > > RealmContextResolver has been enhanced with "RealmContext lookups" > by > > > > > realm-ID. That in turn led to splitting the > > HTTP/REST-to-realm-context > > > > > resolution into two parts: HTTP/REST-to-realm-ID and > > > realm-ID-to-context. > > > > > > > > > > I'm not sure whether this refactor is beneficial. Avoiding the > > copying > > > > of a > > > > > CDI bean would require introducing a global RealmContext map to > > > maintain > > > > > the mapping between realmId and RealmContext instances. That feels > > > > heavier > > > > > and more complex than simply copying the context whenever needed. > > > > > > > > > > Yufei > > > > > > > > > > > > > > > On Tue, Aug 12, 2025 at 6:18 AM Robert Stupp <sn...@snazy.de> > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > quick heads up that there's a PR to remove CallContext.copy(), > > which > > > > > > is only used from tasks. This change is also part of the effort > to > > > > > > have async & reliable tasks running "anywhere". > > > > > > > > > > > > Robert > > > > > > > > > > > > [1] https://github.com/apache/polaris/pull/2294 > > > > > > > > > > > > > > > >
