Yicong-Huang commented on PR #4220: URL: https://github.com/apache/texera/pull/4220#issuecomment-3976226572
> Earlier, I mentioned that the MCS could not include source operators due to certain ECM constraints. However, I have since identified a simple solution to address that issue, so this restriction has now been removed. > > That said, reconfiguring source operators themselves is still not supported. I’ve updated the PR description to explain the reasoning behind this decision. I am actually fine with excluding source operators. On the technical side it might be hard. Our implementation of source is a process function of one single special kick off tuple. This means the entire state of a source operator is associated with this single special tuple, which is hard to be broken and transferred. On the use case side, reconfigure a source operator is less useful: if you need to change the source (e.g., read another csv, fetch from another api/db), it's likely you need to rerun the entire workflow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
