GitHub user chenlica added a comment to the discussion: Two efforts to convert a Python program to a Texera workflow
Some of the contributors had an offline discussion about this topic. I want to share my thoughts on this topic: - The two approaches have their own pros and cons. The compiler-based approach has the benefits of guaranteeing the equivalence of the translation, keeping the user's code private, and having a high efficiency. - It's not on the "critical path" of the engine, so we can be more tolerant of bugs. - A significant effort has been put into the design and implementation of this compiler-based approach (about two people one summer). - It does have certain restrictive assumptions. - I understand the overhead of reviewing this feature and relaxing those assumptions. - Having said that, I am OK with the plan (to be expressed by other contributors) that we want to first collect users' feedback on the LLM approach before deciding if this compiler approach is really needed. @carloea2 @mengw15 @zyratlo @sshiv012 @parshimers Please share your thoughts based on our discussion. Thanks. GitHub link: https://github.com/apache/texera/discussions/8521#discussioncomment-18684559 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
