GitHub user bobbai00 added a comment to the discussion: Proposal - Supporting user-provided ML models in workflows
At high level, it is indeed related to #4242 . Specifically, the motivation and the solution are shared across two problems. However, I did see a major difference here: - In Postgres-credential case, CU communicates using **Restful API over HTTP protocol** - In ML model mounting case, CU uses **POSIX file I/O over FUSE protocol**, not a network protocol. Therefore, design wise: - Postgres-credential case uses a micro service - ML-model case uses something like DaemonSet. I am not quite familiar with DaemonSet. But I want to propose few questions that the final design needs to answer: 1. **What damage can a malicious user cause over other users' models ?** In postgres-credential case, a malicious user cannot cause any damages to other users' Iceberg tables, because the access is controlled using JWT token. 2. **Is the solution general?** Let's say, a CU is running on an EC2 instance. Postgres-credential solution is general because the a CU on EC2 can still talk to a micro service running somewhere else. However, for DaemonSet case, the the CU has to be **running as a K8s pod and co-located with a daemon set pod**. This seems to indicate that the solution is not general. I hope @tanishqgandhi1908 and @aicam can answer these two questions. I hope there is a design that can limit the damage and be general. GitHub link: https://github.com/apache/texera/discussions/6616#discussioncomment-17930762 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
