GitHub user Yicong-Huang added a comment to the discussion: Supporting Directory-Based Dataset Access in UDFs
on the high level, I think mounting is a more feasible option. related topic: we also would need to be able solve different env management that @SarahAsad23 is exploring. I can see the relationship: python virtual env is also just a directory, and CU just need a path of the python callable, which has all the dependencies available in the directory. The mounting approach may not be that hard: instead of download the files to the CU's local volume, download it to a shared volume instead, and mount it to CU. Then in python, the Document API can be resolved and point to a local path. GitHub link: https://github.com/apache/texera/discussions/4352#discussioncomment-16484338 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
