The GitHub Actions job "Required Checks" on 
texera.git/feat/geesefs-helm-packaging has failed.
Run started by GitHub user aicam (triggered by aicam).

Head commit for run:
4d99cf4bdf9920bc0ff816790c17766705b63040 / ali <[email protected]>
feat(engine): mount a LakeFS repo per Python UDF via the JWT S3 proxy

Wires the engine consumer that activates in-pod LakeFS-repository mounting, so a
Python UDF can read a repository's files (a dataset or a model) from a local
path.

- PythonUDFOpDescV2: new "Mount dataset" property (/ownerEmail/datasetName/
  versionName); at compile time it is resolved to a repository:commitHash
  locator and carried on the PhysicalOp.
- FileResolver.resolveDatasetVersion: DB lookup mapping the path to
  (repositoryName, versionHash).
- PhysicalOp / WorkerConfig: carry the locator to the Python worker.
- DatasetMountManager: before the Python worker starts, exchange the pod's
  per-user JWT for a mount session at file-service and run GeeseFS against the
  JWT S3 proxy (no global LakeFS credentials in the pod); idempotent per
  (repository, commit) for the pod lifetime.
- PythonWorkflowWorker: mount before spawning the worker and pass the path in
  its startup config; the Python side (texera_run_python_worker.py,
  executor_manager.py) exposes it to UDF code as MOUNTED_DATASET_PATH.

End-to-end on minikube: a Python UDF loaded a ~2 GB sharded PyTorch model from
the mount via torch.load() with bit-exact output, shards streaming as ranged
reads through the proxy, and no LakeFS credentials in the pod.

Part of #6606. Closes #6606.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_014GjAdd2Q7og15uDrzLLUF8
Signed-off-by: ali <[email protected]>

Report URL: https://github.com/apache/texera/actions/runs/29781744422

With regards,
GitHub Actions via GitBox

Reply via email to