Sentiaus opened a new pull request, #6447:
URL: https://github.com/apache/texera/pull/6447
### What changes were proposed in this PR?
This PR adds server-side Google Drive export endpoints for datasets and
workflows.
**New endpoints:**
- `POST /dataset/{did}/drive-export` — Exports a dataset version (specified
by `dvid` or `latest=true`) as a zip archive directly to a Google Drive
resumable upload session URI.
- `POST /dataset/drive-export/file` — Exports a single file from a dataset
version to Drive.
- `POST /workflow/{wid}/export/drive` — Exports a workflow's JSON content to
Drive.
**Security:**
- All three endpoints validate that the session URI starts with
`https://www.googleapis.com/upload/drive/` before making any outbound
connection, preventing SSRF.
- Dataset endpoints check read access and the dataset's `isDownloadable`
flag before streaming data.
- Workflow endpoint checks read access via `WorkflowAccessResource`.
**Size limits:**
- Dataset version export: capped at 500 MB
(`DRIVE_EXPORT_MAX_DATASET_BYTES`). The zip is built in memory before the PUT
to Drive; this limit will be lifted once chunked/streaming resumable upload is
implemented.
- Single file export: 5 TB (`DRIVE_EXPORT_MAX_FILE_BYTES`) — Google Drive's
resumable upload ceiling.
- Workflow export: capped at 500 MB (`DRIVE_EXPORT_MAX_BYTES`) since the
JSON content is read into a byte array before upload.
**Config (`user-system.conf`):**
- Adds `clientSecret` and `apiKey` fields (empty by default, overridable via
`USER_SYS_GOOGLE_CLIENT_SECRET` / `USER_SYS_GOOGLE_API_KEY` env vars) so the
frontend can fetch them from `GoogleAuthResource`.
**`GoogleAuthResource`:**
- Adds `GET /auth/google/drive/apikey` endpoint returning
`UserSystemConfig.googleApiKey` so the frontend can initialize the Google
Picker without embedding credentials in the build.
### Any related issues, documentation, discussions?
Part of the Google Drive export feature. The frontend PR (stacked on this
one) wires up the UI.
### How was this PR tested?
- `DatasetResourceSpec` — new integration tests (using `MockTexeraDB` +
`MockLakeFS` testcontainers) cover:
- Successful version export to Drive (verifying the PUT to the session URI
is made)
- Successful single-file export to Drive
- Rejection of non-googleapis session URIs (`400 Bad Request`)
- `403` when the dataset is not downloadable
- `400` when both `dvid` and `latest` are provided (or neither)
- `WorkflowResource` endpoint tested manually (recompile + restart required
to pick up the new route).
- SSRF validation verified: passing a
non-`https://www.googleapis.com/upload/drive/` URI returns `400`.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 4.6
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
