marcoadasilvaa opened a new issue, #9243: URL: https://github.com/apache/devlake/issues/9243
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened After updating an **advanced-mode** blueprint through the API (`PATCH /blueprints/:id`), the next pipeline started by the **cron schedule** runs with a sanitized (masked) plan. Every `gitextractor` task fails with: ``` subtask Clone Git Repo ended unexpectedly git cmd [git clone https://git:*****@<gitlab-host>/<group>/<repo>.git /tmp/gitextractorXXXX --bare] failed remote: HTTP Basic: Access denied. ... ``` The token stored in the database is correct: - Triggering the same blueprint manually (`POST /blueprints/:id/trigger`) clones every repo fine. - After a `lake` restart, cron runs work again until the next PATCH. In our case it was 1330/1330 `gitextractor` tasks failing in a cron run, while the same repos cloned fine in a manual run of the same plan a few hours earlier. **Root cause (v1.0.3-beta15, still present on `main`):** 1. `PatchBlueprint` → `saveBlueprint` → `reloadBlueprint(blueprint)` registers the cron job with **the same pointer**: `cronManager.AddJob(blueprint.CronConfig, &BlueprintJob{blueprint})` (`backend/server/services/blueprint.go`). 2. Right after saving, `PatchBlueprint` calls `SanitizeBlueprint(blueprint)` on that same object to build the API response. 3. The `gitextractor` sanitizer (`pluginOptionSanitizers` in `backend/server/services/pipeline.go`) replaces the password in `options["url"]` with `*` **in place**. 4. When the cron fires, `BlueprintJob.Run()` calls `createPipelineByBlueprint(bj.Blueprint, ...)`, which for advanced mode uses `blueprint.Plan` as is. The new pipeline gets the masked URL. `TriggerBlueprint` and `ReloadBlueprints` (at startup) load the blueprint from the DB, which is why manual runs and runs after a restart are fine. Normal-mode blueprints are not affected, because their plan is regenerated from the connections on each run. ### What do you expect to happen Cron-triggered pipelines should use the stored (unsanitized) plan, the same as manual triggers. Sanitizing the API response should not change the object the scheduler holds. ### How to reproduce 1. Create an advanced-mode blueprint with a `gitextractor` task whose `url` contains credentials (`https://git:<token>@host/group/repo.git`), with `enable: true`, `isManual: false` and a cron, e.g. `*/5 * * * *`. 2. Update it through the API, e.g. `PATCH /blueprints/:id` with `{"cronConfig": "*/5 * * * *"}` or with a new `plan`. 3. Wait for the cron to fire. 4. The `gitextractor` task fails with `HTTP Basic: Access denied`, and the pipeline's task options show the masked URL. 5. Trigger the same blueprint with `POST /blueprints/:id/trigger`: it succeeds. ### Anything else Possible fixes, either of: - In `BlueprintJob.Run()`, reload the blueprint from the DB by id before creating the pipeline. - In `PatchBlueprint` (and other handlers that sanitize after `saveBlueprint`), sanitize a deep copy rather than the object passed to `reloadBlueprint`. `SanitizeBlueprint` writes into `blueprint.Plan[i][j]`, and the option maps are shared. Workaround: set the blueprint to manual and trigger it from an external scheduler with `POST /blueprints/:id/trigger`, or restart `lake` after every blueprint update. ### Version v1.0.3-beta15 (helm chart 1.0.3-beta15). Code verified unchanged on `main`. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
