chenwei791129 opened a new issue, #9227: URL: https://github.com/apache/devlake/issues/9227
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened Some metric plugin settings on the Config UI **Project → Settings** page take a regex, e.g. the **Deploy job pattern** of Monorepo Sub-Projects. If the regex is invalid (e.g. `^deploy-(a`, missing a closing parenthesis), clicking **Save** reports success and the setting is stored. The error only surfaces the next time a pipeline plan is generated (**Collect Data** or the scheduled run): - **The whole blueprint fails to start**, not just that metric — data collection from every data source stops as well. - The error is reported on the pipeline / blueprint, far from the Settings page where the mistake was made, so it is hard to connect the two. - With a scheduled blueprint, it can take hours before anyone notices the data stopped updating. The same happens when Monorepo Sub-Projects is enabled with no sub-projects at all. Root cause: - `POST /projects` and `PATCH /projects/:projectName` store each metric's `pluginOption` as-is via `refreshProjectMetrics` (`backend/server/services/project.go`), without any validation. - The actual validation lives in each metric plugin's `MakeMetricPluginPipelinePlanV200` (monorepo rejects an empty `subProjects` list and compiles every `deployJobPattern`, returning `BadInput`), which is only called when a pipeline plan is generated — i.e. when a blueprint is saved or triggered. - The monorepo code comment says *"Validate eagerly so a bad regex is reported when the blueprint is saved"*, but editing metric settings from Project Settings never goes through the blueprint save flow, so this path is never validated. ### What do you expect to happen Invalid metric plugin options should be rejected when the project is saved: - Saving with an invalid regex shows an error, and the previous settings remain unchanged. - Saving with a valid regex (including Go-only syntax such as `(?i)`) succeeds. - Disabled metrics carrying invalid options do not block saving (they are not used for plan generation either). - Creating a project (`POST /projects`) is validated the same way. Proposed fix: before writing to the database, `CreateProject` / `PatchProject` dry-run `MakeMetricPluginPipelinePlanV200(projectName, pluginOption)` for every **enabled** metric, discard the generated plan, and reject the request with HTTP 400 (`BadInput`) as `invalid options for metric plugin <name>`, wrapping the plugin's own error (e.g. `subProjects[0] (serviceA): invalid deployJobPattern` caused by ``error parsing regexp: missing closing ): `^deploy-(a` ``). - Every current metric plugin (dora, linker, issue_trace, monorepo) only parses JSON and builds a plan in `MakeMetricPluginPipelinePlanV200` — no database access or other side effects — so it is safe to reuse as a validator. - Some plugins return `errors.Default` (500) on a JSON unmarshal failure; wrapping with `BadInput` makes it a 400 instead of a server error. - No Config UI change is needed: the Save button already shows the API error message as a toast. - Any metric plugin that validates its options in `MakeMetricPluginPipelinePlanV200` automatically gets save-time validation, with no further server change. Out of scope: - Client-side regex validation: JavaScript `RegExp` differs from Go RE2 (e.g. no `(?i)`) and would reject valid patterns. - The linker `prToIssueRegexp`: it is only compiled at task runtime, not in `MakeMetricPluginPipelinePlanV200`, so this fix does not cover it. Could be a follow-up. - A dedicated plugin validation interface (e.g. `ValidateMetricPluginOption`): cleaner semantically, but requires a `core/plugin` interface change. ### How to reproduce 1. Create a project and enable **Monorepo Sub-Projects** on the Settings tab. 2. Add a sub-project with Deploy job pattern `^deploy-(a` and click **Save** — it reports success. 3. Go to the Blueprint tab and click **Collect Data** — plan generation fails and no pipeline runs, so no data is collected at all. ### Anything else Happens every time an enabled metric has invalid options. ### Version main (4ad2f66df042386888cbcd7a0b585c4a2c13f90a) ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
