vbhanuchander-lang commented on PR #9038: URL: https://github.com/apache/devlake/pull/9038#issuecomment-5234867625
Pushed `18ec61c` adding the e2e dataflow test I said was missing. I ran a MySQL 8.4 container locally to set `E2E_DB_URL`, so this is verified rather than written blind. `TestIssueFieldValueDataFlow` covers both halves. **Extraction** — asserts the value normalisation per data type against the documented response shapes: ``` connection_id,issue_id,field_id,field_name,data_type,value,raw_value,option_color 1,346842831,123,Priority,single_select,Critical,"""Critical""",ff0000 1,346842831,456,Effort,number,5,5, 1,346842831,789,Target date,date,2024-12-31,"""2024-12-31""", 1,347255859,456,Effort,number,2.5,2.5, 1,347255859,901,Squad,multi_select,"backend,urgent","[""backend"",""urgent""]", 1,348630179,123,Priority,single_select,,null, ``` Note `5` rather than `5.0`, `2.5` keeping its precision, the multi_select joined while `raw_value` keeps the JSON array, and a null value producing an empty `value`. **Conversion** — asserts the mapping reaches the domain issue, including the failure paths: ``` id,issue_key,priority,component,story_point,due_date github:GithubIssue:1:346842831,5,Critical,,5,2024-12-31T00:00:00.000+00:00 github:GithubIssue:1:347255859,6,Low,"backend,urgent",2.5, github:GithubIssue:1:348630179,7,,,, ``` Issue #7 is the interesting row: it carries `"soon"` in a date field and a null priority, and comes out with neither set — skipped with a warning rather than failing the task or writing a wrong value. The log confirms the path fires: ``` level=warning msg="issue #7: field \"Target date\" value \"soon\" is not a date, due date left unset" ``` The other 23 issues in the fixture are untouched, which is what verifies that an unmapped issue is left alone. Whole plugin green with the database attached, including every pre-existing e2e test: ``` ok github.com/apache/incubator-devlake/plugins/github/e2e 3.667s ok github.com/apache/incubator-devlake/plugins/github/impl ok github.com/apache/incubator-devlake/plugins/github/tasks ok github.com/apache/incubator-devlake/plugins/github/models ``` One fixture note in case it helps anyone else: the `params` column has to match the serialised form byte for byte. My first attempt used `json.dumps` defaults, which inserts spaces after separators, and the extractor silently found 0 rows because the `params = ?` clause did not match. Regenerating with compact separators fixed it, and the fixture is LF-terminated to match the existing ones. Still happy to add config-ui inputs for the new scope config keys and the `devlake-website` docs, either here or as follow-ups. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
