narrowizard opened a new issue, #3749: URL: https://github.com/apache/incubator-devlake/issues/3749
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened I have wrote some e2e test with e2ehelper.DataFlowTester, it's a great experience. But it reports error when my snapshot data is with nil column value.  ### What do you expect to happen As a helper, `DataFlowTester` should support nil value properly. I noticed that `ImportCsvIntoTabler` will convert empty string to nil value before write to db. It is not a perfect solution as it confused empty string and nil value. https://github.com/apache/incubator-devlake/blob/fb6c5e2e6131cd8c868183430b57a84e9c89c827/helpers/e2ehelper/data_flow_tester.go#L145-L147 > we always have columns with nil value, and requrement to validate these data. ### How to reproduce Refer to `What happened` section ### Anything else The problem is that csv file cannot distinct empty string and nil value. A solution is that we can use a placeholder string to represent nil value. (maybe a `<nil value>`) When read from csv file, convert the `<nil value>` to nil. And write a `<nil value>` when writing a nil value. ### Version main ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
