CamilleTeruel opened a new issue, #4880: URL: https://github.com/apache/incubator-devlake/issues/4880
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement. ### Use case As a python plugin developer, I want to have an easier way of declaring which attribute are used as checkpoint for incremental collection. ### Description Currently, `collect` methods get access to a state dict parameter that they can fill with arbitrary data that they will need in subsequent execution to implement incremental data collection. This is too low-level and complexifies the implementation of those methods. We should introduce a higher-level "bookmark" or "checkpoint" concept that is used to extract and compare specific JSON properties in order to limit collection to new data only. ### Related issues _No response_ ### Are you willing to submit a PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
