Nickcw6 commented on issue #7797: URL: https://github.com/apache/incubator-devlake/issues/7797#issuecomment-2262846077
Hey @klesh, I believe i've fixed the issue in #7820 - however I have a question on your comment here: > If you collect data in 180 days and then reduce the Time Range to 90 days, all collected data stay in the database, it is by design. Is this also true for plugins where only `Full Refresh` is supported (ie this CircleCI plugin as per [here](https://devlake.apache.org/docs/Overview/SupportedDataSources/))? As after fixing this bug I notice that for the circleci tables, if I run a collection from `2024-07-01` (for example), then run another from `2024-07-25` (not in full refresh mode), all the previous data is removed and the earliest data point will be from `2024-07-25` onwards. E.g. initial 2024-07-01 run (779 rows in `_raw_circleci_pipelines`), ordered by `created_at` property in `data` column:  Excerpt from top row: ``` { "id": "82a54e28-cdbb-4654-a0c4-e043b7ddf920", "updated_at": "2024-07-01T05:37:12.933Z", "created_at": "2024-07-01T05:37:12.933Z", .... } ``` After rerunning from 2024-07-25 - only 141 rows in `_raw_circleci_pipelines` - top record - NOT in full refresh mode:  ``` { "id": "965dae3d-09b7-42ad-a582-8029a2071693", "updated_at": "2024-07-25T07:06:50.288Z", "created_at": "2024-07-25T07:06:50.288Z" .... } ``` Is this the expected behaviour? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@devlake.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
