gustavobini commented on issue #7516:
URL: 
https://github.com/apache/incubator-devlake/issues/7516#issuecomment-2132780921

   
https://github.com/apache/incubator-devlake/blob/6dca288d5e33bb55a8b98995df97ead71b76a082/backend/plugins/jenkins/tasks/build_collector.go#L143
 does not collect data incrementally. It could use 
`NewStatefulApiCollectorForFinalizableEntity` but that does not support 
iterators. What about changing `collectMultiBranchJobApiBuilds` to call 
https://github.com/apache/incubator-devlake/blob/6dca288d5e33bb55a8b98995df97ead71b76a082/backend/plugins/jenkins/tasks/build_collector.go#L72
 for each entry in the iterator?
   
   This could help, but 
https://github.com/apache/incubator-devlake/blob/3ae23bfdda0537e3cbe5eed17340e4fb115af1ce/backend/plugins/jenkins/tasks/build_extractor.go#L40
 will still select all data from the raw table, including processed data. I 
don't know if raw tables are expected to have that data forever, or if they 
should be cleaned every now and then.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@devlake.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to