rohitgulia opened a new issue, #8784: URL: https://github.com/apache/incubator-devlake/issues/8784
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened bufio.Scanner uses a 64KB default max token size. Python plugins that manage large numbers of scopes (e.g. 100+ Harness pipelines) return a sync plan JSON that exceeds this limit, causing silent truncation. This results in only a subset of scopes appearing in project_mapping and being synced. Increase the buffer to 5MB in both scanOutputPipe and scanErrorPipe eg of what to change: scanner.Buffer(make([]byte, 5*1024*1024), 5*1024*1024) ### What do you expect to happen increase scanner buffer from 64KB to 5MB for Python plugin IPC bufio.Scanner uses a 64KB default max token size. Python plugins that manage large numbers of scopes (e.g. 100+ Harness pipelines) return a sync plan JSON that exceeds this limit, causing silent truncation. This results in only a subset of scopes appearing in project_mapping and being synced. Increase the buffer to 5MB in both scanOutputPipe and scanErrorPipe eg of what to change: scanner.Buffer(make([]byte, 5*1024*1024), 5*1024*1024) ### How to reproduce try syncing more than 100 pipelines via custom plugin ### Anything else _No response_ ### Version v1.0.3-beta10 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
