Hi All, I understand that Kafka connect is not meant for static/batch processing scenarios.
I am trying to accommodate static/batch load also with Kafka so that i dont want to introduce additional layers in my architecture for doing batch. I created a FTP connector , and it works well with small files. But the moment when i have large files in FTP server, the classic problem of Out of Memory Java Heap is coming. My question is , is there any way in connect framework to distribute the data load across tasks ? Any pointers on this please ? Thank you Sreejith.
