sijie commented on pull request #9448: URL: https://github.com/apache/pulsar/pull/9448#issuecomment-773795534
> let's find together the right way to provide features to the users in the best way for the project. yes. I already pointed out the right direction that we should head down. > We already have KafkaBytesSource and KafkaStringSource, so I am just adding a new flavour of the KafkaSource, in fact the implementation is just about adding a new subclass of KafkaAbstractSource. I am following the current style. It was there doesn't mean it is the right pattern to follow. Kafka connector was not the first connect added it. Most of the MQ connectors don't have the schema information. Hence it is okay to maintain `bytes` or `string` connectors. But if we are talking about schema-aware connectors, let's avoid using this pattern. It is impossible to maintain the list of connectors. I would avoid going down this route. > we can work on this issue as well (and that's on my backlog), I didn't want to introduce too many features. It is not about introducing too many features or not. This approach is hard to maintain. > Using the Java Model with GenericRecord adds that additional cost, but the benefit are: The cost is huge when you have a very large struct. The connector should be designed to efficiently transfer data, instead of burning unnecessary CPUs. Let's avoid it as much as we can. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
