kishoreg commented on issue #4941: Pinot module re-structure URL: https://github.com/apache/incubator-pinot/issues/4941#issuecomment-568024274 @elonazoulay Decoders and record readers are typically associated with format of the input data (avro, parquet, thrift etc). RecordReader work on a file and used in batch mode while decoders work at a row level and used in streaming mode. There is scope for unifying and redesigning these interfaces in the future but its not the right time. Given this, my thinking was to keep everything related to the data format in one module. connectors, on the other hand, depend on datasource type (kafka, eventhub, pubsub etc). connectors should be responsible for reading from any of these sources irrespective of the actual data format.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
