udaysagar2177 commented on PR #17688: URL: https://github.com/apache/pinot/pull/17688#issuecomment-3925053884
@noob-se7en In some deployments, an upstream system may already be producing Avro or Parquet files and storing them in object storage. In such cases, requiring an additional ingestion-specific export step just for Pinot can introduce extra complexity. With this microbatch framework, the upstream system can simply pass along the file metadata for those already-produced files, and Pinot can index them directly. The goal is to make it easier to integrate Pinot into existing data pipelines without forcing changes to how data is produced or stored upstream. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
