icefury71 opened a new issue #3592: Proposal: Remove strong dependency on deep storage for RT tables URL: https://github.com/apache/incubator-pinot/issues/3592 Currently the Pinot real time tables have a strong dependency on some type of deep storage system (either POSIX compliant FS, HDFS or other types) when ingesting data using Low Level Consumers. In the current design, the Pinot controller makes a synchronous copy of the segment being committed to the deep storage (which can then be used for synchronizing data between replicas or for backing up old data). This is a problem since any availability issues on the deep storage system will directly affect Pinot's real-time ingestion (eg: HDFS going down for a few hours). It would be nice to remove this strong dependency and make Pinot self sufficient for real-time ingestion.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
