mangrrua opened a new issue #5344: URL: https://github.com/apache/incubator-pinot/issues/5344
Apache Pinot have a great abstraction layer to convert any file format to pinot segment file. But it is not useful when we want to convert in-memory data(eg: InputStream, lists etc.) to the pinot segment. `RecordReader` waits input file path for segment creating, and `SegmentIndexCreationDriverImpl` takes RecordReader implementation. Then it read input file and create Pinot segment file. But implementation is based on input File. This is a bad limitation. I want to create Pinot segment file from in memory data. We can expand the `RecordReader` and `SegmentIndexCreatingDriverImpl` implementations? Or how can we do anything for that? I want that, because I do not want to write any data to the temp files while converting it to pinot segment file. This is a costly operation. I want to convert data to pinot segment from in-memory data directly. Thanks. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
