chenboat opened a new issue #3847: Controller downloadSegment() api not working on Low Level Kafka consumer on deep storage. URL: https://github.com/apache/incubator-pinot/issues/3847 When using Kafka low level realtime consumer, a slow server now may download the segments from controller. The controller's segment download api uses FileUploadPathProvider. Its getBaseDataDir() method now assume all segments are stored locally and requires the data dir path scheme being "file" -- thus deep storage scheme can not work. ================================== 2019-02-15 22:38:14 INFO FileUploadPathProvider:55 - Data directory: viewfs://ns-default/app/upinot/llc_staging 2019-02-15 22:38:14 INFO FileUploadPathProvider:57 - Schema temporary directory: viewfs://ns-default/app/upinot/llc_staging/schemasTemp 2019-02-15 22:38:14 INFO FileUploadPathProvider:71 - Local temporary directory: file:/var/upinot/stream-pinot-controller/data 2019-02-15 22:38:14 INFO FileUploadPathProvider:76 - File upload temporary directory: file:/var/upinot/stream-pinot-controller/data/fileUploadTemp 2019-02-15 22:38:14 INFO FileUploadPathProvider:78 - Untarred file temporary directory: file:/var/upinot/stream-pinot-controller/data/fileUploadTemp/untarred 2019-02-15 22:38:14 ERROR WebApplicationExceptionMapper:39 - Server error: java.lang.IllegalArgumentException: URI scheme is not "file" at java.io.File.<init>(File.java:421) at com.linkedin.pinot.controller.api.resources.FileUploadPathProvider.getBaseDataDir(FileUploadPathProvider.java:129) at com.linkedin.pinot.controller.api.resources.PinotSegmentUploadRestletResource.downloadSegment(PinotSegmentUploadRestletResource.java:184) .......... 2019-02-15 22:38:14 INFO ControllerResponseFilter:47 - Handled request from 10.76.133.45 GET http://streampinot-sandbox01-sjc1:5984/segments/llc_test_split_commit/llc_test_split_commit__1__1__20190215T2224Z, content-type null status code 500 Internal Server Error
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
