anantmf edited a comment on issue #6980: With latest version of druid is it possible to use OpenStack Swift custom implementation (not Amazon S3)? URL: https://github.com/apache/incubator-druid/issues/6980#issuecomment-460006580 Hi @jihoonson , Thanks for the reply. I found couple of configuration issues after which I was able to make it work. So now segments are correctly stored in Swift, however when Historical is trying to get it back from Deep storage it fails. I think the issue is https://github.com/apache/incubator-druid/pull/5382#discussion_r175912273 This is exactly what is happening in my case. Where result.getKeyCount() is 0 but result.getObjectSummaries() have size > 0. When it is doing ListObjectsV2Request, i tried to debug the code and the response actually has object summary but KeyCount is Zero. Which is causing it to fail. I tried to debug the code of unmarshalling of List object result, and it seems the Unmarshaller expects the response to have a element called as KeyCount then only it sets it in the result object, otherwise it stays zero. This is the actual log I get: 2019-02-02T15:04:19,789 ERROR [ZkCoordinator] org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager - Failed to load segment in current location /data/projects/imply-2.8.9/var/druid/segment-cache, try next location if any: {class=org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager, exceptionType=class org.apache.druid.segment.loading.SegmentLoadingException, exceptionMessage=Cannot find object for bucket[test4] and key[druid/segments/Consumer_Complaints/2016-01-01T00:00:00.000Z_2017-01-01T00:00:00.000Z/2019-02-02T15:03:18.693Z/0/index.zip], location=/data/projects/imply-2.8.9/var/druid/segment-cache} org.apache.druid.segment.loading.SegmentLoadingException: Cannot find object for bucket[test4] and key[druid/segments/Consumer_Complaints/2016-01-01T00:00:00.000Z_2017-01-01T00:00:00.000Z/2019-02-02T15:03:18.693Z/0/index.zip] at org.apache.druid.storage.s3.S3DataSegmentPuller.getSegmentFiles(S3DataSegmentPuller.java:139) ~[?:?] at org.apache.druid.storage.s3.S3LoadSpec.loadSegment(S3LoadSpec.java:60) ~[?:?] at org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocation(SegmentLoaderLocalCacheManager.java:200) ~[druid-server-0.13.0-incubating-iap6-SNAPSHOT.jar:0.13.0-incubating-iap6-SNAPSHOT] at org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocationWithStartMarker(SegmentLoaderLocalCacheManager.java:188) ~[druid-server-0.13.0-incubating-iap6-SNAPSHOT.jar:0.13.0-incubating-iap6-SNAPSHOT] at org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager.loadSegmentWithRetry(SegmentLoaderLocalCacheManager.java:151) [
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
