Manish,
400 error typically happens on the client side. Check to see if the common
configuration has the proper configuration to read/write from/to S3.
Paste here as well the Overlord log.
Rommel Garcia
> On Mar 25, 2019, at 5:11 AM, Manish Jaiswal <manish.jais...@go-mmt.com> wrote:
>
> Dear Team,
>
> Since last week I was trying to setting up S3 as a Deep data storage in druid
> but I was getting error while access in the data.
>
> Would be great if you can help us to resolve this issue.
>
> Getting below error in historical.log
>
> 019-03-25T09:03:26,235 ERROR [ZkCoordinator]
> io.druid.server.coordination.SegmentLoadDropHandler - Failed to load segment
> for dataSource: {class=io.druid.server.coordination.SegmentLoadDropHandler,
> exceptionType=class io.druid.segment.loading.SegmentLoadingException,
> exceptionMessage=Exception loading
> segment[sample-data_2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z_2019-03-22T07:18:37.993Z],
> segment=DataSegment{size=4826887,
> shardSpec=NumberedShardSpec{partitionNum=0, partitions=0}, metrics=[],
> dimensions=[channel, cityName, comment, countryIsoCode, countryName,
> isAnonymous, isMinor, isNew, isRobot, isUnpatrolled, metroCode, namespace,
> page, regionIsoCode, regionName, user, added, deleted, delta],
> version='2019-03-22T07:18:37.993Z', loadSpec={type=>s3_zip, bucket=>gi-druid,
> key=>druid/segments/sample-data/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2019-03-22T07:18:37.993Z/0/index.zip,
> S3Schema=>s3n}, interval=2015-09-12T00:00:00.000Z/2015-09-13T00:00:00.000Z,
> dataSource='sample-data', binaryVersion='9'}}
> io.druid.segment.loading.SegmentLoadingException: Exception loading
> segment[sample-data_2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z_2019-03-22T07:18:37.993Z]
> at
> io.druid.server.coordination.SegmentLoadDropHandler.loadSegment(SegmentLoadDropHandler.java:265)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.server.coordination.SegmentLoadDropHandler.addSegment(SegmentLoadDropHandler.java:307)
> [druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.server.coordination.SegmentChangeRequestLoad.go(SegmentChangeRequestLoad.java:47)
> [druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.server.coordination.ZkCoordinator$1.childEvent(ZkCoordinator.java:118)
> [druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:520)
> [curator-recipes-4.0.0.jar:4.0.0]
> at
> org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:514)
> [curator-recipes-4.0.0.jar:4.0.0]
> at
> org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:93)
> [curator-framework-4.0.0.jar:4.0.0]
> at
> org.apache.curator.shaded.com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:296)
> [curator-client-4.0.0.jar:?]
> at
> org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:85)
> [curator-framework-4.0.0.jar:4.0.0]
> at
> org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:512)
> [curator-recipes-4.0.0.jar:4.0.0]
> at
> org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35)
> [curator-recipes-4.0.0.jar:4.0.0]
> at
> org.apache.curator.framework.recipes.cache.PathChildrenCache$9.run(PathChildrenCache.java:771)
> [curator-recipes-4.0.0.jar:4.0.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_112]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_112]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_112]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [?:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [?:1.8.0_112]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
> Caused by: java.lang.RuntimeException: org.jets3t.service.ServiceException:
> Request Error. -- ResponseCode: 400, ResponseStatus: Bad Request, RequestId:
> 77A29FC0BC77C548, HostId:
> 3lzWI3pz67xokOKj1zYLvNCdzsK3BfVk8aAViMyayvd9DDmKo+VL1plzAWbvzsDFoD0+GU2ieRw=
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> ~[guava-16.0.1.jar:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller.isObjectInBucket(S3DataSegmentPuller.java:327)
> ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller.getSegmentFiles(S3DataSegmentPuller.java:176)
> ~[?:?]
> at io.druid.storage.s3.S3LoadSpec.loadSegment(S3LoadSpec.java:60) ~[?:?]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocation(SegmentLoaderLocalCacheManager.java:205)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocationWithStartMarker(SegmentLoaderLocalCacheManager.java:193)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadSegmentWithRetry(SegmentLoaderLocalCacheManager.java:151)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.getSegmentFiles(SegmentLoaderLocalCacheManager.java:133)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.getSegment(SegmentLoaderLocalCacheManager.java:108)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.server.SegmentManager.getAdapter(SegmentManager.java:196)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.server.SegmentManager.loadSegment(SegmentManager.java:157)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.server.coordination.SegmentLoadDropHandler.loadSegment(SegmentLoadDropHandler.java:261)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> ... 18 more
> Caused by: org.jets3t.service.ServiceException: Request Error.
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:426)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:279)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestHead(RestStorageService.java:1052)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectImpl(RestStorageService.java:2264)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectDetailsImpl(RestStorageService.java:2193)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.StorageService.getObjectDetails(StorageService.java:1120)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.StorageService.getObjectDetails(StorageService.java:575)
> ~[jets3t-0.9.4.jar:0.9.4]
> at io.druid.storage.s3.S3Utils.isObjectInBucket(S3Utils.java:96) ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller$4.call(S3DataSegmentPuller.java:318)
> ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller$4.call(S3DataSegmentPuller.java:314)
> ~[?:?]
> at io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:63)
> ~[java-util-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81)
> ~[java-util-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.storage.s3.S3Utils.retryS3Operation(S3Utils.java:89) ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller.isObjectInBucket(S3DataSegmentPuller.java:312)
> ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller.getSegmentFiles(S3DataSegmentPuller.java:176)
> ~[?:?]
> at io.druid.storage.s3.S3LoadSpec.loadSegment(S3LoadSpec.java:60) ~[?:?]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocation(SegmentLoaderLocalCacheManager.java:205)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocationWithStartMarker(SegmentLoaderLocalCacheManager.java:193)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadSegmentWithRetry(SegmentLoaderLocalCacheManager.java:151)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.getSegmentFiles(SegmentLoaderLocalCacheManager.java:133)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.getSegment(SegmentLoaderLocalCacheManager.java:108)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.server.SegmentManager.getAdapter(SegmentManager.java:196)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.server.SegmentManager.loadSegment(SegmentManager.java:157)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.server.coordination.SegmentLoadDropHandler.loadSegment(SegmentLoadDropHandler.java:261)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> ... 18 more
> Caused by: org.jets3t.service.impl.rest.HttpException: 400 Bad Request
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:425)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:279)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestHead(RestStorageService.java:1052)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectImpl(RestStorageService.java:2264)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectDetailsImpl(RestStorageService.java:2193)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.StorageService.getObjectDetails(StorageService.java:1120)
> ~[jets3t-0.9.4.jar:0.9.4]
> at
> org.jets3t.service.StorageService.getObjectDetails(StorageService.java:575)
> ~[jets3t-0.9.4.jar:0.9.4]
> at io.druid.storage.s3.S3Utils.isObjectInBucket(S3Utils.java:96) ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller$4.call(S3DataSegmentPuller.java:318)
> ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller$4.call(S3DataSegmentPuller.java:314)
> ~[?:?]
> at io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:63)
> ~[java-util-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81)
> ~[java-util-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.storage.s3.S3Utils.retryS3Operation(S3Utils.java:89) ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller.isObjectInBucket(S3DataSegmentPuller.java:312)
> ~[?:?]
> at
> io.druid.storage.s3.S3DataSegmentPuller.getSegmentFiles(S3DataSegmentPuller.java:176)
> ~[?:?]
> at io.druid.storage.s3.S3LoadSpec.loadSegment(S3LoadSpec.java:60) ~[?:?]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocation(SegmentLoaderLocalCacheManager.java:205)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocationWithStartMarker(SegmentLoaderLocalCacheManager.java:193)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.loadSegmentWithRetry(SegmentLoaderLocalCacheManager.java:151)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.getSegmentFiles(SegmentLoaderLocalCacheManager.java:133)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.segment.loading.SegmentLoaderLocalCacheManager.getSegment(SegmentLoaderLocalCacheManager.java:108)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.server.SegmentManager.getAdapter(SegmentManager.java:196)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at io.druid.server.SegmentManager.loadSegment(SegmentManager.java:157)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> at
> io.druid.server.coordination.SegmentLoadDropHandler.loadSegment(SegmentLoadDropHandler.java:261)
> ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
> ... 18 more
> 2019-03-25T09:03:26,242 INFO [ZkCoordinator]
> io.druid.server.coordination.ZkCoordinator - Completed request [LOAD:
> sample-data_2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z_2019-03-22T07:18:37.993Z]
>
>
>
> --
> Thanks & Regards
> Manish Jaiswal
> 9379572574
>
> ::DISCLAIMER::
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
>
>
> This message is intended only for the use of the addressee and may contain
> information that is privileged, confidential and exempt from disclosure under
> applicable law. If the reader of this message is not the intended recipient,
> or the employee or agent responsible for delivering the message to the
> intended recipient, you are hereby notified that any dissemination,
> distribution or copying of this communication is strictly prohibited. If you
> have received this e-mail in error, please notify us immediately by return
> e-mail and delete this e-mail and all attachments from your system.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Druid User" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-user+unsubscr...@googlegroups.com
> <mailto:druid-user+unsubscr...@googlegroups.com>.
> To post to this group, send email to druid-u...@googlegroups.com
> <mailto:druid-u...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/druid-user/CAM3MDBDL0myWPY7uVtYrZ5S6OOmmn1TT18J5RAQi_jztb9xHpg%40mail.gmail.com
>
> <https://groups.google.com/d/msgid/druid-user/CAM3MDBDL0myWPY7uVtYrZ5S6OOmmn1TT18J5RAQi_jztb9xHpg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
> <broker.log><coordinator.log><historical.log><common.runtime.properties>
