chairmank edited a comment on issue #3715: Pulsar functions worker fails to initialize URL: https://github.com/apache/pulsar/issues/3715#issuecomment-477584106 I see this problem only when triggering offload to tiered storage: ``` 14:13:53.419 [bookkeeper-ml-workers-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [foo/test/persistent/redacted] End Offlo[1825/30696] 60, uuid=dc6c1e97-3e9c-4bbd-aa2a-fba14250eedc 14:13:53.420 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [foo/test/persistent/redacted] Preparing metadata to offload ledger 161 with uuid a929b551-f701-4219-828b-e22dcf79a0e7 14:13:53.428 [bookkeeper-ml-workers-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [foo/test/persistent/redacted] Metadata prepared for offload of ledger 161 with uuid a929b551-f701-4219-828b-e22dcf79a0e7 14:13:53.447 [offloader-OrderedScheduler-1-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught java.lang.ArrayIndexOutOfBoundsException: Index -2 out of bounds for length 3 at com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:60) ~[com.google.guava-guava-21.0.jar:?] at org.apache.bookkeeper.client.PendingReadOp$SequenceReadRequest.sendNextRead(PendingReadOp.java:400) ~[org.apache.bookkeeper-bookkeeper-server-4.9.0.jar:4.9.0] at org.apache.bookkeeper.client.PendingReadOp$SequenceReadRequest.read(PendingReadOp.java:382) ~[org.apache.bookkeeper-bookkeeper-server-4.9.0.jar:4.9.0] at org.apache.bookkeeper.client.PendingReadOp.initiate(PendingReadOp.java:526) ~[org.apache.bookkeeper-bookkeeper-server-4.9.0.jar:4.9.0] at org.apache.bookkeeper.client.PendingReadOp.safeRun(PendingReadOp.java:536) ~[org.apache.bookkeeper-bookkeeper-server-4.9.0.jar:4.9.0] at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.9.0.jar:4.9.0] at org.apache.bookkeeper.client.LedgerHandle.readEntriesInternalAsync(LedgerHandle.java:894) [org.apache.bookkeeper-bookkeeper-server-4.9.0.jar:4.9.0] at org.apache.bookkeeper.client.LedgerHandle.readAsync(LedgerHandle.java:757) [org.apache.bookkeeper-bookkeeper-server-4.9.0.jar:4.9.0] at org.apache.bookkeeper.mledger.offload.jcloud.impl.BlockAwareSegmentInputStreamImpl.readNextEntriesFromLedger(BlockAwareSegmentInputStreamImpl.java:117) [tiered-storage-jc loud-2.4.0-SNAPSHOT.nar-unpacked/:?] at org.apache.bookkeeper.mledger.offload.jcloud.impl.BlockAwareSegmentInputStreamImpl.readEntries(BlockAwareSegmentInputStreamImpl.java:89) [tiered-storage-jcloud-2.4.0-SNAP SHOT.nar-unpacked/:?] at org.apache.bookkeeper.mledger.offload.jcloud.impl.BlockAwareSegmentInputStreamImpl.read(BlockAwareSegmentInputStreamImpl.java:157) [tiered-storage-jcloud-2.4.0-SNAPSHOT.n ar-unpacked/:?] at java.io.InputStream.read(InputStream.java:280) [?:?] at java.io.InputStream.read(InputStream.java:205) [?:?] at org.apache.pulsar.shaded.com.google.common.io.ByteStreams.copy(ByteStreams.java:106) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:298) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0- SNAPSHOT] at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:171) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:65) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:97) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at com.sun.proxy.$Proxy72.uploadPart(Unknown Source) [?:?] at org.jclouds.s3.blobstore.S3BlobStore.uploadMultipartPart(S3BlobStore.java:393) [jclouds-shaded-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT] at org.apache.bookkeeper.mledger.offload.jcloud.impl.BlobStoreManagedLedgerOffloader.lambda$offload$2(BlobStoreManagedLedgerOffloader.java:424) [tiered-storage-jcloud-2.4.0- SNAPSHOT.nar-unpacked/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111) [com.google.guava-guav a-21.0.jar:?] at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) [com.google.guava-guava-21.0.jar:?] at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75) [com.google.guava-guava-21.0.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] ``` On https://github.com/apache/bookkeeper/issues/1970, I described my investigation of this issue. Something (I haven't traced exactly what) is calling a method with an entry ID of `-2`, and `org.apache.bookkeeper.client.RoundRobinDistributionSchedule` can not handle a negative `entryId`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
