acherla opened a new issue, #12820:
URL: https://github.com/apache/druid/issues/12820
Hi Folks,
Im noticing occassionally when attempting to push large segments (> 1 GB in
size) to GCS, I occasionally get this error thrown back by the index_parallel
tasks. Im wondering if this is due to an issue
```
2022-07-26T11:59:52,722 INFO [task-runner-0-priority-0]
org.apache.druid.indexing.common.task.batch.parallel.SinglePhaseSubTask -
Pushed [1] segments
2022-07-26T11:59:52,723 INFO [task-runner-0-priority-0]
org.apache.druid.indexing.common.task.batch.parallel.SinglePhaseSubTask -
Pushed segments:
[truecall_geogrid_env_hourly_2022-07-26T00:00:00.000Z_2022-07-26T01:00:00.000Z_2022-07-26T11:44:36.085Z_8]
2022-07-26T11:59:53,232 WARN [task-runner-0-priority-0]
org.apache.druid.java.util.common.RetryUtils - Retrying (1 of 10) in 709ms.
java.io.IOException: Connection closed prematurely: bytesRead = 27303936,
Content-Length = 84586633
at
com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.throwIfFalseEOF(NetHttpResponse.java:209)
~[google-http-client-1.26.0.jar:?]
at
com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.read(NetHttpResponse.java:171)
~[google-http-client-1.26.0.jar:?]
at
com.google.common.io.CountingInputStream.read(CountingInputStream.java:62)
~[guava-16.0.1.jar:?]
at
org.apache.druid.data.input.impl.RetryingInputStream.read(RetryingInputStream.java:144)
[druid-core-0.22.1.jar:0.22.1]
at java.io.FilterInputStream.read(FilterInputStream.java:133)
[?:1.8.0_275]
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
[?:1.8.0_275]
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
[?:1.8.0_275]
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) [?:1.8.0_275]
at java.io.InputStreamReader.read(InputStreamReader.java:184)
[?:1.8.0_275]
at java.io.BufferedReader.fill(BufferedReader.java:161) [?:1.8.0_275]
at java.io.BufferedReader.readLine(BufferedReader.java:324)
[?:1.8.0_275]
at java.io.BufferedReader.readLine(BufferedReader.java:389)
[?:1.8.0_275]
at org.apache.commons.io.LineIterator.hasNext(LineIterator.java:96)
[commons-io-2.11.0.jar:2.11.0]
at org.apache.druid.data.input.TextReader$1.hasNext(TextReader.java:73)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.data.input.IntermediateRowParsingReader$1.hasNext(IntermediateRowParsingReader.java:60)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$2.findNextIteratorIfNecessary(CloseableIterator.java:74)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$2.next(CloseableIterator.java:108)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$1.next(CloseableIterator.java:52)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.FilteringCloseableInputRowIterator.hasNext(FilteringCloseableInputRowIterator.java:68)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.batch.parallel.SinglePhaseSubTask.generateAndPushSegments(SinglePhaseSubTask.java:375)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.batch.parallel.SinglePhaseSubTask.runTask(SinglePhaseSubTask.java:209)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.AbstractBatchIndexTask.run(AbstractBatchIndexTask.java:159)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:471)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:443)
[druid-indexing-service-0.22.1.jar:0.22.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_275]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_275]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
2022-07-26T11:59:53,942 WARN [task-runner-0-priority-0]
org.apache.druid.java.util.common.RetryUtils - Retrying (2 of 10) in 1,737ms.
java.io.IOException: Connection closed prematurely: bytesRead = 27303936,
Content-Length = 84586633
at
com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.throwIfFalseEOF(NetHttpResponse.java:209)
~[google-http-client-1.26.0.jar:?]
at
com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.read(NetHttpResponse.java:171)
~[google-http-client-1.26.0.jar:?]
at
com.google.common.io.CountingInputStream.read(CountingInputStream.java:62)
~[guava-16.0.1.jar:?]
at
org.apache.druid.data.input.impl.RetryingInputStream.read(RetryingInputStream.java:144)
[druid-core-0.22.1.jar:0.22.1]
at java.io.FilterInputStream.read(FilterInputStream.java:133)
[?:1.8.0_275]
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
[?:1.8.0_275]
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
[?:1.8.0_275]
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) [?:1.8.0_275]
at java.io.InputStreamReader.read(InputStreamReader.java:184)
[?:1.8.0_275]
at java.io.BufferedReader.fill(BufferedReader.java:161) [?:1.8.0_275]
at java.io.BufferedReader.readLine(BufferedReader.java:324)
[?:1.8.0_275]
at java.io.BufferedReader.readLine(BufferedReader.java:389)
[?:1.8.0_275]
at org.apache.commons.io.LineIterator.hasNext(LineIterator.java:96)
[commons-io-2.11.0.jar:2.11.0]
at org.apache.druid.data.input.TextReader$1.hasNext(TextReader.java:73)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.data.input.IntermediateRowParsingReader$1.hasNext(IntermediateRowParsingReader.java:60)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$2.findNextIteratorIfNecessary(CloseableIterator.java:74)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$2.next(CloseableIterator.java:108)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$1.next(CloseableIterator.java:52)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.FilteringCloseableInputRowIterator.hasNext(FilteringCloseableInputRowIterator.java:68)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.batch.parallel.SinglePhaseSubTask.generateAndPushSegments(SinglePhaseSubTask.java:375)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.batch.parallel.SinglePhaseSubTask.runTask(SinglePhaseSubTask.java:209)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.AbstractBatchIndexTask.run(AbstractBatchIndexTask.java:159)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:471)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:443)
[druid-indexing-service-0.22.1.jar:0.22.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_275]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_275]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
2022-07-26T11:59:55,680 WARN [task-runner-0-priority-0]
org.apache.druid.java.util.common.RetryUtils - Retrying (3 of 10) in 3,773ms.
java.io.IOException: Connection closed prematurely: bytesRead = 27303936,
Content-Length = 84586633
at
com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.throwIfFalseEOF(NetHttpResponse.java:209)
~[google-http-client-1.26.0.jar:?]
at
com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.read(NetHttpResponse.java:171)
~[google-http-client-1.26.0.jar:?]
at
com.google.common.io.CountingInputStream.read(CountingInputStream.java:62)
~[guava-16.0.1.jar:?]
at
org.apache.druid.data.input.impl.RetryingInputStream.read(RetryingInputStream.java:144)
[druid-core-0.22.1.jar:0.22.1]
at java.io.FilterInputStream.read(FilterInputStream.java:133)
[?:1.8.0_275]
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
[?:1.8.0_275]
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
[?:1.8.0_275]
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
[?:1.8.0_275]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) [?:1.8.0_275]
at java.io.InputStreamReader.read(InputStreamReader.java:184)
[?:1.8.0_275]
at java.io.BufferedReader.fill(BufferedReader.java:161) [?:1.8.0_275]
at java.io.BufferedReader.readLine(BufferedReader.java:324)
[?:1.8.0_275]
at java.io.BufferedReader.readLine(BufferedReader.java:389)
[?:1.8.0_275]
at org.apache.commons.io.LineIterator.hasNext(LineIterator.java:96)
[commons-io-2.11.0.jar:2.11.0]
at org.apache.druid.data.input.TextReader$1.hasNext(TextReader.java:73)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.data.input.IntermediateRowParsingReader$1.hasNext(IntermediateRowParsingReader.java:60)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$2.findNextIteratorIfNecessary(CloseableIterator.java:74)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$2.next(CloseableIterator.java:108)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.java.util.common.parsers.CloseableIterator$1.next(CloseableIterator.java:52)
[druid-core-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.FilteringCloseableInputRowIterator.hasNext(FilteringCloseableInputRowIterator.java:68)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.batch.parallel.SinglePhaseSubTask.generateAndPushSegments(SinglePhaseSubTask.java:375)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.batch.parallel.SinglePhaseSubTask.runTask(SinglePhaseSubTask.java:209)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.common.task.AbstractBatchIndexTask.run(AbstractBatchIndexTask.java:159)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:471)
[druid-indexing-service-0.22.1.jar:0.22.1]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:443)
[druid-indexing-service-0.22.1.jar:0.22.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_275]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_275]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
