adampolomski opened a new pull request, #742: URL: https://github.com/apache/arrow-java/pull/742
## What's Changed In some cases [InflaterInputStream]( https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/zip/InflaterInputStream.html) can return a non zero on `available()` method call, while it is actually at EOS. A subsequent `read()` call would respond with -1 and eventually cause: <pre> Suppressed: java.lang.IllegalStateException: Memory was leaked by query. Memory leaked: (32768) Allocator(flight-client) 0/32768/45416128/9223372036854775807 (res/actual/peak/limit) at org.apache.arrow.memory.BaseAllocator.close(BaseAllocator.java:504) at org.apache.arrow.flight.FlightClient.close(FlightClient.java:754) at com.relativity.rae.spark.mds.MDSReaderBuilder$$anon$1.close(MDSPartitionReader.scala:97) at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$advanceToNextIter$1(DataSourceRDD.scala:94) at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$advanceToNextIter$1$adapted(DataSourceRDD.scala:89) at org.apache.spark.TaskContext$$anon$1.onTaskCompletion(TaskContext.scala:137) at org.apache.spark.TaskContextImpl.$anonfun$invokeTaskCompletionListeners$1(TaskContextImpl.scala:144) at org.apache.spark.TaskContextImpl.$anonfun$invokeTaskCompletionListeners$1$adapted(TaskContextImpl.scala:144) at org.apache.spark.TaskContextImpl.invokeListeners(TaskContextImpl.scala:199) ... 12 common frames omitted Caused by: org.apache.arrow.flight.FlightRuntimeException: Failed to read message. at org.apache.arrow.flight.CallStatus.toRuntimeException(CallStatus.java:121) at org.apache.arrow.flight.grpc.StatusUtils.fromGrpcRuntimeException(StatusUtils.java:161) at org.apache.arrow.flight.grpc.StatusUtils.fromThrowable(StatusUtils.java:182) at org.apache.arrow.flight.FlightStream$Observer.onError(FlightStream.java:489) at org.apache.arrow.flight.FlightClient$1.onError(FlightClient.java:371) at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:564) at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) at org.apache.arrow.flight.grpc.ClientInterceptorAdapter$FlightClientCallListener.onClose(ClientInterceptorAdapter.java:118) at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:564) at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:72) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:729) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:710) at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) ... 3 common frames omitted Caused by: java.lang.RuntimeException: com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length. at org.apache.arrow.flight.ArrowMessage.frame(ArrowMessage.java:363) at org.apache.arrow.flight.ArrowMessage$ArrowMessageHolderMarshaller.parse(ArrowMessage.java:575) at org.apache.arrow.flight.ArrowMessage$ArrowMessageHolderMarshaller.parse(ArrowMessage.java:560) at io.grpc.MethodDescriptor.parseResponse(MethodDescriptor.java:284) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInternal(ClientCallImpl.java:657) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:644) ... 5 common frames omitted Caused by: com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length. at com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:92) at com.google.protobuf.CodedInputStream.readRawVarint32(CodedInputStream.java:568) at org.apache.arrow.flight.ArrowMessage.readRawVarint32(ArrowMessage.java:369) at org.apache.arrow.flight.ArrowMessage.frame(ArrowMessage.java:290) ... 10 common frames omitted Suppressed: java.lang.IllegalStateException: Memory was leaked by query. Memory leaked: (32768) Allocator(flight-client) 0/32768/45416128/9223372036854775807 (res/actual/peak/limit) at org.apache.arrow.memory.BaseAllocator.close(BaseAllocator.java:504) at org.apache.arrow.flight.FlightClient.close(FlightClient.java:754) at com.relativity.rae.spark.mds.MDSReaderBuilder$$anon$1.close(MDSPartitionReader.scala:97) at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$advanceToNextIter$1(DataSourceRDD.scala:94) at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$advanceToNextIter$1$adapted(DataSourceRDD.scala:89) at org.apache.spark.TaskContext$$anon$1.onTaskCompletion(TaskContext.scala:137) at org.apache.spark.TaskContextImpl.$anonfun$invokeTaskCompletionListeners$1(TaskContextImpl.scala:144) at org.apache.spark.TaskContextImpl.$anonfun$invokeTaskCompletionListeners$1$adapted(TaskContextImpl.scala:144) at org.apache.spark.TaskContextImpl.invokeListeners(TaskContextImpl.scala:199) ... 12 common frames omitted Caused by: org.apache.arrow.flight.FlightRuntimeException: Failed to read message. at org.apache.arrow.flight.CallStatus.toRuntimeException(CallStatus.java:121) at org.apache.arrow.flight.grpc.StatusUtils.fromGrpcRuntimeException(StatusUtils.java:161) at org.apache.arrow.flight.grpc.StatusUtils.fromThrowable(StatusUtils.java:182) at org.apache.arrow.flight.FlightStream$Observer.onError(FlightStream.java:489) at org.apache.arrow.flight.FlightClient$1.onError(FlightClient.java:371) at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:564) at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) at org.apache.arrow.flight.grpc.ClientInterceptorAdapter$FlightClientCallListener.onClose(ClientInterceptorAdapter.java:118) at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:564) at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:72) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:729) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:710) at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) ... 3 common frames omitted Caused by: java.lang.RuntimeException: com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length. at org.apache.arrow.flight.ArrowMessage.frame(ArrowMessage.java:363) at org.apache.arrow.flight.ArrowMessage$ArrowMessageHolderMarshaller.parse(ArrowMessage.java:575) at org.apache.arrow.flight.ArrowMessage$ArrowMessageHolderMarshaller.parse(ArrowMessage.java:560) at io.grpc.MethodDescriptor.parseResponse(MethodDescriptor.java:284) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInternal(ClientCallImpl.java:657) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:644) ... 5 common frames omitted Caused by: com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length. at com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:92) at com.google.protobuf.CodedInputStream.readRawVarint32(CodedInputStream.java:568) at org.apache.arrow.flight.ArrowMessage.readRawVarint32(ArrowMessage.java:369) at org.apache.arrow.flight.ArrowMessage.frame(ArrowMessage.java:290) ... 10 common frames omitted </pre> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
