[
https://issues.apache.org/jira/browse/IMPALA-8521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041379#comment-17041379
]
Sahil Takiar commented on IMPALA-8521:
--------------------------------------
Okay, looks like this is a bug in the HDFS client code. At Joe pointed out,
this test is intentionally written to cause a call to {{hadoopReadZero}} to
fail in order to test the fallback logic. I confirmed that a call to
{{hadoopReadZero}} fails because of the combination of encryption zones + ZCR
(as reported in IMPALA-3679). The stack trace thrown by the HDFS client is the
same one shown in IMPALA-3679. The bug seems to be that if a
{{CryptoInputStream}} fails to decrypt a buffer obtained from a zero-copy read
in {{DFSInputStream}}, it doesn't clean up the buffer via
{{DFSInputStream#releaseBuffer}}. It seems that for each zero-copy read there
needs to be an explicit call to {{#releaseBuffer}}.
So I don't think there is anything Impala should be doing differently here.
I'll surface this with some HDFS folks to see if encryption zones + ZCR are now
supported and if this bug should be fixed.
> Lots of "unreleased ByteBuffers allocated by read()" errors from HDFS client
> ----------------------------------------------------------------------------
>
> Key: IMPALA-8521
> URL: https://issues.apache.org/jira/browse/IMPALA-8521
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Sahil Takiar
> Priority: Critical
>
> I'm looking at some job logs and seeing a bunch of errors like this. I don't
> know if it's benign or if it's something more serious.
> {noformat}
> I0507 07:34:53.934693 20195 scan-range.cc:607]
> dd4d6eb8d2ad9587:6b44fe1b00000002] Cache read failed for scan range:
> file=hdfs://localhost:20500/test-warehouse/f861f1a3/nation.tbl disk_id=0
> offset=1024 exclusive_hdfs_fh=0xec09220 num_remote_bytes=0 cancel_status=
> buffer_queue=0 num_buffers_in_readers=0 unused_iomgr_buffers=0
> unused_iomgr_buffer_bytes=0 blocked_on_buffer=0. Switching to disk read path.
> W0507 07:34:53.934787 20195 DFSInputStream.java:668]
> dd4d6eb8d2ad9587:6b44fe1b00000002] closing file
> /test-warehouse/f861f1a3/nation.tbl, but there are still unreleased
> ByteBuffers allocated by read(). Please release
> java.nio.DirectByteBufferR[pos=1024 lim=2048 cap=2199].
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
