[
https://issues.apache.org/jira/browse/HIVE-19395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Doe updated HIVE-19395:
----------------------------
Description:
When the bufferSize is configured to be 0 in the class initialization, the
while loop in OutStream.write function hangs endlessly.
This is because when the bufferSize is 0, current.remaining is 0, length will
always > 0.
Here is the code snippet.
{code:java}
OutStream(String name, int bufferSize, CompressionCodec codec,
OutputReceiver receiver) throws IOException {
...
this.bufferSize = bufferSize; //bufferSize can be configured with 0
...
}
private void getNewInputBuffer() throws IOException {
...
current = ByteBuffer.allocate(bufferSize);
...
}
public void write(byte[] bytes, int offset, int length) throws IOException {
if (current == null) {
getNewInputBuffer();
}
int remaining = Math.min(current.remaining(), length);
current.put(bytes, offset, remaining);
uncompressedBytes += remaining;
length -= remaining;
while (length != 0) {//length > 0
spill();
offset += remaining;
remaining = Math.min(current.remaining(), length);//current.remaining()
== 0
current.put(bytes, offset, remaining);
uncompressedBytes += remaining;
length -= remaining;
}
}
{code}
The similar case is HDFS-13513, HDFS-13514
was:
When the bufferSize is configured to be 0 in the class initialization, the
while loop in OutStream.write function hangs endlessly.
This is because when the bufferSize is 0, current.remaining is 0, length will
always > 0.
Here is the code snippet.
{code:java}
OutStream(String name, int bufferSize, CompressionCodec codec,
OutputReceiver receiver) throws IOException {
...
this.bufferSize = bufferSize; //bufferSize can be configured with 0
...
}
private void getNewInputBuffer() throws IOException {
...
current = ByteBuffer.allocate(bufferSize);
...
}
public void write(byte[] bytes, int offset, int length) throws IOException {
if (current == null) {
getNewInputBuffer();
}
int remaining = Math.min(current.remaining(), length);
current.put(bytes, offset, remaining);
uncompressedBytes += remaining;
length -= remaining;
while (length != 0) {//length > 0
spill();
offset += remaining;
remaining = Math.min(current.remaining(), length);//current.remaining()
== 0
current.put(bytes, offset, remaining);
uncompressedBytes += remaining;
length -= remaining;
}
}
{code}
The similar case is
[HDFS-13513|https://issues.apache.org/jira/browse/HDFS-13513],
[HDFS-13514|https://issues.apache.org/jira/browse/HDFS-13514]
> OutStream.write hangs with misconfigured bufferSize
> ---------------------------------------------------
>
> Key: HIVE-19395
> URL: https://issues.apache.org/jira/browse/HIVE-19395
> Project: Hive
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.0.0
> Reporter: John Doe
> Priority: Minor
>
> When the bufferSize is configured to be 0 in the class initialization, the
> while loop in OutStream.write function hangs endlessly.
> This is because when the bufferSize is 0, current.remaining is 0, length will
> always > 0.
> Here is the code snippet.
> {code:java}
> OutStream(String name, int bufferSize, CompressionCodec codec,
> OutputReceiver receiver) throws IOException {
> ...
> this.bufferSize = bufferSize; //bufferSize can be configured with 0
> ...
> }
> private void getNewInputBuffer() throws IOException {
> ...
> current = ByteBuffer.allocate(bufferSize);
> ...
> }
> public void write(byte[] bytes, int offset, int length) throws IOException {
> if (current == null) {
> getNewInputBuffer();
> }
> int remaining = Math.min(current.remaining(), length);
> current.put(bytes, offset, remaining);
> uncompressedBytes += remaining;
> length -= remaining;
> while (length != 0) {//length > 0
> spill();
> offset += remaining;
> remaining = Math.min(current.remaining(), length);//current.remaining()
> == 0
> current.put(bytes, offset, remaining);
> uncompressedBytes += remaining;
> length -= remaining;
> }
> }
> {code}
> The similar case is HDFS-13513, HDFS-13514
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
