[
https://issues.apache.org/jira/browse/HDDS-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16725605#comment-16725605
]
Nanda kumar commented on HDDS-934:
----------------------------------
Committed this to trunk. Thanks [~shashikant] for review.
> Freon run hung on non-functional pipeline
> -----------------------------------------
>
> Key: HDDS-934
> URL: https://issues.apache.org/jira/browse/HDDS-934
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Client
> Reporter: Nilotpal Nandi
> Assignee: Nanda kumar
> Priority: Major
> Fix For: 0.4.0
>
> Attachments: HDDS-934.000.patch, jstack.txt
>
>
> steps taken:
> # created docker cluster with 3 datanodes where datanodes are not able to
> communicate with each other.
> # ran freon to write 1 key
> freon run threw following exception but it did not terminate.
> exception:
> ------------------
>
> {noformat}
> 2018-12-18 10:38:06 INFO RandomKeyGenerator:227 - Number of Threads: 10
> 2018-12-18 10:38:06 INFO RandomKeyGenerator:233 - Number of Volumes: 1.
> 2018-12-18 10:38:06 INFO RandomKeyGenerator:234 - Number of Buckets per
> Volume: 1.
> 2018-12-18 10:38:06 INFO RandomKeyGenerator:235 - Number of Keys per Bucket:
> 1.
> 2018-12-18 10:38:06 INFO RandomKeyGenerator:236 - Key size: 10240 bytes
> 2018-12-18 10:38:06 INFO RandomKeyGenerator:266 - Starting progress bar
> Thread.
> 0.00% |█ | 0/1 Time: 0:00:002018-12-18 10:38:06 INFO RpcClient:250 - Creating
> Volume: vol-0-74492, with hadoop as owner and quota set to
> 1152921504606846976 bytes.
> 2018-12-18 10:38:06 INFO RpcClient:379 - Creating Bucket:
> vol-0-74492/bucket-0-16002, with Versioning false and Storage Type set to DISK
> 0.00% |█ | 0/1 Time: 0:02:402018-12-18 10:40:46 ERROR
> ChunkGroupOutputStream:275 - Try to allocate more blocks for write failed,
> already allocated 0 blocks for this write.
> 2018-12-18 10:40:46 ERROR RandomKeyGenerator:624 - Exception while adding
> key: key-0-28925 in bucket:
> org.apache.hadoop.ozone.client.OzoneBucket@1675c402 of volume:
> org.apache.hadoop.ozone.client.OzoneVolume@5b6bfafd.
> java.io.IOException: Allocate block failed, error:INTERNAL_ERROR
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.allocateBlock(OzoneManagerProtocolClientSideTranslatorPB.java:620)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.allocateNewBlock(ChunkGroupOutputStream.java:437)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleWrite(ChunkGroupOutputStream.java:272)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleException(ChunkGroupOutputStream.java:377)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleFlushOrClose(ChunkGroupOutputStream.java:473)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleFlushOrClose(ChunkGroupOutputStream.java:474)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleFlushOrClose(ChunkGroupOutputStream.java:474)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleFlushOrClose(ChunkGroupOutputStream.java:474)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleWrite(ChunkGroupOutputStream.java:309)
> at
> org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.write(ChunkGroupOutputStream.java:255)
> at
> org.apache.hadoop.ozone.client.io.OzoneOutputStream.write(OzoneOutputStream.java:49)
> at java.io.OutputStream.write(OutputStream.java:75)
> at
> org.apache.hadoop.ozone.freon.RandomKeyGenerator$OfflineProcessor.run(RandomKeyGenerator.java:606)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 0.00% |█ | 0/1 Time: 0:11:01
> 0.00% |█ | 0/1 Time: 0:11:19
> {noformat}
> Here is the jstack for the freon process :
> [^jstack.txt]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
