[
https://issues.apache.org/jira/browse/HDDS-2322?focusedWorklogId=334410&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-334410
]
ASF GitHub Bot logged work on HDDS-2322:
----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Oct/19 22:48
Start Date: 25/Oct/19 22:48
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #89:
HDDS-2322. DoubleBuffer flush termination and OM shutdown's after that. Make
entry returned from cache a new copy.
URL: https://github.com/apache/hadoop-ozone/pull/89
## What changes were proposed in this pull request?
Whenever value is returned from cache, it returns a copy object. So, that
when doubleBuffer flushes, we don't see random ConcurrentModificationException
errors like this.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2322
Please replace this section with the link to the Apache JIRA)
Ran freon tests with 100K Keys multiple times, now I am not seeing the error
given in Jira description. Without this patch, we will see the error. (If you
are not seeing error run multiple times in non-HA or just enable
ozone.om.ratis.enable, and you will see the error in the first run)
**Command used for testing:**
ozone freon rk --numOfBuckets=1 --numOfVolumes=1 --numOfKeys=100000
--keySize=0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 334410)
Remaining Estimate: 0h
Time Spent: 10m
> DoubleBuffer flush termination and OM shutdown's after that.
> ------------------------------------------------------------
>
> Key: HDDS-2322
> URL: https://issues.apache.org/jira/browse/HDDS-2322
> Project: Hadoop Distributed Data Store
> Issue Type: Task
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> om1_1 | 2019-10-18 00:34:45,317 [OMDoubleBufferFlushThread] ERROR
> - Terminating with exit status 2: OMDoubleBuffer flush
> threadOMDoubleBufferFlushThreadencountered Throwable error
> om1_1 | java.util.ConcurrentModificationException
> om1_1 | at
> java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1660)
> om1_1 | at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
> om1_1 | at
> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
> om1_1 | at
> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
> om1_1 | at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> om1_1 | at
> java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
> om1_1 | at
> org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfoGroup.getProtobuf(OmKeyLocationInfoGroup.java:65)
> om1_1 | at
> java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
> om1_1 | at
> java.base/java.util.Collections$2.tryAdvance(Collections.java:4745)
> om1_1 | at
> java.base/java.util.Collections$2.forEachRemaining(Collections.java:4753)
> om1_1 | at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
> om1_1 | at
> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
> om1_1 | at
> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
> om1_1 | at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> om1_1 | at
> java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
> om1_1 | at
> org.apache.hadoop.ozone.om.helpers.OmKeyInfo.getProtobuf(OmKeyInfo.java:362)
> om1_1 | at
> org.apache.hadoop.ozone.om.codec.OmKeyInfoCodec.toPersistedFormat(OmKeyInfoCodec.java:37)
> om1_1 | at
> org.apache.hadoop.ozone.om.codec.OmKeyInfoCodec.toPersistedFormat(OmKeyInfoCodec.java:31)
> om1_1 | at
> org.apache.hadoop.hdds.utils.db.CodecRegistry.asRawData(CodecRegistry.java:68)
> om1_1 | at
> org.apache.hadoop.hdds.utils.db.TypedTable.putWithBatch(TypedTable.java:125)
> om1_1 | at
> org.apache.hadoop.ozone.om.response.key.OMKeyCreateResponse.addToDBBatch(OMKeyCreateResponse.java:58)
> om1_1 | at
> org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.lambda$flushTransactions$0(OzoneManagerDoubleBuffer.java:139)
> om1_1 | at
> java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
> om1_1 | at
> org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushTransactions(OzoneManagerDoubleBuffer.java:137)
> om1_1 | at java.base/java.lang.Thread.run(Thread.java:834)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
