[
https://issues.apache.org/jira/browse/HDFS-16422?focusedWorklogId=715015&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-715015
]
ASF GitHub Bot logged work on HDFS-16422:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Jan/22 17:06
Start Date: 25/Jan/22 17:06
Worklog Time Spent: 10m
Work Description: jojochuang commented on a change in pull request #3881:
URL: https://github.com/apache/hadoop/pull/3881#discussion_r791286115
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripeReader.java
##########
@@ -432,7 +432,9 @@ void decodeAndFillBuffer(boolean fillBuffer) throws
IOException {
long start = Time.monotonicNow();
// Step 2: decode into prepared output buffers
- decoder.decode(decodeInputs, decodeIndices, outputs);
+ synchronized (decoder) {
Review comment:
IMO, we should make RawErasureDecoder.decode() synchronized instead.
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripeReader.java
##########
@@ -432,7 +432,9 @@ void decodeAndFillBuffer(boolean fillBuffer) throws
IOException {
long start = Time.monotonicNow();
// Step 2: decode into prepared output buffers
- decoder.decode(decodeInputs, decodeIndices, outputs);
+ synchronized (decoder) {
Review comment:
IMO, we should make RawErasureDecoder.decode(ByteBuffer[] inputs, int[]
erasedIndexes, ByteBuffer[] outputs) synchronized instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 715015)
Time Spent: 1h 40m (was: 1.5h)
> Fix thread safety of EC decoding during concurrent preads
> ---------------------------------------------------------
>
> Key: HDFS-16422
> URL: https://issues.apache.org/jira/browse/HDFS-16422
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: dfsclient, ec, erasure-coding
> Affects Versions: 3.3.0, 3.3.1
> Reporter: daimin
> Assignee: daimin
> Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Reading data on an erasure-coded file with missing replicas(internal block of
> block group) will cause online reconstruction: read dataUnits part of data
> and decode them into the target missing data. Each DFSStripedInputStream
> object has a RawErasureDecoder object, and when we doing pread concurrently,
> RawErasureDecoder.decode will be invoked concurrently too.
> RawErasureDecoder.decode is not thread safe, as a result of that we get wrong
> data from pread occasionally.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
