[ https://issues.apache.org/jira/browse/HADOOP-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359718#comment-14359718 ]
Yi Liu commented on HADOOP-11708: --------------------------------- I also +1 for changing CryptoOutputStream to behave the same as HDFS. We could not make DFSOutputStream or CryptOutputStream *synchronized* for all methods, that would affect performance, in most cases, applications should handle the synchronization, so it's enough we keep the same behave as HDFS. Sorry that I could not get time working on HDFS-7911 in the past two days for personal reason. Since [~busbey] has a patch in HADOOP-11710, I would mark HDFS-7911 as duplicated. > CryptoOutputStream synchronization differences from DFSOutputStream break > HBase > ------------------------------------------------------------------------------- > > Key: HADOOP-11708 > URL: https://issues.apache.org/jira/browse/HADOOP-11708 > Project: Hadoop Common > Issue Type: Bug > Components: fs > Affects Versions: 2.6.0 > Reporter: Sean Busbey > Assignee: Sean Busbey > Priority: Critical > > For the write-ahead-log, HBase writes to DFS from a single thread and sends > sync/flush/hflush from a configurable number of other threads (default 5). > FSDataOutputStream does not document anything about being thread safe, and it > is not thread safe for concurrent writes. > However, DFSOutputStream is thread safe for concurrent writes + syncs. When > it is the stream FSDataOutputStream wraps, the combination is threadsafe for > 1 writer and multiple syncs (the exact behavior HBase relies on). > When HDFS Transparent Encryption is turned on, CryptoOutputStream is inserted > between FSDataOutputStream and DFSOutputStream. It is proactively labeled as > not thread safe, and this composition is not thread safe for any operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)