[
https://issues.apache.org/jira/browse/HADOOP-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359054#comment-14359054
]
Sean Busbey commented on HADOOP-11708:
--------------------------------------
{quote}
The runup to a release is not the time to change something so foundational.
{quote}
FWIW, I just picked the first unreleased versions on the jira. I'm not working
to meet any particular release schedule.
{quote}
1. HADOOP-9361 skipped on the output stream stuff in the rush to get things
into hadoop2.5, and for all its stuff is very vague about concurrency
guarantees. As it extends java.io.OutputStream, the real concurrency semantics
should be defined there. And, looking at that openjdk code, it appears to be
"no guarantees". HBase have some expectations that may already be invalid.
{quote}
I agree. DFSOutputStream is the first byte stream implementation I've seen that
tries to be thread safe.
{quote}
3. If you are proposing we define those concurrency semantics more strictly, in
the filesystem specification and the code that would be great: someone did need
to sit down and define the APIs as based on HDFS behaviour. This is not
something we can rush into in the last minute though. I also worry about the
proposed culling of +10 sync blocks, especially on the basis that "they appear
to be unneeded". That's the kind of question that needs to be answered by
looking at the git/svn history of those lines & correlating them with past
JIRAs, before doing what could potentially cause problems. And, at write()
time, that means data-loss/corruption problems beyond just encrypted data.
{quote}
That tracing is what I spent last night doing. AFAICT, most of those block come
without comment in commit or jira about why. They appear to be just matching
what was already present. The earliest ones I ran into problems tracing because
of svn merge commits in ~2009. The lack of thread safety when attempting to
write from FSDataOutputStream is also a big flag.
By "they appear to be unneeded" I mean I have some worked through
rationalizations (in the absence of written comments from their authors) about
what they're trying to protect and why that either isn't necessary or isn't
done correctly. I can get these polished up.
{quote}
Changing HBase WAL to work with unsynced streams or fixing CryptoOutputStream
to implement same expectations of HDFS are much lower risk.
{quote}
+1, working on this in HBASE-13221
> CryptoOutputStream synchronization differences from DFSOutputStream break
> HBase
> -------------------------------------------------------------------------------
>
> Key: HADOOP-11708
> URL: https://issues.apache.org/jira/browse/HADOOP-11708
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.6.0
> Reporter: Sean Busbey
> Assignee: Sean Busbey
> Priority: Critical
>
> For the write-ahead-log, HBase writes to DFS from a single thread and sends
> sync/flush/hflush from a configurable number of other threads (default 5).
> FSDataOutputStream does not document anything about being thread safe, and it
> is not thread safe for concurrent writes.
> However, DFSOutputStream is thread safe for concurrent writes + syncs. When
> it is the stream FSDataOutputStream wraps, the combination is threadsafe for
> 1 writer and multiple syncs (the exact behavior HBase relies on).
> When HDFS Transparent Encryption is turned on, CryptoOutputStream is inserted
> between FSDataOutputStream and DFSOutputStream. It is proactively labeled as
> not thread safe, and this composition is not thread safe for any operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
