Gopal V created HADOOP-11029:
--------------------------------
Summary: LocalFS Statistics performs thread local call per byte
written
Key: HADOOP-11029
URL: https://issues.apache.org/jira/browse/HADOOP-11029
Project: Hadoop Common
Issue Type: Bug
Reporter: Gopal V
Attachments: local-fs-locking.png
This code is there in the hot-path of IFile writer via RawLocalFileSystem.
!local-fs-locking.png!
>From a preliminary glance, the lock prefix calls are coming from a
>threadlocal.get() within FileSystem.Statistics
{code}
/**
* Get or create the thread-local data associated with the current thread.
*/
private StatisticsData getThreadData() {
StatisticsData data = threadData.get();
if (data == null) {
data = new StatisticsData(
new WeakReference<Thread>(Thread.currentThread()));
threadData.set(data);
synchronized(this) {
if (allData == null) {
allData = new LinkedList<StatisticsData>();
}
allData.add(data);
}
}
return data;
}
/**
* Increment the bytes read in the statistics
* @param newBytes the additional bytes read
*/
public void incrementBytesRead(long newBytes) {
getThreadData().bytesRead += newBytes;
}
{code}
This is incredibly inefficient when used from FSDataOutputStream
{code}
public void write(int b) throws IOException {
out.write(b);
position++;
if (statistics != null) {
statistics.incrementBytesWritten(1);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
