[
https://issues.apache.org/jira/browse/HBASE-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137613#comment-13137613
]
Ted Yu commented on HBASE-4528:
-------------------------------
Found the following in output file for a failed test:
{code}
2011-10-27 14:25:15,878 DEBUG
[10.246.204.28,52533,1319750687794.splitLogManagerTimeoutMonitor]
master.SplitLogManager$TimeoutMonitor(829): total tasks = 1 unassigned = 1
2011-10-27 14:25:16,126 INFO
[SplitLogWorker-10.246.204.28,52535,1319750687853]
regionserver.SplitLogWorker(308): worker 10.246.204.28,52535,1319750687853 done
with task
/hbase/splitlog/hdfs%3A%2F%2Flocalhost%3A52522%2Fuser%2Fzhihyu%2F.logs%2F10.246.204.28%2C52535%2C1319750687853%2F10.246.204.28%252C52535%252C1319750687853.1319750690240
in 5049ms
2011-10-27 14:25:16,126 ERROR
[SplitLogWorker-10.246.204.28,52535,1319750687853]
regionserver.SplitLogWorker(169): unexpected error
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeThreads(DFSClient.java:3648)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3691)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3626)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:61)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:86)
at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:966)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.close(SequenceFileLogWriter.java:177)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFileToTemp(HLogSplitter.java:458)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFileToTemp(HLogSplitter.java:352)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:113)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.grabTask(SplitLogWorker.java:266)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.taskLoop(SplitLogWorker.java:197)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.run(SplitLogWorker.java:165)
at java.lang.Thread.run(Thread.java:680)
2011-10-27 14:25:16,126 INFO
[SplitLogWorker-10.246.204.28,52535,1319750687853]
regionserver.SplitLogWorker(171): SplitLogWorker
10.246.204.28,52535,1319750687853 exiting
2011-10-27 14:25:16,878 DEBUG
[10.246.204.28,52533,1319750687794.splitLogManagerTimeoutMonitor]
master.SplitLogManager$TimeoutMonitor(829): total tasks = 1 unassigned = 1
{code}
I don't think NPE out of DFSClient.java for closing is critical.
The following patch allowed me to loop through TestDistributedLogSplitting on
MacBook 9 times without hitting test failure.
{code}
Index: src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
===================================================================
--- src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
(revision 1190003)
+++ src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
(working copy)
@@ -455,7 +455,11 @@
}
n++;
WriterAndPath wap = (WriterAndPath)o;
- wap.w.close();
+ try {
+ wap.w.close();
+ } catch (Exception e) {
+ LOG.debug("splitLogFileToTemp closing SequenceFileLogWriter", e);
+ }
LOG.debug("Closed " + wap.p);
}
String msg = ("processed " + editsCount + " edits across " + n + "
regions" +
{code}
> The put operation can release the rowlock before sync-ing the Hlog
> ------------------------------------------------------------------
>
> Key: HBASE-4528
> URL: https://issues.apache.org/jira/browse/HBASE-4528
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: HBASE-4528-Trunk-FINAL.patch, appendNoSync5.txt,
> appendNoSyncPut1.txt, appendNoSyncPut2.txt, appendNoSyncPut3.txt,
> appendNoSyncPut4.txt, appendNoSyncPut5.txt, appendNoSyncPut6.txt,
> appendNoSyncPut7.txt, appendNoSyncPut8.txt
>
>
> This allows for better throughput when there are hot rows. A single row
> update improves from 100 puts/sec/server to 5000 puts/sec/server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
