[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613643#comment-14613643
]
Hudson commented on HBASE-13970:
FAILURE: Integrated in HBase-TRUNK #6626 (See
[https://builds.apache.org/job/HBase-TRUNK/6626/])
HBASE-13970 NPE during compaction in trunk (zhangduo: rev
28a035000f3cbf7c2b242d6dd9ba3bf84f4e2503)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event:
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613921#comment-14613921
]
Hudson commented on HBASE-13970:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #999 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/999/])
HBASE-13970 NPE during compaction in trunk (zhangduo: rev
ebb476ba87ba3ce5894ed1ad9350c5c89b4e0f6c)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613802#comment-14613802
]
Hudson commented on HBASE-13970:
FAILURE: Integrated in HBase-1.3 #33 (See
[https://builds.apache.org/job/HBase-1.3/33/])
HBASE-13970 NPE during compaction in trunk (zhangduo: rev
9aeda36613e97e74507b223c3b894a744048fff1)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event:
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613843#comment-14613843
]
Hudson commented on HBASE-13970:
FAILURE: Integrated in HBase-0.98 #1046 (See
[https://builds.apache.org/job/HBase-0.98/1046/])
HBASE-13970 NPE during compaction in trunk (zhangduo: rev
ebb476ba87ba3ce5894ed1ad9350c5c89b4e0f6c)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event:
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613703#comment-14613703
]
Hudson commented on HBASE-13970:
SUCCESS: Integrated in HBase-1.1 #572 (See
[https://builds.apache.org/job/HBase-1.1/572/])
HBASE-13970 NPE during compaction in trunk (zhangduo: rev
426a5239c89b939399665b3881c240811279e50c)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event:
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613765#comment-14613765
]
Hudson commented on HBASE-13970:
SUCCESS: Integrated in HBase-1.2-IT #36 (See
[https://builds.apache.org/job/HBase-1.2-IT/36/])
HBASE-13970 NPE during compaction in trunk (zhangduo: rev
42445754057e442ecd9c2b0117b2e3560742ef06)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event:
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613774#comment-14613774
]
Hudson commented on HBASE-13970:
SUCCESS: Integrated in HBase-1.3-IT #18 (See
[https://builds.apache.org/job/HBase-1.3-IT/18/])
HBASE-13970 NPE during compaction in trunk (zhangduo: rev
9aeda36613e97e74507b223c3b894a744048fff1)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event:
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613786#comment-14613786
]
Hudson commented on HBASE-13970:
FAILURE: Integrated in HBase-1.2 #49 (See
[https://builds.apache.org/job/HBase-1.2/49/])
HBASE-13970 NPE during compaction in trunk (zhangduo: rev
42445754057e442ecd9c2b0117b2e3560742ef06)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event:
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611805#comment-14611805
]
ramkrishna.s.vasudevan commented on HBASE-13970:
I have lost my server for some time. I think you can commit it. It may take
some more time for me to test. So +1.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.0.2, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611801#comment-14611801
]
Anoop Sam John commented on HBASE-13970:
[~Apache9] You have 2 +1s on this Jira.. It is ready for commit.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.0.2, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612845#comment-14612845
]
ramkrishna.s.vasudevan commented on HBASE-13970:
I verified in cluster. this issue is no longer there. Thanks [~Apache9].
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: Duo Zhang
Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609386#comment-14609386
]
Duo Zhang commented on HBASE-13970:
---
[~ram_krish] Any progress? Thanks.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603032#comment-14603032
]
ramkrishna.s.vasudevan commented on HBASE-13970:
I can test this but will take some time. I think this should be fine. +1
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602704#comment-14602704
]
Duo Zhang commented on HBASE-13970:
---
No, counter can not be same for 2 threads. For a given value of counter, only
one CAS operation can succeed. The failed thread will try another round and the
value of counter will be changed.
This is the getAndIncrement methods in AtomicInteger. The only difference is
how to calculate the next value.
{code}
/**
* Atomically increments by one the current value.
*
* @return the previous value
*/
public final int getAndIncrement() {
for (;;) {
int current = get();
int next = current + 1;
if (compareAndSet(current, next))
return current;
}
}
{code}
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602778#comment-14602778
]
Anoop Sam John commented on HBASE-13970:
Oh ya we have a loop and the get is inside the loop.. fine np.. +1.. We can
commit after Ram test the scenario.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600976#comment-14600976
]
ramkrishna.s.vasudevan commented on HBASE-13970:
Note that when I tried to load some more data and when a compaction was
triggered no NPE happened.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601010#comment-14601010
]
ramkrishna.s.vasudevan commented on HBASE-13970:
The reason here is that there are 2 compactions getting triggered from the
CompactionSplitThread on a region that is newly split. One may be due to the
split that happened?
So when both the compaction goes parallely then the
PressureAwareCompactionController is started by the Compactor thread
{code}
@Override
public void start(String compactionName) {
activeCompactions.put(compactionName, new ActiveCompaction());
}
{code}
Now by the time the second compaction is in progress the first compaction
completes and does a finish
{code}
@Override
public void finish(String compactionName) {
ActiveCompaction compaction = activeCompactions.remove(compactionName);
long elapsedTime = Math.max(1, EnvironmentEdgeManager.currentTime() -
compaction.startTime);
.
{code}
The compactionName is going to be common here because
{code}
String compactionName =
store.getRegionInfo().getRegionNameAsString() + # +
store.getFamily().getNameAsString();
{code}
When the second compaction comes for completion the compaction has already been
removed and hence we get an NPE.
Logs
One compaction started with 4 files
{code}
2015-06-25 22:07:49,135 INFO
[regionserver/stobdtserver3/10.224.54.70:16040-shortCompactions-1435250269126]
regionserver.HRegion: Starting compaction on info in region
TestTable,283887,1435250266343.5eb54f001fd85035ab448f44d049ab84.
2015-06-25 22:07:49,135 INFO
[regionserver/stobdtserver3/10.224.54.70:16040-shortCompactions-1435250269126]
regionserver.HStore: Starting compaction of 4 file(s) in info of
TestTable,283887,1435250266343.5eb54f001fd85035ab448f44d049ab84.
into
tmpdir=hdfs://stobdtserver3:9010/hbase/data/default/TestTable/5eb54f001fd85035ab448f44d049ab84/.tmp,
totalSize=285.6 M
2015-06-25 22:07:49,165 INFO
[regionserver/stobdtserver3/10.224.54.70:16040-shortCompactions-1435250269126]
hfile.CacheConfig:
blockCache=org.apache.hadoop.hbase.io.hfile.CombinedBlockCache@71f1ce16,
cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
prefetchOnOpen=false
2015-06-25 22:07:49,954 INFO
[regionserver/stobdtserver3/10.224.54.70:16040.logRoller] wal.FSHLog: Rolled
WAL
/hbase/WALs/stobdtserver3,16040,1435250244539/stobdtserver3%2C16040%2C1435250244539.default.1435250268365
with entries=90, filesize=124.14 MB; new WAL
/hbase/WALs/stobdtserver3,16040,1435250244539/stobdtserver3%2C16040%2C1435250244539.default.1435250269933
20
{code}
Another compaction has been started with 3 files
{code}
2015-06-25 22:07:53,405 INFO
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435250245998]
regionserver.HRegion: Starting compaction on info in region
TestTable,283887,1435250266343.5eb54f001fd85035ab448f44d049ab84.
2015-06-25 22:07:53,406 INFO
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435250245998]
regionserver.HStore: Starting compaction of 3 file(s) in info of
TestTable,283887,1435250266343.5eb54f001fd85035ab448f44d049ab84.
into
tmpdir=hdfs://stobdtserver3:9010/hbase/data/default/TestTable/5eb54f001fd85035ab448f44d049ab84/.tmp,
totalSize=343.4 M
2015-06-25 22:07:53,411 INFO
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435250245998]
hfile.CacheConfig:
blockCache=org.apache.hadoop.hbase.io.hfile.CombinedBlockCache@71f1ce16,
cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
prefetchOnOpen=false
2015-06-25 22:07:54,211 INFO [MemStoreFlusher.1] regionserver.HRegion:
Flushing 1/1 column families, memstore=128.23 MB
2015-06-25 22:07:54,639 INFO
[regionserver/stobdtserver3/10.224.54.70:16040.logRoller] wal.FSHLog: Rolled
WAL
/hbase/WALs/stobdtserver3,16040,1435250244539/stobdtserver3%2C16040%2C1435250244539.default.1435250273034
with entries=90, filesize=
{code}
{code}
2015-06-25 22:08:09,446 INFO
[regionserver/stobdtserver3/10.224.54.70:16040-shortCompactions-1435250269126]
compactions.PressureAwareCompactionThroughputController:
TestTable,283887,1435250266343.5eb54f001fd85035ab448f44d049ab84.#info
average throughput is 19.80 MB/sec, slept 30 time(s) and total slept time is
27694 ms. 0 active compactions remaining, total limit is 10.00 MB/sec
2015-06-25 22:08:09,520 INFO
[regionserver/stobdtserver3/10.224.54.70:16040-shortCompactions-1435250269126]
regionserver.HStore: Completed compaction of 4 (all) file(s) in info of
TestTable,283887,1435250266343.5eb54f001fd85035ab448f44d049ab84.
into
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601376#comment-14601376
]
Anoop Sam John commented on HBASE-13970:
Ya this can work.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601295#comment-14601295
]
Duo Zhang commented on HBASE-13970:
---
Oh, this should be a bug on all branches which contains HBASE-8329. I used to
assume that compactions can not be executed parallel on the same store, so the
compationName only contains regionName and storeName. I think we could add a
counter in the name to avoid conflict.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601451#comment-14601451
]
ramkrishna.s.vasudevan commented on HBASE-13970:
I will test this out. But this should work. Can we create this per counter per
store?
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601491#comment-14601491
]
Anoop Sam John commented on HBASE-13970:
Per store counter means we will have to keep it in Map or so which is adding
overhead. The integer even if it overflows over the run, is fine(?) Will go to
-ve integers. We need only unique Strings. So this should be ok IMO.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601533#comment-14601533
]
ramkrishna.s.vasudevan commented on HBASE-13970:
What I thought was just do store.incrementandGetCounter()? That will return
that atomic counter. Did not think as a map. Main concern was that overflow
part only. (negative is fine then ok).
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is the reason behind it.
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601661#comment-14601661
]
Hadoop QA commented on HBASE-13970:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12741873/HBASE-13970.patch
against master branch at commit edef3d64bce41fffbc5649ffa19b2cf80ce28d7a.
ATTACHMENT ID: 12741873
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14571//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14571//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14571//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14571//console
This message is automatically generated.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602332#comment-14602332
]
Hadoop QA commented on HBASE-13970:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12741997/HBASE-13970-v1.patch
against master branch at commit 70f0ca3c6d485d504d49652801fd841de2086157.
ATTACHMENT ID: 12741997
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransaction.testWholesomeMerge(TestRegionMergeTransaction.java:229)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14581//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14581//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14581//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14581//console
This message is automatically generated.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
[jira] [Commented] (HBASE-13970) NPE during compaction in trunk
[
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602376#comment-14602376
]
Anoop Sam John commented on HBASE-13970:
return store.getRegionInfo().getRegionNameAsString() + #
241 + store.getFamily().getNameAsString() + # + counter;
You have to use 'next' instead of counter no? 'counter' can get same for 2
threads.
NPE during compaction in trunk
--
Key: HBASE-13970
URL: https://issues.apache.org/jira/browse/HBASE-13970
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2
Attachments: HBASE-13970-v1.patch, HBASE-13970.patch
Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure
all data is flushed out to disk. When the first compaction is triggered we
get an NPE and this is very easy to reproduce
{code}
015-06-25 21:33:46,041 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,051 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
2015-06-25 21:33:46,159 ERROR
[regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M),
priority=3, time=7536968291719985
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
at
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
at
org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-06-25 21:33:46,745 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M,
hasBloomFilter=true, into tmp file
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
2015-06-25 21:33:46,772 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HStore: Added
hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
entries=68116, sequenceid=1534, filesize=68.7 M
2015-06-25 21:33:46,773 INFO
[rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1]
regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344,
currentsize=0 B/0 for region
TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
in 723ms, sequenceid=1534, compaction requested=true
2015-06-25 21:33:46,780 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/reached/TestTable
2015-06-25 21:33:46,790 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received created
event:/hbase/flush-table-proc/abort/TestTable
2015-06-25 21:33:46,791 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
2015-06-25 21:33:46,803 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure start children changed
event: /hbase/flush-table-proc/acquired
2015-06-25 21:33:46,818 INFO [main-EventThread]
procedure.ZKProcedureMemberRpcs: Received procedure abort children changed
event: /hbase/flush-table-proc/abort
{code}
Will check this on what is
