[
https://issues.apache.org/jira/browse/HBASE-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17387925#comment-17387925
]
chenglei commented on HBASE-26142:
----------------------------------
[~anoop.hbase], the default value of
{{hbase.hregion.memstore.mslab.indexchunksize.percent}} is 0.1, that is to say
, the byte size of the index chunk pool is
{hbase.regionserver.global.memstore.size}*0.1 by default
> NullPointerException when set
> 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-26142
> URL: https://issues.apache.org/jira/browse/HBASE-26142
> Project: HBase
> Issue Type: Bug
> Affects Versions: 3.0.0-alpha-1, 2.4.0
> Reporter: chenglei
> Assignee: chenglei
> Priority: Critical
>
> By default, we use {{DefaultMemStore}} , which use no {{IndexChunk}} and
> {{ChunkCreator.indexChunksPool}} is useless, so we could set
> {{hbase.hregion.memstore.mslab.indexchunksize.percent}} to 0 to save memory
> space,
> But after running a while, the {{RegionServer}} throws
> {{NullPointerException}} and abort:
> {code:java}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.ChunkCreator$MemStoreChunkPool.access$900(ChunkCreator.java:310)
> at
> org.apache.hadoop.hbase.regionserver.ChunkCreator.putbackChunks(ChunkCreator.java:608)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.recycleChunks(MemStoreLABImpl.java:297)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.close(MemStoreLABImpl.java:268)
> at
> org.apache.hadoop.hbase.regionserver.Segment.close(Segment.java:149)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.clearSnapshot(AbstractMemStore.java:251)
> at
> org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1244)
> at
> org.apache.hadoop.hbase.regionserver.HStore.access$700(HStore.java:137)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2461)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2963)
> {code}
> The problem is caused by line 608 in {{ChunkCreator.putbackChunks}}
> :{{Chunk.isIndexChunk}} return true for{{DataChunk}} , and
> {{ChunkCreator.indexChunksPool}} is null:
> {code:java}
> 594 synchronized void putbackChunks(Set<Integer> chunks) {
> 595 // if there is no pool just try to clear the chunkIdMap in case there
> is something
> 596 if (dataChunksPool == null && indexChunksPool == null) {
> 597 this.removeChunks(chunks);
> 598 return;
> 599 }
> 600
> 601 // if there is a pool, go over all chunk IDs that came back, the chunks
> may be from pool or not
> 602 for (int chunkID : chunks) {
> 603 // translate chunk ID to chunk, if chunk initially wasn't in pool
> 604 // this translation will (most likely) return null
> 605 Chunk chunk = ChunkCreator.this.getChunk(chunkID);
> 606 if (chunk != null) {
> 607 if (chunk.isFromPool() && chunk.isIndexChunk()) {
> 608 indexChunksPool.putbackChunks(chunk);
> {code}
> For {{DataChunk}} , {{Chunk.isIndexChunk}} return true because
> {{Chunk.isIndexChunk}} determines the type of {{chunk}} based on
> {{Chunk.size}}
> {code:java}
> boolean isIndexChunk() {
> return size ==
> ChunkCreator.getInstance().getChunkSize(ChunkCreator.ChunkType.INDEX_CHUNK);
> }
> {code}
> and {{ChunkCreator.getChunkSize}} incorrectly return {{DataChunk}} size when
> {{ChunkCreator.indexChunksPool}} is null:
> {code:java}
> int getChunkSize(ChunkType chunkType) {
> switch (chunkType) {
> case INDEX_CHUNK:
> if (indexChunksPool != null) {
> return indexChunksPool.getChunkSize();
> }
> case DATA_CHUNK:
> if (dataChunksPool != null) {
> return dataChunksPool.getChunkSize();
> } else { // When pools are empty
> return chunkSize;
> }
> default:
> throw new IllegalArgumentException(
> "chunkType must either be INDEX_CHUNK or DATA_CHUNK");
> }
> }
> {code}
> In my opinion, in addition to erroneous implementation of
> {{ChunkCreator.getChunkSize}}, we would better not determine the type of
> {{Chunk}} based on {{Chunk.size}}, because
> {{hbase.hregion.memstore.mslab.indexchunksize.percent}} is set by user and
> the size of {{IndexChunk}} and {{DataChunk}} could be the same.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
