subject:"hbase git commit\: HBASE\-19658 make the test testFlatteningToJumboCellChunkMap\(\) stable, by eliminating the possibility of third cell to be added while in\-memory\-flush is still in progress"

[01/50] hbase git commit: HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress [For

2018-02-06 Thread zhangduo

Repository: hbase
Updated Branches:
  refs/heads/HBASE-19397-branch-2 e6097eb3a -> d5fb97426 (forced update)


HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by 
eliminating the possibility of third cell to be added while in-memory-flush is 
still in progress


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/514eadbe
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/514eadbe
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/514eadbe

Branch: refs/heads/HBASE-19397-branch-2
Commit: 514eadbe9503c1162085890a86217a23457eb21d
Parents: c2e3d52
Author: anastas 
Authored: Mon Feb 5 11:29:10 2018 +0200
Committer: anastas 
Committed: Mon Feb 5 11:29:10 2018 +0200

--
 .../TestCompactingToCellFlatMapMemStore.java | 19 ---
 1 file changed, 12 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/514eadbe/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
--
diff --git 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
index 0036426..25265b3 100644
--- 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
+++ 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
@@ -754,9 +754,9 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 // set memstore to flat into CellChunkMap
 MemoryCompactionPolicy compactionType = MemoryCompactionPolicy.BASIC;
 
memstore.getConfiguration().set(CompactingMemStore.COMPACTING_MEMSTORE_TYPE_KEY,
-String.valueOf(compactionType));
-((MyCompactingMemStore)memstore).initiateType(compactionType, 
memstore.getConfiguration());
-
((CompactingMemStore)memstore).setIndexType(CompactingMemStore.IndexType.CHUNK_MAP);
+String.valueOf(compactionType));
+((MyCompactingMemStore) memstore).initiateType(compactionType, 
memstore.getConfiguration());
+((CompactingMemStore) 
memstore).setIndexType(CompactingMemStore.IndexType.CHUNK_MAP);
 
 int numOfCells = 1;
 char[] chars = new char[MemStoreLAB.CHUNK_SIZE_DEFAULT];
@@ -764,7 +764,7 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
   chars[i] = 'A';
 }
 String bigVal = new String(chars);
-String[] keys1 = { "A"};
+String[] keys1 = {"A"};
 
 // make one cell
 byte[] row = Bytes.toBytes(keys1[0]);
@@ -784,7 +784,7 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 assertEquals(totalCellsLen, regionServicesForStores.getMemStoreSize());
 assertEquals(totalHeapSize, ((CompactingMemStore) memstore).heapSize());
 
-((CompactingMemStore)memstore).flushInMemory(); // push keys to pipeline 
and flatten
+((CompactingMemStore) memstore).flushInMemory(); // push keys to pipeline 
and flatten
 while (((CompactingMemStore) memstore).isMemStoreFlushingInMemory()) {
   Threads.sleep(10);
 }
@@ -809,12 +809,17 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 
 memstore.clearSnapshot(snapshot.getId());
 
-String[] keys2 = { "C", "D", "E"};
+// Allocating two big cells (too big for being copied into a regular 
chunk).
+String[] keys2 = {"C", "D"};
 addRowsByKeys(memstore, keys2, val);
 while (((CompactingMemStore) memstore).isMemStoreFlushingInMemory()) {
   Threads.sleep(10);
 }
-totalHeapSize = 1 * oneCellOnCSLMHeapSize + MutableSegment.DEEP_OVERHEAD
+
+// The in-memory flush size is bigger than the size of a single cell,
+// but smaller than the size of two cells.
+// Therefore, the two created cells are flattened together.
+totalHeapSize = MutableSegment.DEEP_OVERHEAD
 + CellChunkImmutableSegment.DEEP_OVERHEAD_CCM
 + 2 * oneCellOnCCMHeapSize;
 assertEquals(totalHeapSize, ((CompactingMemStore) memstore).heapSize());

hbase git commit: HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress

2018-02-05 Thread anastasia

Repository: hbase
Updated Branches:
  refs/heads/branch-2 c2e3d5208 -> 514eadbe9


HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by 
eliminating the possibility of third cell to be added while in-memory-flush is 
still in progress


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/514eadbe
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/514eadbe
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/514eadbe

Branch: refs/heads/branch-2
Commit: 514eadbe9503c1162085890a86217a23457eb21d
Parents: c2e3d52
Author: anastas 
Authored: Mon Feb 5 11:29:10 2018 +0200
Committer: anastas 
Committed: Mon Feb 5 11:29:10 2018 +0200

--
 .../TestCompactingToCellFlatMapMemStore.java | 19 ---
 1 file changed, 12 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/514eadbe/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
--
diff --git 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
index 0036426..25265b3 100644
--- 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
+++ 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
@@ -754,9 +754,9 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 // set memstore to flat into CellChunkMap
 MemoryCompactionPolicy compactionType = MemoryCompactionPolicy.BASIC;
 
memstore.getConfiguration().set(CompactingMemStore.COMPACTING_MEMSTORE_TYPE_KEY,
-String.valueOf(compactionType));
-((MyCompactingMemStore)memstore).initiateType(compactionType, 
memstore.getConfiguration());
-
((CompactingMemStore)memstore).setIndexType(CompactingMemStore.IndexType.CHUNK_MAP);
+String.valueOf(compactionType));
+((MyCompactingMemStore) memstore).initiateType(compactionType, 
memstore.getConfiguration());
+((CompactingMemStore) 
memstore).setIndexType(CompactingMemStore.IndexType.CHUNK_MAP);
 
 int numOfCells = 1;
 char[] chars = new char[MemStoreLAB.CHUNK_SIZE_DEFAULT];
@@ -764,7 +764,7 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
   chars[i] = 'A';
 }
 String bigVal = new String(chars);
-String[] keys1 = { "A"};
+String[] keys1 = {"A"};
 
 // make one cell
 byte[] row = Bytes.toBytes(keys1[0]);
@@ -784,7 +784,7 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 assertEquals(totalCellsLen, regionServicesForStores.getMemStoreSize());
 assertEquals(totalHeapSize, ((CompactingMemStore) memstore).heapSize());
 
-((CompactingMemStore)memstore).flushInMemory(); // push keys to pipeline 
and flatten
+((CompactingMemStore) memstore).flushInMemory(); // push keys to pipeline 
and flatten
 while (((CompactingMemStore) memstore).isMemStoreFlushingInMemory()) {
   Threads.sleep(10);
 }
@@ -809,12 +809,17 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 
 memstore.clearSnapshot(snapshot.getId());
 
-String[] keys2 = { "C", "D", "E"};
+// Allocating two big cells (too big for being copied into a regular 
chunk).
+String[] keys2 = {"C", "D"};
 addRowsByKeys(memstore, keys2, val);
 while (((CompactingMemStore) memstore).isMemStoreFlushingInMemory()) {
   Threads.sleep(10);
 }
-totalHeapSize = 1 * oneCellOnCSLMHeapSize + MutableSegment.DEEP_OVERHEAD
+
+// The in-memory flush size is bigger than the size of a single cell,
+// but smaller than the size of two cells.
+// Therefore, the two created cells are flattened together.
+totalHeapSize = MutableSegment.DEEP_OVERHEAD
 + CellChunkImmutableSegment.DEEP_OVERHEAD_CCM
 + 2 * oneCellOnCCMHeapSize;
 assertEquals(totalHeapSize, ((CompactingMemStore) memstore).heapSize());

[17/28] hbase git commit: HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress

2018-02-04 Thread zhangduo

HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by 
eliminating the possibility of third cell to be added while in-memory-flush is 
still in progress


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/170ffbba
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/170ffbba
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/170ffbba

Branch: refs/heads/HBASE-19064
Commit: 170ffbba683217bdb30e5c99f0e728e0dc660d56
Parents: 14420e1
Author: anastas 
Authored: Sun Feb 4 14:59:10 2018 +0200
Committer: anastas 
Committed: Sun Feb 4 14:59:10 2018 +0200

--
 .../TestCompactingToCellFlatMapMemStore.java | 19 ---
 1 file changed, 12 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/170ffbba/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
--
diff --git 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
index 91a4b04..9b81c7f 100644
--- 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
+++ 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
@@ -752,9 +752,9 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 // set memstore to flat into CellChunkMap
 MemoryCompactionPolicy compactionType = MemoryCompactionPolicy.BASIC;
 
memstore.getConfiguration().set(CompactingMemStore.COMPACTING_MEMSTORE_TYPE_KEY,
-String.valueOf(compactionType));
-((MyCompactingMemStore)memstore).initiateType(compactionType, 
memstore.getConfiguration());
-
((CompactingMemStore)memstore).setIndexType(CompactingMemStore.IndexType.CHUNK_MAP);
+String.valueOf(compactionType));
+((MyCompactingMemStore) memstore).initiateType(compactionType, 
memstore.getConfiguration());
+((CompactingMemStore) 
memstore).setIndexType(CompactingMemStore.IndexType.CHUNK_MAP);
 
 int numOfCells = 1;
 char[] chars = new char[MemStoreLAB.CHUNK_SIZE_DEFAULT];
@@ -762,7 +762,7 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
   chars[i] = 'A';
 }
 String bigVal = new String(chars);
-String[] keys1 = { "A"};
+String[] keys1 = {"A"};
 
 // make one cell
 byte[] row = Bytes.toBytes(keys1[0]);
@@ -782,7 +782,7 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 assertEquals(totalCellsLen, regionServicesForStores.getMemStoreSize());
 assertEquals(totalHeapSize, ((CompactingMemStore) memstore).heapSize());
 
-((CompactingMemStore)memstore).flushInMemory(); // push keys to pipeline 
and flatten
+((CompactingMemStore) memstore).flushInMemory(); // push keys to pipeline 
and flatten
 while (((CompactingMemStore) memstore).isMemStoreFlushingInMemory()) {
   Threads.sleep(10);
 }
@@ -807,12 +807,17 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 
 memstore.clearSnapshot(snapshot.getId());
 
-String[] keys2 = { "C", "D", "E"};
+// Allocating two big cells (too big for being copied into a regular 
chunk).
+String[] keys2 = {"C", "D"};
 addRowsByKeys(memstore, keys2, val);
 while (((CompactingMemStore) memstore).isMemStoreFlushingInMemory()) {
   Threads.sleep(10);
 }
-totalHeapSize = 1 * oneCellOnCSLMHeapSize + MutableSegment.DEEP_OVERHEAD
+
+// The in-memory flush size is bigger than the size of a single cell,
+// but smaller than the size of two cells.
+// Therefore, the two created cells are flattened together.
+totalHeapSize = MutableSegment.DEEP_OVERHEAD
 + CellChunkImmutableSegment.DEEP_OVERHEAD_CCM
 + 2 * oneCellOnCCMHeapSize;
 assertEquals(totalHeapSize, ((CompactingMemStore) memstore).heapSize());

hbase git commit: HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress

2018-02-04 Thread anastasia

Repository: hbase
Updated Branches:
  refs/heads/master 14420e1b4 -> 170ffbba6


HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by 
eliminating the possibility of third cell to be added while in-memory-flush is 
still in progress


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/170ffbba
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/170ffbba
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/170ffbba

Branch: refs/heads/master
Commit: 170ffbba683217bdb30e5c99f0e728e0dc660d56
Parents: 14420e1
Author: anastas 
Authored: Sun Feb 4 14:59:10 2018 +0200
Committer: anastas 
Committed: Sun Feb 4 14:59:10 2018 +0200

--
 .../TestCompactingToCellFlatMapMemStore.java | 19 ---
 1 file changed, 12 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/170ffbba/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
--
diff --git 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
index 91a4b04..9b81c7f 100644
--- 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
+++ 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
@@ -752,9 +752,9 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 // set memstore to flat into CellChunkMap
 MemoryCompactionPolicy compactionType = MemoryCompactionPolicy.BASIC;
 
memstore.getConfiguration().set(CompactingMemStore.COMPACTING_MEMSTORE_TYPE_KEY,
-String.valueOf(compactionType));
-((MyCompactingMemStore)memstore).initiateType(compactionType, 
memstore.getConfiguration());
-
((CompactingMemStore)memstore).setIndexType(CompactingMemStore.IndexType.CHUNK_MAP);
+String.valueOf(compactionType));
+((MyCompactingMemStore) memstore).initiateType(compactionType, 
memstore.getConfiguration());
+((CompactingMemStore) 
memstore).setIndexType(CompactingMemStore.IndexType.CHUNK_MAP);
 
 int numOfCells = 1;
 char[] chars = new char[MemStoreLAB.CHUNK_SIZE_DEFAULT];
@@ -762,7 +762,7 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
   chars[i] = 'A';
 }
 String bigVal = new String(chars);
-String[] keys1 = { "A"};
+String[] keys1 = {"A"};
 
 // make one cell
 byte[] row = Bytes.toBytes(keys1[0]);
@@ -782,7 +782,7 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 assertEquals(totalCellsLen, regionServicesForStores.getMemStoreSize());
 assertEquals(totalHeapSize, ((CompactingMemStore) memstore).heapSize());
 
-((CompactingMemStore)memstore).flushInMemory(); // push keys to pipeline 
and flatten
+((CompactingMemStore) memstore).flushInMemory(); // push keys to pipeline 
and flatten
 while (((CompactingMemStore) memstore).isMemStoreFlushingInMemory()) {
   Threads.sleep(10);
 }
@@ -807,12 +807,17 @@ public class TestCompactingToCellFlatMapMemStore extends 
TestCompactingMemStore
 
 memstore.clearSnapshot(snapshot.getId());
 
-String[] keys2 = { "C", "D", "E"};
+// Allocating two big cells (too big for being copied into a regular 
chunk).
+String[] keys2 = {"C", "D"};
 addRowsByKeys(memstore, keys2, val);
 while (((CompactingMemStore) memstore).isMemStoreFlushingInMemory()) {
   Threads.sleep(10);
 }
-totalHeapSize = 1 * oneCellOnCSLMHeapSize + MutableSegment.DEEP_OVERHEAD
+
+// The in-memory flush size is bigger than the size of a single cell,
+// but smaller than the size of two cells.
+// Therefore, the two created cells are flattened together.
+totalHeapSize = MutableSegment.DEEP_OVERHEAD
 + CellChunkImmutableSegment.DEEP_OVERHEAD_CCM
 + 2 * oneCellOnCCMHeapSize;
 assertEquals(totalHeapSize, ((CompactingMemStore) memstore).heapSize());

[01/50] hbase git commit: HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress [For

hbase git commit: HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress

[17/28] hbase git commit: HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress

hbase git commit: HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress

4 matches

Site Navigation

Mail list logo

Footer information