[jira] [Commented] (HBASE-15359) Simplifying Segment hierarchy
[ https://issues.apache.org/jira/browse/HBASE-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175461#comment-15175461 ] Hudson commented on HBASE-15359: FAILURE: Integrated in HBase-Trunk_matrix #750 (See [https://builds.apache.org/job/HBase-Trunk_matrix/750/]) HBASE-15359 Simplifying segment hierarchy (stack: rev dc449436662d8520a02628c438d0da665ab6da2e) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MutableSegment.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableSegmentAdapter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableSegment.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SegmentFactory.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MutableCellSetSegment.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AbstractMemStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MutableCellSetSegmentScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SegmentScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Segment.java > Simplifying Segment hierarchy > - > > Key: HBASE-15359 > URL: https://issues.apache.org/jira/browse/HBASE-15359 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 2.0.0 > > Attachments: HBASE-14918-FIX-SEGMENT.patch, HBASE-15359-V01.patch > > > Now that it is clear that no memstore segment will be implemented as an > HFIle, and that all segments store their data in some representation of > CellSet (skip-list or flat), the segment hierarchy can be much simplified. > The attached patch includes only 3 classes in the hierarchy: > Segment - comprises most of the state and implementation > MutableSegment - extends API with add and rollback functionality > ImmutableSegment - extends API with key-value scanner for snapshot > SegmentScanner is the scanner for all types of segments. > In addition, the option to rollback immutable segment in the memstore is > disabled. > This code would allow us to make progress independently in the compaction > subtask (HBASE-14920) and the flat index representation subtask > (HBASE-14921). It also means that the new immutable segment can reuse the > existing SegmentScanner, instead of implementing a new scanner. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15359) Simplifying Segment hierarchy
[
https://issues.apache.org/jira/browse/HBASE-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173730#comment-15173730
]
Hadoop QA commented on HBASE-15359:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 58s
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 49s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 112m 43s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_72.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 115m 9s
{color} | {color:green} hbase-server in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 278m 33s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_72 Failed junit tests |
hadoop.hbase.security.token.TestGenerateDelegationToken |
| | hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-03-01 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12790687/HBASE-15359-V01.patch
|
|
[jira] [Commented] (HBASE-15359) Simplifying Segment hierarchy
[ https://issues.apache.org/jira/browse/HBASE-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173428#comment-15173428 ] Eshcar Hillel commented on HBASE-15359: --- In the umbrella jira [~stack] says bq. We want CellBlocks right? Not HFile blocks? and bq. The HFile instance (with all its great possibilities) seams like something too complex for this simple task of storing the data flat. Agree. >From this I conjecture segments will not be implemented as an HFile instance. rollback is declared in Store.java and MemStore.java and implemented in HStore ans AbstractMemStore. In the meanwhile it is also supported by MutableSegment, but can be removed once it is cleaned everywhere else. > Simplifying Segment hierarchy > - > > Key: HBASE-15359 > URL: https://issues.apache.org/jira/browse/HBASE-15359 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 0.98.18 > > Attachments: HBASE-14918-FIX-SEGMENT.patch > > > Now that it is clear that no memstore segment will be implemented as an > HFIle, and that all segments store their data in some representation of > CellSet (skip-list or flat), the segment hierarchy can be much simplified. > The attached patch includes only 3 classes in the hierarchy: > Segment - comprises most of the state and implementation > MutableSegment - extends API with add and rollback functionality > ImmutableSegment - extends API with key-value scanner for snapshot > SegmentScanner is the scanner for all types of segments. > In addition, the option to rollback immutable segment in the memstore is > disabled. > This code would allow us to make progress independently in the compaction > subtask (HBASE-14920) and the flat index representation subtask > (HBASE-14921). It also means that the new immutable segment can reuse the > existing SegmentScanner, instead of implementing a new scanner. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15359) Simplifying Segment hierarchy
[ https://issues.apache.org/jira/browse/HBASE-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172329#comment-15172329 ] stack commented on HBASE-15359: --- bq. Now that it is clear that no memstore segment will be implemented as an HFIle Where does the above assertion come from [~eshcar]? Thanks for doing this cleanup if above is true. Why rollback? We don't need it anymore since HBASE-12751 and reorder of write pipeline so edit is in sync'd and in WAL before we add to memstore. Otherwise, patch looks great. > Simplifying Segment hierarchy > - > > Key: HBASE-15359 > URL: https://issues.apache.org/jira/browse/HBASE-15359 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 0.98.18 > > Attachments: HBASE-14918-FIX-SEGMENT.patch > > > Now that it is clear that no memstore segment will be implemented as an > HFIle, and that all segments store their data in some representation of > CellSet (skip-list or flat), the segment hierarchy can be much simplified. > The attached patch includes only 3 classes in the hierarchy: > Segment - comprises most of the state and implementation > MutableSegment - extends API with add and rollback functionality > ImmutableSegment - extends API with key-value scanner for snapshot > SegmentScanner is the scanner for all types of segments. > In addition, the option to rollback immutable segment in the memstore is > disabled. > This code would allow us to make progress independently in the compaction > subtask (HBASE-14920) and the flat index representation subtask > (HBASE-14921). It also means that the new immutable segment can reuse the > existing SegmentScanner, instead of implementing a new scanner. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15359) Simplifying Segment hierarchy
[
https://issues.apache.org/jira/browse/HBASE-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172086#comment-15172086
]
Hadoop QA commented on HBASE-15359:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 4s
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 21s
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-server
(total was 2, now 3). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 1s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 49s
{color} | {color:green} hbase-server in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 168m 40s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_72 Failed junit tests | hadoop.hbase.ipc.TestSimpleRpcScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-02-29 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12790473/HBASE-14918-FIX-SEGMENT.patch
|
| JIRA Issue | HBASE-15359
