[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126991#comment-16126991 ] Chia-Ping Tsai commented on HBASE-18546: bq. Does we really need setting timestamp for append/increment at client? Opening this issue is due to behavior inconsistency. I noticed that we will use the TS passed by client for Append (if we don't find any old cells), but we always ignore the TS for Increment. No documents discuss about such differences. After reading HBASE-17112, IMO, It is straightforward to regard Append/Increment as PUT if no previous cells are found. In contrast, we should use the max(now, old ts+1). > Append/Increment a cell with custom timestamp > - > > Key: HBASE-18546 > URL: https://issues.apache.org/jira/browse/HBASE-18546 > Project: HBase > Issue Type: New Feature > Components: API, Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Labels: incompatibleChange > Fix For: 2.0.0 > > Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch > > > We don't pass the custom timestamp for Increment, and the increment's > timestamp always be rewrite. Hence, user can't increment a cell with custom > timestamp. > {code:title=ProtobufUtil.java} > if (values != null && values.size() > 0) { > for (Cell cell: values) { > valueBuilder.clear(); > valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap( > cell.getQualifierArray(), cell.getQualifierOffset(), > cell.getQualifierLength())); > valueBuilder.setValue(UnsafeByteOperations.unsafeWrap( > cell.getValueArray(), cell.getValueOffset(), > cell.getValueLength())); > if (cell.getTagsLength() > 0) { > > valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(), > cell.getTagsOffset(), cell.getTagsLength())); > } > columnBuilder.addQualifierValue(valueBuilder.build()); > } > } > {code} > In contrast to Increment, user can append the cell with custom timestamp. It > would be better that make their behavior consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126758#comment-16126758 ] Phil Yang commented on HBASE-18546: --- IIRC, for append/increment we will set a ts which must be larger than the previous one in WAL. Does we really need setting timestamp for append/increment at client? > Append/Increment a cell with custom timestamp > - > > Key: HBASE-18546 > URL: https://issues.apache.org/jira/browse/HBASE-18546 > Project: HBase > Issue Type: New Feature > Components: API, Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Labels: incompatibleChange > Fix For: 2.0.0 > > Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch > > > We don't pass the custom timestamp for Increment, and the increment's > timestamp always be rewrite. Hence, user can't increment a cell with custom > timestamp. > {code:title=ProtobufUtil.java} > if (values != null && values.size() > 0) { > for (Cell cell: values) { > valueBuilder.clear(); > valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap( > cell.getQualifierArray(), cell.getQualifierOffset(), > cell.getQualifierLength())); > valueBuilder.setValue(UnsafeByteOperations.unsafeWrap( > cell.getValueArray(), cell.getValueOffset(), > cell.getValueLength())); > if (cell.getTagsLength() > 0) { > > valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(), > cell.getTagsOffset(), cell.getTagsLength())); > } > columnBuilder.addQualifierValue(valueBuilder.build()); > } > } > {code} > In contrast to Increment, user can append the cell with custom timestamp. It > would be better that make their behavior consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123636#comment-16123636 ] Chia-Ping Tsai commented on HBASE-18546: Ping for reviews~ > Append/Increment a cell with custom timestamp > - > > Key: HBASE-18546 > URL: https://issues.apache.org/jira/browse/HBASE-18546 > Project: HBase > Issue Type: New Feature > Components: API, Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Labels: incompatibleChange > Fix For: 2.0.0 > > Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch > > > We don't pass the custom timestamp for Increment, and the increment's > timestamp always be rewrite. Hence, user can't increment a cell with custom > timestamp. > {code:title=ProtobufUtil.java} > if (values != null && values.size() > 0) { > for (Cell cell: values) { > valueBuilder.clear(); > valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap( > cell.getQualifierArray(), cell.getQualifierOffset(), > cell.getQualifierLength())); > valueBuilder.setValue(UnsafeByteOperations.unsafeWrap( > cell.getValueArray(), cell.getValueOffset(), > cell.getValueLength())); > if (cell.getTagsLength() > 0) { > > valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(), > cell.getTagsOffset(), cell.getTagsLength())); > } > columnBuilder.addQualifierValue(valueBuilder.build()); > } > } > {code} > In contrast to Increment, user can append the cell with custom timestamp. It > would be better that make their behavior consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122542#comment-16122542 ] Hadoop QA commented on HBASE-18546: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m 42s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 7 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 8s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 27s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 5s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 31m 51s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 36s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green}113m 58s{color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 38s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}187m 7s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 | | JIRA Issue | HBASE-18546 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12881315/HBASE-18546.v1.patch | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6c093e2536c0 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / d5f34ad | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC3 | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/8020/testReport/ | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/8020/console | | Powered by | Apache Yetus 0.4.0 http://yetus.apache.org | This message was automatically generated. > Append/Increment a cell with custom
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121093#comment-16121093 ] Hadoop QA commented on HBASE-18546: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 46s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 53s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 27s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 16s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 33m 46s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 40s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 44s{color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 81m 49s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.protobuf.TestProtobufUtil | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 | | JIRA Issue | HBASE-18546 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12881134/HBASE-18546.v0.patch | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux dd9eea926f1d 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 6246523 | | Default Java | 1.8.0_131 | | findbugs | v3.1.0-RC3 | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/8007/artifact/patchprocess/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/8007/testReport/ | | modules | C: hbase-client hbase-server U: . | | Console output |
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120967#comment-16120967 ] Chia-Ping Tsai commented on HBASE-18546: bq. IIRC the server side implementation will not take care about the timestamp. The custom timestamp will be adopted if server doesn't find the related cell. As i see it, it is more like a bug. {code} CellUtil.updateLatestStamp(delta, now); // the custom timestamp won't be rewrited newCell = delta; tags = TagUtil.carryForwardTTLTag(tags, mutation.getTTL()); if (tags != null) { newCell = CellUtil.createCell(delta, tags); } {code} bq. I think this should be a new feature? We allow user to use custom timestamp for PUT/DELETE/APPEND. Ya, this should be an new feature even if there is no use case at present. > Append/Increment a cell with custom timestamp > - > > Key: HBASE-18546 > URL: https://issues.apache.org/jira/browse/HBASE-18546 > Project: HBase > Issue Type: Bug > Components: API, Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Labels: incompatibleChange > Fix For: 3.0.0, 2.0.0-alpha-2 > > > We don't pass the custom timestamp for Increment, and the increment's > timestamp always be rewrite. Hence, user can't increment a cell with custom > timestamp. > {code:title=ProtobufUtil.java} > if (values != null && values.size() > 0) { > for (Cell cell: values) { > valueBuilder.clear(); > valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap( > cell.getQualifierArray(), cell.getQualifierOffset(), > cell.getQualifierLength())); > valueBuilder.setValue(UnsafeByteOperations.unsafeWrap( > cell.getValueArray(), cell.getValueOffset(), > cell.getValueLength())); > if (cell.getTagsLength() > 0) { > > valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(), > cell.getTagsOffset(), cell.getTagsLength())); > } > columnBuilder.addQualifierValue(valueBuilder.build()); > } > } > {code} > In contrast to Increment, user can append the cell with custom timestamp. It > would be better that make their behavior consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120880#comment-16120880 ] Duo Zhang commented on HBASE-18546: --- I think this should be a new feature? IIRC the server side implementation will not take care about the timestamp. > Append/Increment a cell with custom timestamp > - > > Key: HBASE-18546 > URL: https://issues.apache.org/jira/browse/HBASE-18546 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > > We don't pass the custom timestamp for Increment, and the increment's > timestamp always be rewrite. Hence, user can't increment a cell with custom > timestamp. > {code:title=ProtobufUtil.java} > if (values != null && values.size() > 0) { > for (Cell cell: values) { > valueBuilder.clear(); > valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap( > cell.getQualifierArray(), cell.getQualifierOffset(), > cell.getQualifierLength())); > valueBuilder.setValue(UnsafeByteOperations.unsafeWrap( > cell.getValueArray(), cell.getValueOffset(), > cell.getValueLength())); > if (cell.getTagsLength() > 0) { > > valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(), > cell.getTagsOffset(), cell.getTagsLength())); > } > columnBuilder.addQualifierValue(valueBuilder.build()); > } > } > {code} > In contrast to Increment, user can append the cell with custom timestamp. It > would be better that make their behavior consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120464#comment-16120464 ] Chia-Ping Tsai commented on HBASE-18546: Ditto for Increment#setTimeRange > Append/Increment a cell with custom timestamp > - > > Key: HBASE-18546 > URL: https://issues.apache.org/jira/browse/HBASE-18546 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > > We don't pass the custom timestamp for Increment, and the increment's > timestamp always be rewrite. Hence, user can't increment a cell with custom > timestamp. > {code:title=ProtobufUtil.java} > if (values != null && values.size() > 0) { > for (Cell cell: values) { > valueBuilder.clear(); > valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap( > cell.getQualifierArray(), cell.getQualifierOffset(), > cell.getQualifierLength())); > valueBuilder.setValue(UnsafeByteOperations.unsafeWrap( > cell.getValueArray(), cell.getValueOffset(), > cell.getValueLength())); > if (cell.getTagsLength() > 0) { > > valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(), > cell.getTagsOffset(), cell.getTagsLength())); > } > columnBuilder.addQualifierValue(valueBuilder.build()); > } > } > {code} > In contrast to Increment, user can append the cell with custom timestamp. It > would be better that make their behavior consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[ https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120459#comment-16120459 ] Chia-Ping Tsai commented on HBASE-18546: For operational compatibility, we should fix the Increment. But i have no use cases.. > Append/Increment a cell with custom timestamp > - > > Key: HBASE-18546 > URL: https://issues.apache.org/jira/browse/HBASE-18546 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > > We don't pass the custom timestamp for Increment, and the increment's > timestamp always be rewrite. Hence, user can't increment a cell with custom > timestamp. > {code:title=ProtobufUtil.java} > if (values != null && values.size() > 0) { > for (Cell cell: values) { > valueBuilder.clear(); > valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap( > cell.getQualifierArray(), cell.getQualifierOffset(), > cell.getQualifierLength())); > valueBuilder.setValue(UnsafeByteOperations.unsafeWrap( > cell.getValueArray(), cell.getValueOffset(), > cell.getValueLength())); > if (cell.getTagsLength() > 0) { > > valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(), > cell.getTagsOffset(), cell.getTagsLength())); > } > columnBuilder.addQualifierValue(valueBuilder.build()); > } > } > {code} > In contrast to Increment, user can append the cell with custom timestamp. It > would be better that make their behavior consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)