[jira] Commented: (HBASE-2946) Increment multiple columns in a row at once
[ https://issues.apache.org/jira/browse/HBASE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12925047#action_12925047 ] HBase Review Board commented on HBASE-2946: --- Message from: thekhem...@facebook.com --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1088/#review1669 --- trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java http://review.cloudera.org/r/1088/#comment5545 This might not work if getLastIncrement() returns fewer counters in the result than what was queried. - khemani Increment multiple columns in a row at once --- Key: HBASE-2946 URL: https://issues.apache.org/jira/browse/HBASE-2946 Project: HBase Issue Type: New Feature Components: client, regionserver Reporter: Jonathan Gray Assignee: Jonathan Gray Currently there is no way to do multiple increments to a single row in one RPC. This jira is about adding an HTable and HRegionInterface method to increment multiple columns within a single row at once. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2946) Increment multiple columns in a row at once
[ https://issues.apache.org/jira/browse/HBASE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12924728#action_12924728 ] HBase Review Board commented on HBASE-2946: --- Message from: Jonathan Gray jg...@apache.org bq. On 2010-10-24 21:41:48, khemani wrote: bq. trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java, line 45 bq. http://review.cloudera.org/r/1088/diff/2/?file=15904#file15904line45 bq. bq. setWriteToWal() is missing? yup. will add. bq. On 2010-10-24 21:41:48, khemani wrote: bq. trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java, lines 46-47 bq. http://review.cloudera.org/r/1088/diff/2/?file=15904#file15904line46 bq. bq. why a navigable map? why not just a map? u can do things like tailMap[() with it. bq. On 2010-10-24 21:41:48, khemani wrote: bq. trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java, line 3012 bq. http://review.cloudera.org/r/1088/diff/2/?file=15907#file15907line3012 bq. bq. I am not sure how it is ensured that the order of iteration over the columns in family.getValue.entrySet() is same as the order of results returned? bq. bq. Also, if get finds multiple matches then will it return all of them? If yes then this will not work. familyMap and the map of columns to amounts are both TreeMaps ordered with Bytes.BYTES_COMPARATOR. Results are also guaranteed to be in order. And our Get has maxVersions=1 so we will not get multiple matches per column. Increment multiple columns in a row at once --- Key: HBASE-2946 URL: https://issues.apache.org/jira/browse/HBASE-2946 Project: HBase Issue Type: New Feature Components: client, regionserver Reporter: Jonathan Gray Assignee: Jonathan Gray Currently there is no way to do multiple increments to a single row in one RPC. This jira is about adding an HTable and HRegionInterface method to increment multiple columns within a single row at once. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2946) Increment multiple columns in a row at once
[ https://issues.apache.org/jira/browse/HBASE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12924433#action_12924433 ] HBase Review Board commented on HBASE-2946: --- Message from: Jonathan Gray jg...@apache.org --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1088/ --- Review request for hbase, stack and khemani. Summary --- Adds a new Increment class that allows multiple columns (each w/ own increment amount) in a single row being incremented in one call. The big wins here are being able to do multiple columns in a row in a single RPC and having it be appended/synced to the WAL in a single call. The current trade-off is that you lose atomicity to readers (ie. this does not currently use RWCC). Eventually it could but for the current use case I am building this for, it's okay like this. This addresses bug HBASE-2946. http://issues.apache.org/jira/browse/HBASE-2946 Diffs - trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java 1026930 trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java 1026930 trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java PRE-CREATION trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 1026930 trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 1026930 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1026930 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1026930 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java 1026930 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1026930 trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java 1026930 trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1026930 Diff: http://review.cloudera.org/r/1088/diff Testing --- Added TestFromClientSide.testIncrement() which adds some client-side tests of Increment (and mixing w/ original icv call). That passes and most the way through a test suite run. Thanks, Jonathan Increment multiple columns in a row at once --- Key: HBASE-2946 URL: https://issues.apache.org/jira/browse/HBASE-2946 Project: HBase Issue Type: New Feature Components: client, regionserver Reporter: Jonathan Gray Assignee: Jonathan Gray Currently there is no way to do multiple increments to a single row in one RPC. This jira is about adding an HTable and HRegionInterface method to increment multiple columns within a single row at once. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2946) Increment multiple columns in a row at once
[ https://issues.apache.org/jira/browse/HBASE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12924441#action_12924441 ] HBase Review Board commented on HBASE-2946: --- Message from: Jonathan Gray jg...@apache.org bq. On 2010-10-24 20:05:55, Ryan Rawson wrote: bq. trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 195 bq. http://review.cloudera.org/r/1088/diff/2/?file=15905#file15905line195 bq. bq. adding trailing spaces bq. that's my patch removing trailing spaces bq. On 2010-10-24 20:05:55, Ryan Rawson wrote: bq. trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 201 bq. http://review.cloudera.org/r/1088/diff/2/?file=15905#file15905line201 bq. bq. adding more trailing spaces, your ide should have a feature to strip these same here... i'm on the right :) bq. On 2010-10-24 20:05:55, Ryan Rawson wrote: bq. trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java, line 495 bq. http://review.cloudera.org/r/1088/diff/2/?file=15909#file15909line495 bq. bq. yeah you cant compare against memstoreTS because if you have this in here you wont be able to ever increment values that were inserted into the future. you'd just leave them there and continually see it in the 'get' part and then in this code bit leave it in place and create a new KV that is masked by the future KV. bq. bq. It won't be possible for that insert to be part of an uncommitted change because of the rowlock however. So no atomic-rules will have been broken. i'm not sure i follow. am i doing it wrong? this should be exactly the same as what's there. also, we are breaking the atomic-rules across the Increment. Each column is not broken but across them it is (for reads not writes). It seems like we could use RWCC though for increments. I think it's fine that if you're using increment you can never write data into it w/ another API (or really, with manual timestamps). bq. On 2010-10-24 20:05:55, Ryan Rawson wrote: bq. trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java, line 377 bq. http://review.cloudera.org/r/1088/diff/2/?file=15909#file15909line377 bq. bq. I'm not sure how this code is optional for your new 'upsert', here are some use cases that I found painful: bq. bq. - when the KV is in snapshot we can create new KVs in memstore with the same TS. This means you have a dup and before we had this new 'dup' code it ruined counts badly. bq. - the Math.max() bit it to ensure the new KV isnt being created in the past a bit and accidently duplicating a timestamp inside the snapshot. what do you mean by optional? there shouldn't be any real difference. this code is basically the exact same code that was there but now pulled into a method that can be reused. bq. On 2010-10-24 20:05:55, Ryan Rawson wrote: bq. trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java, line 466 bq. http://review.cloudera.org/r/1088/diff/2/?file=15909#file15909line466 bq. bq. im not sure i like the name upsert, it is too rdbms-y for me. bq. bq. I need to poke more at this, but i prever the matchingRow() call, it encapsulates the getRowOffset junk, which leaks wayyy too much all over the place. bq. bq. Sure, could use those calls. Not even sure why I changed this, wrote that part of this patch a few weeks ago. And it's an update if it exists, insert if not operation which I think of as an upsert operation. Open to whatever tho. - Jonathan --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1088/#review1641 --- Increment multiple columns in a row at once --- Key: HBASE-2946 URL: https://issues.apache.org/jira/browse/HBASE-2946 Project: HBase Issue Type: New Feature Components: client, regionserver Reporter: Jonathan Gray Assignee: Jonathan Gray Currently there is no way to do multiple increments to a single row in one RPC. This jira is about adding an HTable and HRegionInterface method to increment multiple columns within a single row at once. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2946) Increment multiple columns in a row at once
[
https://issues.apache.org/jira/browse/HBASE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904883#action_12904883
]
Jonathan Gray commented on HBASE-2946:
--
There are two general approaches from an API perspective.
One would be just adding another single HTable call, something like:
{noformat}
long [] incrementColumns(byte [] row, byte [][] families, byte [][] qualifiers,
long [] amounts)
{noformat}
The other would be to upgrade our increment friend to his own Increment class.
Then he'd operate like Gets/Puts:
{noformat}
long [] increment(new Increment(byte [] row).addColumn(family,
qualifier1).addColumn(family, qualifier2))
{noformat}
The latter would be more amenable to HBASE-2947.
Increment multiple columns in a row at once
---
Key: HBASE-2946
URL: https://issues.apache.org/jira/browse/HBASE-2946
Project: HBase
Issue Type: New Feature
Components: client, regionserver
Reporter: Jonathan Gray
Fix For: 0.90.0
Currently there is no way to do multiple increments to a single row in one
RPC. This jira is about adding an HTable and HRegionInterface method to
increment multiple columns within a single row at once.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2946) Increment multiple columns in a row at once
[ https://issues.apache.org/jira/browse/HBASE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904884#action_12904884 ] Jonathan Gray commented on HBASE-2946: -- Forgot the amount in the second API but you get the picture. Also, it would be nice if the multiple increments to a row would either all pass or all fail, so this would still be atomic. Should be okay from WAL perspective but since it uses memstoreTS=0, not sure that will jive with the RWCC business. @Ryan, thoughts on that? Increment multiple columns in a row at once --- Key: HBASE-2946 URL: https://issues.apache.org/jira/browse/HBASE-2946 Project: HBase Issue Type: New Feature Components: client, regionserver Reporter: Jonathan Gray Fix For: 0.90.0 Currently there is no way to do multiple increments to a single row in one RPC. This jira is about adding an HTable and HRegionInterface method to increment multiple columns within a single row at once. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2946) Increment multiple columns in a row at once
[ https://issues.apache.org/jira/browse/HBASE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904889#action_12904889 ] stack commented on HBASE-2946: -- I like idea of an Increment class -- could it subclass Put? Increment multiple columns in a row at once --- Key: HBASE-2946 URL: https://issues.apache.org/jira/browse/HBASE-2946 Project: HBase Issue Type: New Feature Components: client, regionserver Reporter: Jonathan Gray Fix For: 0.90.0 Currently there is no way to do multiple increments to a single row in one RPC. This jira is about adding an HTable and HRegionInterface method to increment multiple columns within a single row at once. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2946) Increment multiple columns in a row at once
[ https://issues.apache.org/jira/browse/HBASE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904890#action_12904890 ] Jonathan Gray commented on HBASE-2946: -- It could I suppose. Put is a list of KVs. We could actually munge it into a list of KVs but otherwise it'd be more like a Get with a familyMap. Increment multiple columns in a row at once --- Key: HBASE-2946 URL: https://issues.apache.org/jira/browse/HBASE-2946 Project: HBase Issue Type: New Feature Components: client, regionserver Reporter: Jonathan Gray Fix For: 0.90.0 Currently there is no way to do multiple increments to a single row in one RPC. This jira is about adding an HTable and HRegionInterface method to increment multiple columns within a single row at once. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
