date:20140415

[jira] [Commented] (HBASE-10885) Support visibility expressions on Deletes

2014-04-15 Thread Anoop Sam John (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969254#comment-13969254
]

Anoop Sam John commented on HBASE-10885:

bq. Do the delete handling in compaction because for the deleteColumn or
deleteFamily cases if we convert that delete request to a set of per-cell
deletes, this could produce an explosion of tombstones.
+1
bq.If we are not storing visibility expression terminals (LeafExpressionNodes)
in sorted order by ordinal we probably should consider it. (I don't think we
are.) Because e.g. A|B == B|A. It would be most efficient if we can simply do
byte comparison of serialized visibility expressions on the delete marker and
any found while enumerating cells covered by it.
Yes +1.

So when we decide at the compaction time by direct byte compare of expressions
in Delete kv and other kvs, what abt the old HFile data written using 98.0 or
98.1 versions? This will be like not sorted.
Also it wont one simple bytes compare. AB will be stored as one Tag. In case
of '|' it will be stored as 2 tags. So we need to consider all these tags and
compare all. Just a note before starting the impl.

Support visibility expressions on Deletes
-

Key: HBASE-10885
URL: https://issues.apache.org/jira/browse/HBASE-10885
Project: HBase
Issue Type: Improvement
Affects Versions: 0.98.1
Reporter: Andrew Purtell
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0, 0.98.2

Accumulo can specify visibility expressions for delete markers. During
compaction the cells covered by the tombstone are determined in part by
matching the visibility expression. This is useful for the use case of data
set coalescing, where entries from multiple data sets carrying different
labels are combined into one common large table. Later, a subset of entries
can be conveniently removed using visibility expressions.
Currently doing the same in HBase would only be possible with a custom
coprocessor. Otherwise, a Delete will affect all cells covered by the
tombstone regardless of any visibility expression scoping. This is correct
behavior in that no data spill is possible, but certainly could be
surprising, and is only meant to be transitional. We decided not to support
visibility expressions on Deletes to control the complexity of the initial
implementation.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HBASE-10966) RowCounter misinterprets column names that have colons in their qualifier

2014-04-15 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10966:
--

Attachment: 10966v094.txt

v0.94 version that I applied.

 RowCounter misinterprets column names that have colons in their qualifier
 -

 Key: HBASE-10966
 URL: https://issues.apache.org/jira/browse/HBASE-10966
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Alexandre Normand
Assignee: Alexandre Normand
Priority: Trivial
 Attachments: 10966v094.txt, HBASE-10966-1.patch


 RowCounter allows for column names to be specify at command line:
 {code}
 Usage: RowCounter [options] tablename [--range=[startKey],[endKey]] 
 [column1 column2...]
 For performance consider the following options:
 -Dhbase.client.scanner.caching=100
 -Dmapred.map.tasks.speculative.execution=false
 {code}
 However, the column names are parsed assuming that if there is a colon, there 
 are only two parts to the string. In other words, it assumes 
 {{family:qualifier}} where {{qualifier}} wouldn't contain a colon. 
 This came up as I was trying to do a row count on a {{kiji}} table where 
 qualifiers typically have multiple colon-delimited components (i.e. {{B:C}} 
 could be a qualifier in the {{B}} family).
 The flaw is in this code:
 {code}
String  [] fields = columnName.split(:);
 if(fields.length == 1) {
   scan.addFamily(Bytes.toBytes(fields[0]));
 } else {
   byte[] qualifier = Bytes.toBytes(fields[1]);
   qualifiers.add(qualifier);
   scan.addColumn(Bytes.toBytes(fields[0]), qualifier); 
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HBASE-10966) RowCounter misinterprets column names that have colons in their qualifier

2014-04-15 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10966:
--

   Resolution: Fixed
Fix Version/s: 0.96.3
   0.98.2
   0.94.19
   0.99.0
 Release Note: Applied to all branches (0.94 doesn't have TestRowCounter so 
didn't get test change).  Thanks for the patch Alexandre.
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

 RowCounter misinterprets column names that have colons in their qualifier
 -

 Key: HBASE-10966
 URL: https://issues.apache.org/jira/browse/HBASE-10966
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Alexandre Normand
Assignee: Alexandre Normand
Priority: Trivial
 Fix For: 0.99.0, 0.94.19, 0.98.2, 0.96.3

 Attachments: 10966v094.txt, HBASE-10966-1.patch


 RowCounter allows for column names to be specify at command line:
 {code}
 Usage: RowCounter [options] tablename [--range=[startKey],[endKey]] 
 [column1 column2...]
 For performance consider the following options:
 -Dhbase.client.scanner.caching=100
 -Dmapred.map.tasks.speculative.execution=false
 {code}
 However, the column names are parsed assuming that if there is a colon, there 
 are only two parts to the string. In other words, it assumes 
 {{family:qualifier}} where {{qualifier}} wouldn't contain a colon. 
 This came up as I was trying to do a row count on a {{kiji}} table where 
 qualifiers typically have multiple colon-delimited components (i.e. {{B:C}} 
 could be a qualifier in the {{B}} family).
 The flaw is in this code:
 {code}
String  [] fields = columnName.split(:);
 if(fields.length == 1) {
   scan.addFamily(Bytes.toBytes(fields[0]));
 } else {
   byte[] qualifier = Bytes.toBytes(fields[1]);
   qualifiers.add(qualifier);
   scan.addColumn(Bytes.toBytes(fields[0]), qualifier); 
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10960) Enhance HBase Thrift 1 to include append and checkAndPut operations

2014-04-15 Thread Srikanth Srungarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969259#comment-13969259
 ] 

Srikanth Srungarapu commented on HBASE-10960:
-

Added the patch to the review board (https://reviews.apache.org/r/20361/).

 Enhance HBase Thrift 1 to include append and checkAndPut operations
 ---

 Key: HBASE-10960
 URL: https://issues.apache.org/jira/browse/HBASE-10960
 Project: HBase
  Issue Type: Improvement
  Components: Thrift
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
 Fix For: 0.98.2

 Attachments: HBASE-10960.patch


 Both append, and checkAndPut functionalities are available in Thrift 2 
 interface, but not in Thrift. So, adding the support for these 
 functionalities in Thrift1 too.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10845) Memstore snapshot size isn't updated in DefaultMemStore#rollback()

2014-04-15 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969264#comment-13969264
 ] 

stack commented on HBASE-10845:
---

[~ted_yu] Not what I was thinking but its a test... +1 (I was thinking you'd 
just do mock up in TestDefaultMemStore -- it already has a bunch of these kind 
of size calculations).

It looks like trunk already has this fix since HBASE-10648.  It was just not 
backported?

Thanks.

 Memstore snapshot size isn't updated in DefaultMemStore#rollback()
 --

 Key: HBASE-10845
 URL: https://issues.apache.org/jira/browse/HBASE-10845
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.99.0, 0.94.19, 0.98.2, 0.96.3

 Attachments: 10845-0.98.txt, 10845-v1.txt, 10845-v2.txt, 
 10845-v3.txt, 10845-v4.txt


 In DefaultMemStore#rollback() :
 {code}
  if (found != null  found.getMvccVersion() == kv.getMvccVersion()) {
this.snapshot.remove(kv);
  }
 {code}
 this.snapshotSize should be updated in case of rollback.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()

2014-04-15 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969266#comment-13969266
 ] 

Anoop Sam John commented on HBASE-10965:


In initial time of 94 , I remember that   Filter#hasFilterRow() was associated 
with Filter#filterRow(ListKV)
We used this boolean API in 2 places
1. To check whether Scan is with a Filter, which is having row filtering, and 
with a batch value set. This is not correct way and we have to fail then
2. The place before we call Filter#filterRow(ListKV) in HRegion

Later we made it like hasFilterRow() to return true when filterRow() is also 
implemented and so same is used in code where we call filterRow() also.

Actually we need to use this boolean API only in case 1.  When one uses a 
filter and a Scan batch wrongly, we will fail the scan op 1st step itself. 
So there is no need to check this boolean API value before calling the actual 
Filter methods. The impl in FilterBase do nothing and no problem in calling 
that I believe.

So with this Jira, if there is consensus to remove even the hasFilterRow() 
method, we can simplify the things.  Just in place where we check the Scan 
filter and batch to see whether we can proceed with the open scanner, just do 
this auto detect of method impls and act.  In other places of HRegion scan 
flow, no need to rely on any of these boolean values of hasFilterRow() returns.

Make some sense?

When we auto detect presence of methods take care of the FL case. ( FL inside 
FL) which I was saying above.

 Automate detection of presence of Filter#filterRow()
 

 Key: HBASE-10965
 URL: https://issues.apache.org/jira/browse/HBASE-10965
 Project: HBase
  Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
 Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt


 There is potential inconsistency between the return value of 
 Filter#hasFilterRow() and presence of Filter#filterRow().
 Filters may override Filter#filterRow() while leaving return value of 
 Filter#hasFilterRow() being false (inherited from FilterBase).
 This JIRA aims to remove the inconsistency by automatically detecting the 
 presence of overridden Filter#filterRow(). If filterRow() is implemented and 
 not inherited from FilterBase, it is equivalent to having hasFilterRow() 
 return true.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10845) Memstore snapshot size isn't updated in DefaultMemStore#rollback()

2014-04-15 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969269#comment-13969269
 ] 

Anoop Sam John commented on HBASE-10845:


bq.It looks like trunk already has this fix 
No Stack.  This fix is not there in any of the branch.

 Memstore snapshot size isn't updated in DefaultMemStore#rollback()
 --

 Key: HBASE-10845
 URL: https://issues.apache.org/jira/browse/HBASE-10845
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.99.0, 0.94.19, 0.98.2, 0.96.3

 Attachments: 10845-0.98.txt, 10845-v1.txt, 10845-v2.txt, 
 10845-v3.txt, 10845-v4.txt


 In DefaultMemStore#rollback() :
 {code}
  if (found != null  found.getMvccVersion() == kv.getMvccVersion()) {
this.snapshot.remove(kv);
  }
 {code}
 this.snapshotSize should be updated in case of rollback.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

1 2 3 >

1 - 100 of 249 matches

Mail list logo