date:20120604


[ 
https://issues.apache.org/jira/browse/HBASE-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288396#comment-13288396
 ] 

Anoop Sam John commented on HBASE-6132:
---

I think that filterRow() in FilterList should not check with the 
filterAllRemaining() on individual filters. filterRow() is being called to 
check whether the current row under check needs to be included or not. 
filterAllRemaining() speaks abt the rows coming after the current row [In this 
context]
Note : filterAllRemaining() will be called during the read of individual KVs 
belonging to one row where it tells abt skipping the coming KVs.

Whatever KVs were included for one row should get returned by the scanner.  So 
usage of filterAllRemaining() as part of the filterRow() check should be 
avoided I feel.

Going through other method impls in FilterList. Getting doubts with some others 
also.

 ColumnCountGetFilter  PageFilter not working with FilterList
 -

 Key: HBASE-6132
 URL: https://issues.apache.org/jira/browse/HBASE-6132
 Project: HBase
  Issue Type: Bug
  Components: filters
Affects Versions: 0.92.0, 0.92.1, 0.94.0
 Environment: Cent OS 5.5 distributed hbase cluster. Hadoop 1.0.0, 
 zookeeper 3.4.3
Reporter: Benjamin Kim

 Thanks to Anoop and Ramkrishna, here's what we found with FilterList
 If I use FilterList to include ColumnCountGetFilter among other filters, the 
 returning Result has no keyvalues.
 This problem seems to occur when specified column count is less then actual 
 number of existing columns.
 Also same problem arises with PageFilter
 Following is the code of the problem:
 {code}
 Configuration conf = HBaseConfiguration.create();
 HTable table = new HTable(conf, test);
 Get get = new Get(Bytes.toBytes(test1));
 FilterList filterList = new FilterList();
 filterList.addFilter(new ColumnCountGetFilter(100));   
 get.setFilter(filterList);
 Result r = table.get(get);
 System.out.println(r.size()); // prints zero
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96

[
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288414#comment-13288414
]

Lars Hofhansl commented on HBASE-6055:
--

Now that I started to get a bit more familiar with HDFS I am wondering whether
HDFS hardlinks (HDFS-3370) or even HDFS snapshots (HDFS-233, HDFS-2802) are not
a better avenue. We are looking for data consistency here, which would be
better tackled at the data layer.
Now... Both features are some ways off in HDFS (although we can probably push
these forward), so doing something in HBase first is probably needed, but IMHO
it should be something quick.
Lastly if we are considering this for backups HBASE-5547 should be a better
(simpler) solution.

Not trying to derail anything here, just making sure we do not invest a lot of
time in vain.

Snapshots in HBase 0.96
---

Key: HBASE-6055
URL: https://issues.apache.org/jira/browse/HBASE-6055
Project: HBase
Issue Type: New Feature
Components: client, master, regionserver, zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.96.0

Attachments: Snapshots in HBase.docx

Continuation of HBASE-50 for the current trunk. Since the implementation has
drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.


[ 
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288418#comment-13288418
 ] 

Jonathan Hsieh commented on HBASE-5892:
---

Looks good to me.  I was able to apply the v4 version to trunk and to 0.92/0.94 
via 'patch -p1  hbase-5892-4.patch' but at the moment am having problems with 
my account.  Will commit when I get fixed. 

 [hbck] Refactor parallel WorkItem* to Futures.
 --

 Key: HBASE-5892
 URL: https://issues.apache.org/jira/browse/HBASE-5892
 Project: HBase
  Issue Type: Improvement
Reporter: Jonathan Hsieh
Assignee: Andrew Wang
  Labels: noob
 Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, 
 hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892-4-0.90.patch, 
 hbase-5892-4.patch, hbase-5892.patch


 This would convert WorkItem* logic (with low level notifies, and rough 
 exception handling)  into a more canonical Futures pattern.
 Currently there are two instances of this pattern (for loading hdfs dirs, for 
 contacting regionservers for assignments, and soon -- for loading hdfs 
 .regioninfo files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.


 [ 
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5892:
--

Affects Version/s: 0.90.6
   0.92.1
   0.94.0
Fix Version/s: 0.92.3
   0.94.1
   0.96.0
   0.90.7
 Hadoop Flags: Reviewed

 [hbck] Refactor parallel WorkItem* to Futures.
 --

 Key: HBASE-5892
 URL: https://issues.apache.org/jira/browse/HBASE-5892
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Andrew Wang
  Labels: noob
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, 
 hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892-4-0.90.patch, 
 hbase-5892-4.patch, hbase-5892.patch


 This would convert WorkItem* logic (with low level notifies, and rough 
 exception handling)  into a more canonical Futures pattern.
 Currently there are two instances of this pattern (for loading hdfs dirs, for 
 contacting regionservers for assignments, and soon -- for loading hdfs 
 .regioninfo files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.


 [ 
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5892:
--

Component/s: hbck

 [hbck] Refactor parallel WorkItem* to Futures.
 --

 Key: HBASE-5892
 URL: https://issues.apache.org/jira/browse/HBASE-5892
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Andrew Wang
  Labels: noob
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, 
 hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892-4-0.90.patch, 
 hbase-5892-4.patch, hbase-5892.patch


 This would convert WorkItem* logic (with low level notifies, and rough 
 exception handling)  into a more canonical Futures pattern.
 Currently there are two instances of this pattern (for loading hdfs dirs, for 
 contacting regionservers for assignments, and soon -- for loading hdfs 
 .regioninfo files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96


[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288429#comment-13288429
 ] 

Jonathan Hsieh commented on HBASE-6055:
---

I agreee that HDFS-3370, and HDFS-233 are likely a ways off.  Hardlinks would 
probably obviate the need for HBASE-5547.  We could probably take advantage of 
HDFS symlinks HDFS-245 which is in Hadoop 2.x.x hdfs but may not be in Hadoop 
1.x.x hdfs.

I think that HBASE-5547 is a prereq for either consistency approach (even if we 
use symlinks) until we have hdfs hardlinks.  I'll take a closer look into 
HBASE-5547.

 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: client, master, regionserver, zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6096) AccessController v2

2012-06-04 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288444#comment-13288444
 ] 

Andrew Purtell commented on HBASE-6096:
---

bq.  Revoke of Global permission is not taking effect without restart.

Better to make a new JIRA than reopen HBASE-5342 I'd say. 

 AccessController v2
 ---

 Key: HBASE-6096
 URL: https://issues.apache.org/jira/browse/HBASE-6096
 Project: HBase
  Issue Type: Umbrella
  Components: security
Affects Versions: 0.96.0, 0.94.1
Reporter: Andrew Purtell

 Umbrella issue for iteration on the initial AccessController drop.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.


 [ 
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5892:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to 0.90/0.92/0.94/trunk branches.  Thanks Andrew!

 [hbck] Refactor parallel WorkItem* to Futures.
 --

 Key: HBASE-5892
 URL: https://issues.apache.org/jira/browse/HBASE-5892
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Andrew Wang
  Labels: noob
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, 
 hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892-4-0.90.patch, 
 hbase-5892-4.patch, hbase-5892.patch


 This would convert WorkItem* logic (with low level notifies, and rough 
 exception handling)  into a more canonical Futures pattern.
 Currently there are two instances of this pattern (for loading hdfs dirs, for 
 contacting regionservers for assignments, and soon -- for loading hdfs 
 .regioninfo files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5874) When 'fs.default.name' not configured, the hbck tool and Merge tool throw IllegalArgumentException.


 [ 
https://issues.apache.org/jira/browse/HBASE-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5874:
--

Summary: When 'fs.default.name' not configured, the hbck tool and Merge 
tool throw IllegalArgumentException.  (was: The HBase do not configure the 
'fs.default.name' attribute, the hbck tool and Merge tool throw 
IllegalArgumentException.)

 When 'fs.default.name' not configured, the hbck tool and Merge tool throw 
 IllegalArgumentException.
 ---

 Key: HBASE-5874
 URL: https://issues.apache.org/jira/browse/HBASE-5874
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
 Attachments: HBASE-5874-0.90-v2.patch, HBASE-5874-0.90.patch, 
 HBASE-5874-trunk-v2.patch, HBASE-5874-trunk-v3.patch, HBASE-5874-trunk.patch


 The HBase do not configure the 'fs.default.name' attribute, the hbck tool and 
 Merge tool throw IllegalArgumentException.
 the hbck tool and Merge tool, we should add 'fs.default.name' attriubte to 
 the code.
 hbck exception:
 Exception in thread main java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:128)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:301)
   at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:489)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegioninfo(HBaseFsck.java:565)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegionInfos(HBaseFsck.java:596)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:332)
   at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:360)
   at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2907)
 
 Merge exception:  
 [2012-05-05 10:48:24,830] [ERROR] [main] [org.apache.hadoop.hbase.util.Merge 
 381] exiting due to error
 java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:823)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:415)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2679)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2665)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2634)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.openMetaRegion(MetaUtils.java:276)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.scanMetaRegion(MetaUtils.java:261)
   at org.apache.hadoop.hbase.util.Merge.run(Merge.java:115)
   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
   at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6156) Improve multiop performances in HTable#flushCommits

2012-06-04 Thread nkeywal (JIRA)

nkeywal created HBASE-6156:
--

 Summary: Improve multiop performances in HTable#flushCommits
 Key: HBASE-6156
 URL: https://issues.apache.org/jira/browse/HBASE-6156
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0


This code:

{noformat}
  @Override
  public void flushCommits() throws IOException {
try {
  Object[] results = new Object[writeBuffer.size()];
  try {
this.connection.processBatch(writeBuffer, tableName, pool, results);
  } catch (InterruptedException e) {
throw new IOException(e);
  } finally {
// mutate list so that it is empty for complete success, or contains
// only failed records results are returned in the same order as the
// requests in list walk the list backwards, so we can remove from list
// without impacting the indexes of earlier members
for (int i = results.length - 1; i=0; i--) {
  if (results[i] instanceof Result) {
// successful Puts are removed from the list here.
writeBuffer.remove(i);
  }
}
  }
} finally {
  if (clearBufferOnFail) {
writeBuffer.clear();
currentWriteBufferSize = 0;
  } else {
// the write buffer was adjusted by processBatchOfPuts
currentWriteBufferSize = 0;
for (Put aPut : writeBuffer) {
  currentWriteBufferSize += aPut.heapSize();
}
  }
}
  }
{noformat}

Can be improved by:
- not iterating on the list if clearBufferOnFail is set
- not iterating the the list of there are no error
- iterating on the list only once instead of two when we really have to.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5874) When 'fs.default.name' not configured, the hbck tool and Merge tool throw IllegalArgumentException.


 [ 
https://issues.apache.org/jira/browse/HBASE-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5874:
--

   Resolution: Fixed
Fix Version/s: 0.92.3
   0.94.1
   0.96.0
   0.90.7
   Status: Resolved  (was: Patch Available)

Sorry for the delay and thanks for the patch fulin!  

I committed to 0.90/0.92/0.94/0.96 with a minor tweak (change the  to [] to 
show that the -Dfs.* args are optional).  

 When 'fs.default.name' not configured, the hbck tool and Merge tool throw 
 IllegalArgumentException.
 ---

 Key: HBASE-5874
 URL: https://issues.apache.org/jira/browse/HBASE-5874
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: HBASE-5874-0.90-v2.patch, HBASE-5874-0.90.patch, 
 HBASE-5874-trunk-v2.patch, HBASE-5874-trunk-v3.patch, HBASE-5874-trunk.patch


 The HBase do not configure the 'fs.default.name' attribute, the hbck tool and 
 Merge tool throw IllegalArgumentException.
 the hbck tool and Merge tool, we should add 'fs.default.name' attriubte to 
 the code.
 hbck exception:
 Exception in thread main java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:128)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:301)
   at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:489)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegioninfo(HBaseFsck.java:565)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegionInfos(HBaseFsck.java:596)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:332)
   at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:360)
   at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2907)
 
 Merge exception:  
 [2012-05-05 10:48:24,830] [ERROR] [main] [org.apache.hadoop.hbase.util.Merge 
 381] exiting due to error
 java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:823)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:415)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2679)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2665)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2634)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.openMetaRegion(MetaUtils.java:276)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.scanMetaRegion(MetaUtils.java:261)
   at org.apache.hadoop.hbase.util.Merge.run(Merge.java:115)
   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
   at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics


[ 
https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288478#comment-13288478
 ] 

Jonathan Hsieh commented on HBASE-6131:
---

Since this is in 0.92 as well, we should apply there as well?   This is 
probably a blocker for 0.92.2rc0.

 Add attribution for code added by HBASE-5533 metrics
 

 Key: HBASE-6131
 URL: https://issues.apache.org/jira/browse/HBASE-6131
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.94.1

 Attachments: 6131.txt, 6131_094.txt


 See the comment over in 
 https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920
 The metrics histogram code was copied w/o attribution.  Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-6132) ColumnCountGetFilter PageFilter not working with FilterList


 [ 
https://issues.apache.org/jira/browse/HBASE-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John reassigned HBASE-6132:
-

Assignee: Anoop Sam John

 ColumnCountGetFilter  PageFilter not working with FilterList
 -

 Key: HBASE-6132
 URL: https://issues.apache.org/jira/browse/HBASE-6132
 Project: HBase
  Issue Type: Bug
  Components: filters
Affects Versions: 0.92.0, 0.92.1, 0.94.0
 Environment: Cent OS 5.5 distributed hbase cluster. Hadoop 1.0.0, 
 zookeeper 3.4.3
Reporter: Benjamin Kim
Assignee: Anoop Sam John

 Thanks to Anoop and Ramkrishna, here's what we found with FilterList
 If I use FilterList to include ColumnCountGetFilter among other filters, the 
 returning Result has no keyvalues.
 This problem seems to occur when specified column count is less then actual 
 number of existing columns.
 Also same problem arises with PageFilter
 Following is the code of the problem:
 {code}
 Configuration conf = HBaseConfiguration.create();
 HTable table = new HTable(conf, test);
 Get get = new Get(Bytes.toBytes(test1));
 FilterList filterList = new FilterList();
 filterList.addFilter(new ColumnCountGetFilter(100));   
 get.setFilter(filterList);
 Result r = table.get(get);
 System.out.println(r.size()); // prints zero
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.


[ 
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288485#comment-13288485
 ] 

Hudson commented on HBASE-5892:
---

Integrated in HBase-TRUNK #2982 (See 
[https://builds.apache.org/job/HBase-TRUNK/2982/])
HBASE-5892 [hbck] Refactor parallel WorkItem* to Futures (Andrew Wang) 
(Revision 1345890)

 Result = FAILURE
jmhsieh : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java


 [hbck] Refactor parallel WorkItem* to Futures.
 --

 Key: HBASE-5892
 URL: https://issues.apache.org/jira/browse/HBASE-5892
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Andrew Wang
  Labels: noob
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, 
 hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892-4-0.90.patch, 
 hbase-5892-4.patch, hbase-5892.patch


 This would convert WorkItem* logic (with low level notifies, and rough 
 exception handling)  into a more canonical Futures pattern.
 Currently there are two instances of this pattern (for loading hdfs dirs, for 
 contacting regionservers for assignments, and soon -- for loading hdfs 
 .regioninfo files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5874) When 'fs.default.name' not configured, the hbck tool and Merge tool throw IllegalArgumentException.


[ 
https://issues.apache.org/jira/browse/HBASE-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288488#comment-13288488
 ] 

Hudson commented on HBASE-5874:
---

Integrated in HBase-0.94 #244 (See 
[https://builds.apache.org/job/HBase-0.94/244/])
HBASE-5874 When 'fs.default.name' not configured, the hbck tool and Merge 
tool throws IllegalArgumentException (fulin wang) (Revision 1345903)

 Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/Merge.java


 When 'fs.default.name' not configured, the hbck tool and Merge tool throw 
 IllegalArgumentException.
 ---

 Key: HBASE-5874
 URL: https://issues.apache.org/jira/browse/HBASE-5874
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: HBASE-5874-0.90-v2.patch, HBASE-5874-0.90.patch, 
 HBASE-5874-trunk-v2.patch, HBASE-5874-trunk-v3.patch, HBASE-5874-trunk.patch


 The HBase do not configure the 'fs.default.name' attribute, the hbck tool and 
 Merge tool throw IllegalArgumentException.
 the hbck tool and Merge tool, we should add 'fs.default.name' attriubte to 
 the code.
 hbck exception:
 Exception in thread main java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:128)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:301)
   at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:489)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegioninfo(HBaseFsck.java:565)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegionInfos(HBaseFsck.java:596)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:332)
   at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:360)
   at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2907)
 
 Merge exception:  
 [2012-05-05 10:48:24,830] [ERROR] [main] [org.apache.hadoop.hbase.util.Merge 
 381] exiting due to error
 java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:823)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:415)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2679)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2665)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2634)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.openMetaRegion(MetaUtils.java:276)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.scanMetaRegion(MetaUtils.java:261)
   at org.apache.hadoop.hbase.util.Merge.run(Merge.java:115)
   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
   at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics

2012-06-04 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288502#comment-13288502
 ] 

Andrew Purtell commented on HBASE-6131:
---

Remove the copied code and pull in Yammer metrics-core for 0.92 too I'd say, 
with our apologies to Mr. Hale. 

 Add attribution for code added by HBASE-5533 metrics
 

 Key: HBASE-6131
 URL: https://issues.apache.org/jira/browse/HBASE-6131
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.94.1

 Attachments: 6131.txt, 6131_094.txt


 See the comment over in 
 https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920
 The metrics histogram code was copied w/o attribution.  Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6157) Revoke of Global permission is not taking effect without restart.

2012-06-04 Thread Laxman (JIRA)

Laxman created HBASE-6157:
-

 Summary: Revoke of Global permission is not taking effect without 
restart.
 Key: HBASE-6157
 URL: https://issues.apache.org/jira/browse/HBASE-6157
 Project: HBase
  Issue Type: Sub-task
  Components: security
Affects Versions: 0.94.0
Reporter: Laxman
Assignee: Laxman


Revoke of Global permission is not taking effect without restart.
Revoke is updating the acl table but it's not updating the USER_CACHE.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6096) AccessController v2

2012-06-04 Thread Laxman (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288504#comment-13288504
 ] 

Laxman commented on HBASE-6096:
---

Thanks Andy. Added as a subtask [HBASE-6157] to this parent jira. Hope that 
should be fine.

 AccessController v2
 ---

 Key: HBASE-6096
 URL: https://issues.apache.org/jira/browse/HBASE-6096
 Project: HBase
  Issue Type: Umbrella
  Components: security
Affects Versions: 0.96.0, 0.94.1
Reporter: Andrew Purtell

 Umbrella issue for iteration on the initial AccessController drop.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5882) Prcoess RIT on master restart can try assigning the region if the region is found on a dead server instead of waiting for Timeout Monitor


 [ 
https://issues.apache.org/jira/browse/HBASE-5882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5882:
--

Fix Version/s: (was: 0.94.1)

Committed to trunk only.

 Prcoess RIT on master restart can try assigning the region if the region is 
 found on a dead server instead of waiting for Timeout Monitor
 -

 Key: HBASE-5882
 URL: https://issues.apache.org/jira/browse/HBASE-5882
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: ramkrishna.s.vasudevan
Assignee: Ashutosh Jindal
 Fix For: 0.96.0

 Attachments: HBASE-5882_v5.patch, HBASE-5882_v6.patch, 
 hbase_5882.patch, hbase_5882_V2.patch, hbase_5882_V3.patch, 
 hbase_5882_V4.patch


 Currently on  master restart if it tries to do processRIT, any region if 
 found on dead server tries to avoid the nwe assignment so that timeout 
 monitor can take care.
 This case is more prominent if the node is found in RS_ZK_REGION_OPENING 
 state. I think we can handle this by triggering a new assignment with a new 
 plan.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5874) When 'fs.default.name' not configured, the hbck tool and Merge tool throw IllegalArgumentException.


[ 
https://issues.apache.org/jira/browse/HBASE-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288510#comment-13288510
 ] 

Hudson commented on HBASE-5874:
---

Integrated in HBase-TRUNK #2983 (See 
[https://builds.apache.org/job/HBase-TRUNK/2983/])
HBASE-5874 When 'fs.default.name' not configured, the hbck tool and Merge 
tool throws IllegalArgumentException (fulin wang) (Revision 1345902)

 Result = FAILURE
jmhsieh : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/Merge.java


 When 'fs.default.name' not configured, the hbck tool and Merge tool throw 
 IllegalArgumentException.
 ---

 Key: HBASE-5874
 URL: https://issues.apache.org/jira/browse/HBASE-5874
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: HBASE-5874-0.90-v2.patch, HBASE-5874-0.90.patch, 
 HBASE-5874-trunk-v2.patch, HBASE-5874-trunk-v3.patch, HBASE-5874-trunk.patch


 The HBase do not configure the 'fs.default.name' attribute, the hbck tool and 
 Merge tool throw IllegalArgumentException.
 the hbck tool and Merge tool, we should add 'fs.default.name' attriubte to 
 the code.
 hbck exception:
 Exception in thread main java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:128)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:301)
   at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:489)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegioninfo(HBaseFsck.java:565)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegionInfos(HBaseFsck.java:596)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:332)
   at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:360)
   at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2907)
 
 Merge exception:  
 [2012-05-05 10:48:24,830] [ERROR] [main] [org.apache.hadoop.hbase.util.Merge 
 381] exiting due to error
 java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:823)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:415)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2679)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2665)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2634)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.openMetaRegion(MetaUtils.java:276)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.scanMetaRegion(MetaUtils.java:261)
   at org.apache.hadoop.hbase.util.Merge.run(Merge.java:115)
   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
   at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.


[ 
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288519#comment-13288519
 ] 

Hudson commented on HBASE-5892:
---

Integrated in HBase-0.92 #443 (See 
[https://builds.apache.org/job/HBase-0.92/443/])
HBASE-5892 [hbck] Refactor parallel WorkItem* to Futures (Andrew Wang) 
(Revision 1345888)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java


 [hbck] Refactor parallel WorkItem* to Futures.
 --

 Key: HBASE-5892
 URL: https://issues.apache.org/jira/browse/HBASE-5892
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Andrew Wang
  Labels: noob
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, 
 hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892-4-0.90.patch, 
 hbase-5892-4.patch, hbase-5892.patch


 This would convert WorkItem* logic (with low level notifies, and rough 
 exception handling)  into a more canonical Futures pattern.
 Currently there are two instances of this pattern (for loading hdfs dirs, for 
 contacting regionservers for assignments, and soon -- for loading hdfs 
 .regioninfo files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5874) When 'fs.default.name' not configured, the hbck tool and Merge tool throw IllegalArgumentException.


[ 
https://issues.apache.org/jira/browse/HBASE-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288518#comment-13288518
 ] 

Hudson commented on HBASE-5874:
---

Integrated in HBase-0.92 #443 (See 
[https://builds.apache.org/job/HBase-0.92/443/])
HBASE-5874 When 'fs.default.name' not configured, the hbck tool and Merge 
tool throws IllegalArgumentException (fulin wang) (Revision 1345905)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/Merge.java


 When 'fs.default.name' not configured, the hbck tool and Merge tool throw 
 IllegalArgumentException.
 ---

 Key: HBASE-5874
 URL: https://issues.apache.org/jira/browse/HBASE-5874
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: HBASE-5874-0.90-v2.patch, HBASE-5874-0.90.patch, 
 HBASE-5874-trunk-v2.patch, HBASE-5874-trunk-v3.patch, HBASE-5874-trunk.patch


 The HBase do not configure the 'fs.default.name' attribute, the hbck tool and 
 Merge tool throw IllegalArgumentException.
 the hbck tool and Merge tool, we should add 'fs.default.name' attriubte to 
 the code.
 hbck exception:
 Exception in thread main java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:128)
   at 
 org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:301)
   at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:489)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegioninfo(HBaseFsck.java:565)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegionInfos(HBaseFsck.java:596)
   at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:332)
   at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:360)
   at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2907)
 
 Merge exception:  
 [2012-05-05 10:48:24,830] [ERROR] [main] [org.apache.hadoop.hbase.util.Merge 
 381] exiting due to error
 java.lang.IllegalArgumentException: Wrong FS: 
 hdfs://160.176.0.101:9000/hbase/.META./1028785192/.regioninfo, expected: 
 file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:412)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:59)
   at 
 org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:382)
   at 
 org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:285)
   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:823)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:415)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:340)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2679)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2665)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2634)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.openMetaRegion(MetaUtils.java:276)
   at 
 org.apache.hadoop.hbase.util.MetaUtils.scanMetaRegion(MetaUtils.java:261)
   at org.apache.hadoop.hbase.util.Merge.run(Merge.java:115)
   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
   at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem

2012-06-04 Thread Daryn Sharp (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288541#comment-13288541
 ] 

Daryn Sharp commented on HBASE-6067:


{{getMethod}} should be used instead of {{getDeclaredMethod}} since most 
filesystems, like DFS, currently rely on the default impl of the path accepting 
methods in {{FileSystem}} to call the non-path methods (which are or should be 
deprecated).  I don't think {{getDeclaredMethod}} will search the superclasses, 
so it will always fallback to the non-path version when the fs isn't viewfs.

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem

2012-06-04 Thread Daryn Sharp (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288542#comment-13288542
 ] 

Daryn Sharp commented on HBASE-6067:


I forgot to mention, is {{setAccessible}} needed?  The method is public.

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5699) Run with 1 WAL in HRegionServer


 [ 
https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5699:
--

Attachment: PerfHbase.txt

Perf results. 
@Ted
The file attached also has the latency results. Run using LoadTestTool.  Sorry 
for being little late. Patch will upload later.

 Run with  1 WAL in HRegionServer
 -

 Key: HBASE-5699
 URL: https://issues.apache.org/jira/browse/HBASE-5699
 Project: HBase
  Issue Type: Improvement
Reporter: binlijin
Assignee: Li Pi
 Attachments: PerfHbase.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect


 [ 
https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-5974:
--

Attachment: 5974_trunk.patch

Patch for trunk.

 Scanner retry behavior with RPC timeout on next() seems incorrect
 -

 Key: HBASE-5974
 URL: https://issues.apache.org/jira/browse/HBASE-5974
 Project: HBase
  Issue Type: Bug
  Components: client, regionserver
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Todd Lipcon
Assignee: Anoop Sam John
Priority: Critical
 Fix For: 0.96.0, 0.94.1

 Attachments: 5974_94-V4.patch, 5974_trunk.patch, 
 HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch, HBASE-5974_94-V3.patch


 I'm seeing the following behavior:
 - set RPC timeout to a short value
 - call next() for some batch of rows, big enough so the client times out 
 before the result is returned
 - the HConnectionManager stuff will retry the next() call to the same server. 
 At this point, one of two things can happen: 1) the previous next() call will 
 still be processing, in which case you get a LeaseException, because it was 
 removed from the map during the processing, or 2) the next() call will 
 succeed but skip the prior batch of rows.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect


 [ 
https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-5974:
--

Fix Version/s: 0.96.0
   Status: Patch Available  (was: Open)

 Scanner retry behavior with RPC timeout on next() seems incorrect
 -

 Key: HBASE-5974
 URL: https://issues.apache.org/jira/browse/HBASE-5974
 Project: HBase
  Issue Type: Bug
  Components: client, regionserver
Affects Versions: 0.94.0, 0.92.1, 0.90.7, 0.96.0
Reporter: Todd Lipcon
Assignee: Anoop Sam John
Priority: Critical
 Fix For: 0.96.0, 0.94.1

 Attachments: 5974_94-V4.patch, 5974_trunk.patch, 
 HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch, HBASE-5974_94-V3.patch


 I'm seeing the following behavior:
 - set RPC timeout to a short value
 - call next() for some batch of rows, big enough so the client times out 
 before the result is returned
 - the HConnectionManager stuff will retry the next() call to the same server. 
 At this point, one of two things can happen: 1) the previous next() call will 
 still be processing, in which case you get a LeaseException, because it was 
 removed from the map during the processing, or 2) the next() call will 
 succeed but skip the prior batch of rows.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect


[ 
https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288586#comment-13288586
 ] 

Anoop Sam John commented on HBASE-5974:
---

In Trunk InterfaceAudience for RegionScanner to be public I feel ( as per the 
use case that we are having. We wants to impl this interface wrapping the 
actual RegionScanner object created by the Region )..  Not made this change in 
the attached patch. Pls provide suggestions

 Scanner retry behavior with RPC timeout on next() seems incorrect
 -

 Key: HBASE-5974
 URL: https://issues.apache.org/jira/browse/HBASE-5974
 Project: HBase
  Issue Type: Bug
  Components: client, regionserver
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Todd Lipcon
Assignee: Anoop Sam John
Priority: Critical
 Fix For: 0.96.0, 0.94.1

 Attachments: 5974_94-V4.patch, 5974_trunk.patch, 
 HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch, HBASE-5974_94-V3.patch


 I'm seeing the following behavior:
 - set RPC timeout to a short value
 - call next() for some batch of rows, big enough so the client times out 
 before the result is returned
 - the HConnectionManager stuff will retry the next() call to the same server. 
 At this point, one of two things can happen: 1) the previous next() call will 
 still be processing, in which case you get a LeaseException, because it was 
 removed from the map during the processing, or 2) the next() call will 
 succeed but skip the prior batch of rows.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5699) Run with 1 WAL in HRegionServer


[ 
https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288592#comment-13288592
 ] 

Zhihong Ted Yu commented on HBASE-5699:
---

Can you run ycsb with 50% insert and 50% update load ?
Performance numbers in attachment match what I got based on my implementation.

Thanks

 Run with  1 WAL in HRegionServer
 -

 Key: HBASE-5699
 URL: https://issues.apache.org/jira/browse/HBASE-5699
 Project: HBase
  Issue Type: Improvement
Reporter: binlijin
Assignee: Li Pi
 Attachments: PerfHbase.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6060) Regions's in OPENING state from failed regionservers takes a long time to recover

[
https://issues.apache.org/jira/browse/HBASE-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288593#comment-13288593
]

ramkrishna.s.vasudevan commented on HBASE-6060:
---

@Ted
In trunk if we take up this patch we need to solve HBASE-6012. Without that
master will get aborted. But i feel the current patch in HBASE-6012 needs more
testing.

Regions's in OPENING state from failed regionservers takes a long time to
recover
-

Key: HBASE-6060
URL: https://issues.apache.org/jira/browse/HBASE-6060
Project: HBase
Issue Type: Bug
Components: master, regionserver
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.96.0, 0.94.1, 0.92.3

Attachments: 6060-94-v3.patch, 6060-94-v4.patch, 6060-94-v4_1.patch,
6060-94-v4_1.patch, 6060-trunk.patch, 6060-trunk.patch, 6060-trunk_2.patch,
HBASE-6060-94.patch

we have seen a pattern in tests, that the regions are stuck in OPENING state
for a very long time when the region server who is opening the region fails.
My understanding of the process:

- master calls rs to open the region. If rs is offline, a new plan is
generated (a new rs is chosen). RegionState is set to PENDING_OPEN (only in
master memory, zk still shows OFFLINE). See HRegionServer.openRegion(),
HMaster.assign()
- RegionServer, starts opening a region, changes the state in znode. But
that znode is not ephemeral. (see ZkAssign)
- Rs transitions zk node from OFFLINE to OPENING. See
OpenRegionHandler.process()
- rs then opens the region, and changes znode from OPENING to OPENED
- when rs is killed between OPENING and OPENED states, then zk shows OPENING
state, and the master just waits for rs to change the region state, but since
rs is down, that wont happen.
- There is a AssignmentManager.TimeoutMonitor, which does exactly guard
against these kind of conditions. It periodically checks (every 10 sec by
default) the regions in transition to see whether they timedout
(hbase.master.assignment.timeoutmonitor.timeout). Default timeout is 30 min,
which explains what you and I are seeing.
- ServerShutdownHandler in Master does not reassign regions in OPENING
state, although it handles other states.
Lowering that threshold from the configuration is one option, but still I
think we can do better.
Will investigate more.

[jira] [Commented] (HBASE-6012) AssignmentManager#asyncSetOfflineInZooKeeper wouldn't force node offline


[ 
https://issues.apache.org/jira/browse/HBASE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288597#comment-13288597
 ] 

ramkrishna.s.vasudevan commented on HBASE-6012:
---

@Chunhui
But deleting the node, what improvment we get here? Still the scenario that i 
was mentioning may happen right? 
Correct me if am wrong.

 AssignmentManager#asyncSetOfflineInZooKeeper wouldn't force node offline
 

 Key: HBASE-6012
 URL: https://issues.apache.org/jira/browse/HBASE-6012
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
 Fix For: 0.96.0

 Attachments: HBASE-6012.patch, HBASE-6012v2.patch


 As the javadoc of method and the log message
 {code}
 /**
* Set region as OFFLINED up in zookeeper asynchronously.
*/
 boolean asyncSetOfflineInZooKeeper(
 ...
 master.abort(Unexpected ZK exception creating/setting node OFFLINE, e);
 ...
 }
 {code}
 I think AssignmentManager#asyncSetOfflineInZooKeeper should also force node 
 offline, just like AssignmentManager#setOfflineInZooKeeper do. Otherwise, it 
 may cause bulk assign failed which called this method.
 Error log on the master caused by the issue
 2012-05-12 01:40:09,437 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; 
 was=writetest,1YTQDPGLXBTICHOPQ6IL,1336590857771.674da422fc7cb9a7d42c74499ace1d93.
  state=PENDING_CLOSE, ts=1336757876856 
 2012-05-12 01:40:09,437 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:6-0x23736bf74780082 Async create of unassigned node for 
 674da422fc7cb9a7d42c74499ace1d93 with OFFLINE state 
 2012-05-12 01:40:09,446 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback:
  rc != 0 for /hbase-func1/unassigned/674da422fc7cb9a7d42c74499ace1d93 -- 
 retryable connectionloss -- FIX see 
 http://wiki.apache.org/hadoop/ZooKeeper/FAQ#A2 
 2012-05-12 01:40:09,447 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Connectionloss writing unassigned at 
 /hbase-func1/unassigned/674da422fc7cb9a7d42c74499ace1d93, rc=-110 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6060) Regions's in OPENING state from failed regionservers takes a long time to recover

[
https://issues.apache.org/jira/browse/HBASE-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288599#comment-13288599
]

ramkrishna.s.vasudevan commented on HBASE-6060:
---

I also feel if this patch is fine we can commit the patch to trunk also and
work on HBASe-6012 and HBASE-6147 to solve such scenarios mentioned in the
above comment.

Regions's in OPENING state from failed regionservers takes a long time to
recover
-

Attachments: 6060-94-v3.patch, 6060-94-v4.patch, 6060-94-v4_1.patch,
6060-94-v4_1.patch, 6060-trunk.patch, 6060-trunk.patch, 6060-trunk_2.patch,
HBASE-6060-94.patch

we have seen a pattern in tests, that the regions are stuck in OPENING state
for a very long time when the region server who is opening the region fails.
My understanding of the process:

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288606#comment-13288606
 ] 

Zhihong Ted Yu commented on HBASE-6067:
---

@Daryn:
Looking at:
http://docs.oracle.com/javase/1.4.2/docs/api/java/lang/Class.html#getDeclaredMethod%28java.lang.String,%20java.lang.Class[]%29
I don't see why getDeclaredMethod wouldn't find the desired method from this.fs

w.r.t. setAccessible(), I didn't include it in patch v1 for this reason.
Then I found that getGetNumCurrentReplicas() was using the call. And:
{code}
public int getNumCurrentReplicas() throws IOException {
./src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
{code}

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6060) Regions's in OPENING state from failed regionservers takes a long time to recover

[
https://issues.apache.org/jira/browse/HBASE-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288608#comment-13288608
]

Zhihong Ted Yu commented on HBASE-6060:
---

Agreed.
We can integrate once patch for 0.92 is ready.

Regions's in OPENING state from failed regionservers takes a long time to
recover
-

Attachments: 6060-94-v3.patch, 6060-94-v4.patch, 6060-94-v4_1.patch,
6060-94-v4_1.patch, 6060-trunk.patch, 6060-trunk.patch, 6060-trunk_2.patch,
HBASE-6060-94.patch

we have seen a pattern in tests, that the regions are stuck in OPENING state
for a very long time when the region server who is opening the region fails.
My understanding of the process:

[jira] [Commented] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect

[
https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288617#comment-13288617
]

Hadoop QA commented on HBASE-5974:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12530781/5974_trunk.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2097//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2097//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2097//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2097//console

This message is automatically generated.

Scanner retry behavior with RPC timeout on next() seems incorrect
-

Key: HBASE-5974
URL: https://issues.apache.org/jira/browse/HBASE-5974
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Todd Lipcon
Assignee: Anoop Sam John
Priority: Critical
Fix For: 0.96.0, 0.94.1

Attachments: 5974_94-V4.patch, 5974_trunk.patch,
HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch, HBASE-5974_94-V3.patch

I'm seeing the following behavior:
- set RPC timeout to a short value
- call next() for some batch of rows, big enough so the client times out
before the result is returned
- the HConnectionManager stuff will retry the next() call to the same server.
At this point, one of two things can happen: 1) the previous next() call will
still be processing, in which case you get a LeaseException, because it was
removed from the map during the processing, or 2) the next() call will
succeed but skip the prior batch of rows.

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288636#comment-13288636
 ] 

Zhihong Ted Yu commented on HBASE-6067:
---

@Stack:
Do you think this JIRA should be in 0.92.2 RC ?

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect


[ 
https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288644#comment-13288644
 ] 

Zhihong Ted Yu commented on HBASE-5974:
---

I would listen to Todd and Andy's opinion.

 Scanner retry behavior with RPC timeout on next() seems incorrect
 -

 Key: HBASE-5974
 URL: https://issues.apache.org/jira/browse/HBASE-5974
 Project: HBase
  Issue Type: Bug
  Components: client, regionserver
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Todd Lipcon
Assignee: Anoop Sam John
Priority: Critical
 Fix For: 0.96.0, 0.94.1

 Attachments: 5974_94-V4.patch, 5974_trunk.patch, 
 HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch, HBASE-5974_94-V3.patch


 I'm seeing the following behavior:
 - set RPC timeout to a short value
 - call next() for some batch of rows, big enough so the client times out 
 before the result is returned
 - the HConnectionManager stuff will retry the next() call to the same server. 
 At this point, one of two things can happen: 1) the previous next() call will 
 still be processing, in which case you get a LeaseException, because it was 
 removed from the map during the processing, or 2) the next() call will 
 succeed but skip the prior batch of rows.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics


 [ 
https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6131:
-

Attachment: 6131_092.txt

What I applied to 0.92 (Didn't realize the metrics patch had gone into 0.92).

 Add attribution for code added by HBASE-5533 metrics
 

 Key: HBASE-6131
 URL: https://issues.apache.org/jira/browse/HBASE-6131
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.94.1

 Attachments: 6131.txt, 6131_092.txt, 6131_094.txt


 See the comment over in 
 https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920
 The metrics histogram code was copied w/o attribution.  Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics


[ 
https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288671#comment-13288671
 ] 

stack commented on HBASE-6131:
--

I committed to 0.92.  I apologized to Mr. Hale on twitter about a week ago.  He 
was gracious in his response.  Let me roll a new 0.92.2RC.

 Add attribution for code added by HBASE-5533 metrics
 

 Key: HBASE-6131
 URL: https://issues.apache.org/jira/browse/HBASE-6131
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.94.1

 Attachments: 6131.txt, 6131_092.txt, 6131_094.txt


 See the comment over in 
 https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920
 The metrics histogram code was copied w/o attribution.  Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6145) Fix site target post modularization


[ 
https://issues.apache.org/jira/browse/HBASE-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288677#comment-13288677
 ] 

stack commented on HBASE-6145:
--

I reenabled site building and packaging up on jenkins.

Let me know Jesse if you think we need to add other stuff from this issue to 
hbase-6154, the follow-on issue.  Thanks.

 Fix site target post modularization
 ---

 Key: HBASE-6145
 URL: https://issues.apache.org/jira/browse/HBASE-6145
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
 Attachments: 6145v4.txt, 6145v4.txt, site.txt, site2.txt, sitev3.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6060) Regions's in OPENING state from failed regionservers takes a long time to recover

2012-06-04 Thread rajeshbabu (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

rajeshbabu updated HBASE-6060:
--

Attachment: HBASE-6060-92.patch

Regions's in OPENING state from failed regionservers takes a long time to
recover
-

Attachments: 6060-94-v3.patch, 6060-94-v4.patch, 6060-94-v4_1.patch,
6060-94-v4_1.patch, 6060-trunk.patch, 6060-trunk.patch, 6060-trunk_2.patch,
HBASE-6060-92.patch, HBASE-6060-94.patch

we have seen a pattern in tests, that the regions are stuck in OPENING state
for a very long time when the region server who is opening the region fails.
My understanding of the process:

[jira] [Commented] (HBASE-6060) Regions's in OPENING state from failed regionservers takes a long time to recover

2012-06-04 Thread rajeshbabu (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288679#comment-13288679
]

rajeshbabu commented on HBASE-6060:
---

Patch for 92. Running test suite.

Regions's in OPENING state from failed regionservers takes a long time to
recover
-

Attachments: 6060-94-v3.patch, 6060-94-v4.patch, 6060-94-v4_1.patch,
6060-94-v4_1.patch, 6060-trunk.patch, 6060-trunk.patch, 6060-trunk_2.patch,
HBASE-6060-92.patch, HBASE-6060-94.patch

we have seen a pattern in tests, that the regions are stuck in OPENING state
for a very long time when the region server who is opening the region fails.
My understanding of the process:

[jira] [Commented] (HBASE-6145) Fix site target post modularization


[ 
https://issues.apache.org/jira/browse/HBASE-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288690#comment-13288690
 ] 

Jesse Yates commented on HBASE-6145:


The main thing I want to ensure is that we don't have a regression of 
HBASE-6138. I'll look at trunk soon (today?) to see if there is anything I 
wanna change.

 Fix site target post modularization
 ---

 Key: HBASE-6145
 URL: https://issues.apache.org/jira/browse/HBASE-6145
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
 Attachments: 6145v4.txt, 6145v4.txt, site.txt, site2.txt, sitev3.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6060) Regions's in OPENING state from failed regionservers takes a long time to recover

[
https://issues.apache.org/jira/browse/HBASE-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288695#comment-13288695
]

Hadoop QA commented on HBASE-6060:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12530816/HBASE-6060-92.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2098//console

This message is automatically generated.

Regions's in OPENING state from failed regionservers takes a long time to
recover
-

Attachments: 6060-94-v3.patch, 6060-94-v4.patch, 6060-94-v4_1.patch,
6060-94-v4_1.patch, 6060-trunk.patch, 6060-trunk.patch, 6060-trunk_2.patch,
HBASE-6060-92.patch, HBASE-6060-94.patch

we have seen a pattern in tests, that the regions are stuck in OPENING state
for a very long time when the region server who is opening the region fails.
My understanding of the process:

[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96


[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288696#comment-13288696
 ] 

Jesse Yates commented on HBASE-6055:


@Lars even with the HDFS patches, we should have a way to logically 
group/backup/restore snapshots. 

IMO hardlinks with HBase snapshots is the way to go, since we can do it with 
zero downtime, whereas HDFS snapshots require some (though admittedly small) 
downtime. HBSE-5547 is basically just a hack around hardlinks.

 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: client, master, regionserver, zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5699) Run with 1 WAL in HRegionServer


[ 
https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288699#comment-13288699
 ] 

Lars Hofhansl commented on HBASE-5699:
--

@Ted or @Ram: If you have any chance to test HBASE-6116 as well, that'd be 
really cool (although it would be more effort, as it only works against Hadoop 
trunk - and soon Hadoop 2.0-alpha).
Andy said he might test against EC2.

 Run with  1 WAL in HRegionServer
 -

 Key: HBASE-5699
 URL: https://issues.apache.org/jira/browse/HBASE-5699
 Project: HBase
  Issue Type: Improvement
Reporter: binlijin
Assignee: Li Pi
 Attachments: PerfHbase.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96


[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288700#comment-13288700
 ] 

Lars Hofhansl commented on HBASE-6055:
--

bq. IMO hardlinks with HBase snapshots is the way to go... HBSE-5547 is 
basically just a hack around hardlinks

Yep.

 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: client, master, regionserver, zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem

2012-06-04 Thread Daryn Sharp (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288708#comment-13288708
 ] 

Daryn Sharp commented on HBASE-6067:


This code demonstrates that {{getDeclaredMethod}} only returns a method 
explicitly defined in the class.
{code}
import java.lang.reflect.*;

public class Test {
  public static void main(String[] args) throws Exception {
String m;
try {
  m = B.class.getDeclaredMethod(method).toString();
} catch (Exception e) {
  m = e.toString();
}
System.out.println(getDeclaredMethod: +m);
try {
  m = B.class.getMethod(method).toString();
} catch (Exception e) {
  m = e.toString();
}
System.out.println(getMethod: +m);
  }
  static class A { public void method() {} }
  static class B extends A { }
}
{code}

Produces the output:
{code}
getDeclaredMethod: java.lang.NoSuchMethodException: Test$B.method()
getMethod: public void Test$A.method()
{code}

Per the docs and my testing, the get method calls only return public methods, 
so the {{setAccessible}} is a no-op.

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96

[
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288715#comment-13288715
]

Jesse Yates commented on HBASE-6055:

A couple of definitions going forward:
- materialization: the end result of taking a single snapshot, on the same
cluster. It ends up in in the .snapshot/[snapshot_name] directory
- export: sending the snapshot to another cluster or another part of the same
cluster
- restore: taking an exported snapshot and converting the snapshot into an
active table.

{quote}
Hm.. how do you restore a snapshot from references files if it hasn't been
scan/copied yet? Require scan/copy materialization of the snapshot first?
(which means slower restore, but probably would likely be simplest for a first
cut)
{quote}

Right now, you would do a M/R job to distcp over the files to another cluster
or a backup part of your cluster. Since we are just storing references, the
actual file copying will be necessary. This will be helped by using the actual
Reference class for the HFiles (and currently being (mis)used for the WALs,
but I don't think we actually need to keep the WALs - I'll comment in the
timestamp ticket). Since they are just reference files, you could just use the
regular HFile reader to load them into another table.

{quote}
Snapshot restore needs to be transactional like snapshotting right?
{quote}

Yeah, I guess. I don't really see this as a problem - just keep it to one
restore at a time. But it would be all or nothing to get a table online.

{quote}
what is export? is this taking a snapshot or the materialization or the
snapshot restore or something else?
{quote}

Export is taking a snapshot from the .snapshot/ directory and possibly having a
special snapshot distcp to somewhere. I would consider materialization as
taking the exported snapshot and then 'hooking it back up' to another cluster
(or the same) as a new table. You could throw materialization of the exported
snapshot, but they are in fact distinct.
{quote}
If we restore snapshots to the same hbase instance, in dir structure, you
probably need .regioninfo files as well. (contains region startkey/endkey info
necessary to reconsistute META later).
{quote}

+1 I'll make sure that gets in
{quote}
Is restoring to a separate instance in scope? If so bulk loads can be expensive
– if regions don't line up there will be a bunch of spliting that happens.
Again, keeping the regionsinfos and the snapshot's splits may be worthwhile.
{quote}

I'd say restore is part of this. Should be solved by having the region info. -1
for split/compact storms.

{quote}
Where do the materialized versions of the snapshot reference files end up? in
the snapshot dirs? elsewhere?
{quote}

What do you mean materialized? After taking snapshot, where do the snapshot
files end up? In the .snapshot directory. See my earlier comments on the
structure.

{quote}
This potentially gets a little trickier with markers as opposed to log rolls.
{quote}
If we do a log roll, its probably going to take a bit longer. Also, its not
going to be applicable to the timestamp approach, since log rolling will
necessitate doing some kind of locking, which we should avoid, where the
markers will be much faster.

{quote}
The HLog will have edits from regions not relevant to the table's regions. Not
a huge problem but maybe an optmization would be that the materialization step
will do an offline hlogsplit/flush to just keep the data relevent to this
table/region?
{quote}

+1, assuming we need the HLogs. I think there is a minimally impactful way to
avoid this altogether.

Snapshots in HBase 0.96
---

Attachments: Snapshots in HBase.docx

Continuation of HBASE-50 for the current trunk. Since the implementation has
drastically changed, opening as a new ticket.

[jira] [Updated] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.

2012-06-04 Thread Andrew Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang updated HBASE-5892:
---

Attachment: (was: hbase-5892-4.patch)

 [hbck] Refactor parallel WorkItem* to Futures.
 --

 Key: HBASE-5892
 URL: https://issues.apache.org/jira/browse/HBASE-5892
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Andrew Wang
  Labels: noob
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, 
 hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892-4-0.90.patch, 
 hbase-5892-4.patch, hbase-5892.patch


 This would convert WorkItem* logic (with low level notifies, and rough 
 exception handling)  into a more canonical Futures pattern.
 Currently there are two instances of this pattern (for loading hdfs dirs, for 
 contacting regionservers for assignments, and soon -- for loading hdfs 
 .regioninfo files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.

2012-06-04 Thread Andrew Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang updated HBASE-5892:
---

Attachment: hbase-5892-4.patch

 [hbck] Refactor parallel WorkItem* to Futures.
 --

 Key: HBASE-5892
 URL: https://issues.apache.org/jira/browse/HBASE-5892
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Andrew Wang
  Labels: noob
 Fix For: 0.90.7, 0.96.0, 0.94.1, 0.92.3

 Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, 
 hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892-4-0.90.patch, 
 hbase-5892-4.patch, hbase-5892.patch


 This would convert WorkItem* logic (with low level notifies, and rough 
 exception handling)  into a more canonical Futures pattern.
 Currently there are two instances of this pattern (for loading hdfs dirs, for 
 contacting regionservers for assignments, and soon -- for loading hdfs 
 .regioninfo files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6046) Master retry on ZK session expiry causes inconsistent region assignments.


 [ 
https://issues.apache.org/jira/browse/HBASE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-6046:
--

Status: Open  (was: Patch Available)

 Master retry on ZK session expiry causes inconsistent region assignments.
 -

 Key: HBASE-6046
 URL: https://issues.apache.org/jira/browse/HBASE-6046
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.94.0, 0.92.1
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
 Attachments: HBASE_6046_0.94.patch, HBASE_6046_0.94_1.patch, 
 HBASE_6046_0.94_2.patch


 1 ZK Session timeout in the hmaster leads to bulk assignment though all the 
 RSs are online.
 2 While doing bulk assignment, if the master again goes down  restart(or 
 backup comes up) all the node created in the ZK will now be tried to reassign 
 to the new RSs. This is leading to double assignment.
 we had 2800 regions, among this 1900 region got double assignment, taking the 
 region count to 4700. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96


[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288733#comment-13288733
 ] 

Zhihong Ted Yu commented on HBASE-6055:
---

bq. The HLog will have edits from regions not relevant to the table's regions.
Over in HBASE-5699, each one of the multiple WALs can be devised to receive 
edits from single table.

 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: client, master, regionserver, zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


 [ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu updated HBASE-6067:
--

Attachment: 6067-addendum.txt

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6046) Master retry on ZK session expiry causes inconsistent region assignments.


 [ 
https://issues.apache.org/jira/browse/HBASE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-6046:
--

Attachment: HBASE_6046_0.94_3.patch

 Master retry on ZK session expiry causes inconsistent region assignments.
 -

 Key: HBASE-6046
 URL: https://issues.apache.org/jira/browse/HBASE-6046
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.1, 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
 Attachments: HBASE_6046_0.94.patch, HBASE_6046_0.94_1.patch, 
 HBASE_6046_0.94_2.patch, HBASE_6046_0.94_3.patch


 1 ZK Session timeout in the hmaster leads to bulk assignment though all the 
 RSs are online.
 2 While doing bulk assignment, if the master again goes down  restart(or 
 backup comes up) all the node created in the ZK will now be tried to reassign 
 to the new RSs. This is leading to double assignment.
 we had 2800 regions, among this 1900 region got double assignment, taking the 
 region count to 4700. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288740#comment-13288740
 ] 

Zhihong Ted Yu commented on HBASE-6067:
---

Addendum integrated to 0.94 and trunk.

Thanks for the tip, Daryn.

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6046) Master retry on ZK session expiry causes inconsistent region assignments.


 [ 
https://issues.apache.org/jira/browse/HBASE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-6046:
--

Attachment: HBASE_6046-trunk.patch

Updated patch for 0.94 and trunk.
New ServerManager is not created.  This is needed because the Timeout monitor 
in AM and splitlogmanager uses the server manager.

 Master retry on ZK session expiry causes inconsistent region assignments.
 -

 Key: HBASE-6046
 URL: https://issues.apache.org/jira/browse/HBASE-6046
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.1, 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
 Attachments: HBASE_6046-trunk.patch, HBASE_6046_0.94.patch, 
 HBASE_6046_0.94_1.patch, HBASE_6046_0.94_2.patch, HBASE_6046_0.94_3.patch


 1 ZK Session timeout in the hmaster leads to bulk assignment though all the 
 RSs are online.
 2 While doing bulk assignment, if the master again goes down  restart(or 
 backup comes up) all the node created in the ZK will now be tried to reassign 
 to the new RSs. This is leading to double assignment.
 we had 2800 regions, among this 1900 region got double assignment, taking the 
 region count to 4700. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6046) Master retry on ZK session expiry causes inconsistent region assignments.


 [ 
https://issues.apache.org/jira/browse/HBASE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-6046:
--

Status: Patch Available  (was: Open)

 Master retry on ZK session expiry causes inconsistent region assignments.
 -

 Key: HBASE-6046
 URL: https://issues.apache.org/jira/browse/HBASE-6046
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.94.0, 0.92.1
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
 Attachments: HBASE_6046-trunk.patch, HBASE_6046_0.94.patch, 
 HBASE_6046_0.94_1.patch, HBASE_6046_0.94_2.patch, HBASE_6046_0.94_3.patch


 1 ZK Session timeout in the hmaster leads to bulk assignment though all the 
 RSs are online.
 2 While doing bulk assignment, if the master again goes down  restart(or 
 backup comes up) all the node created in the ZK will now be tried to reassign 
 to the new RSs. This is leading to double assignment.
 we had 2800 regions, among this 1900 region got double assignment, taking the 
 region count to 4700. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem

[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288755#comment-13288755
]

Hadoop QA commented on HBASE-6067:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12530821/6067-addendum.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2100//console

This message is automatically generated.

HBase won't start when hbase.rootdir uses ViewFileSystem

Key: HBASE-6067
URL: https://issues.apache.org/jira/browse/HBASE-6067
Project: HBase
Issue Type: Improvement
Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
Fix For: 0.96.0, 0.94.1, 0.92.3

Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt

HBase currently doesn't work with HDFS federation (hbase.rootdir with a
client that uses viewfs) because HLog#init uses
FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
exception because there is no default filesystem in a viewfs client so
there's no way to determine a default block size or replication factor. They
could use the versions of these methods that take a path, however these were
introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

[jira] [Commented] (HBASE-6046) Master retry on ZK session expiry causes inconsistent region assignments.


[ 
https://issues.apache.org/jira/browse/HBASE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288756#comment-13288756
 ] 

Zhihong Ted Yu commented on HBASE-6046:
---

bq. New ServerManager is not created
I think there is a typo above: 'not' - 'now'

 Master retry on ZK session expiry causes inconsistent region assignments.
 -

 Key: HBASE-6046
 URL: https://issues.apache.org/jira/browse/HBASE-6046
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.1, 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
 Attachments: HBASE_6046-trunk.patch, HBASE_6046_0.94.patch, 
 HBASE_6046_0.94_1.patch, HBASE_6046_0.94_2.patch, HBASE_6046_0.94_3.patch


 1 ZK Session timeout in the hmaster leads to bulk assignment though all the 
 RSs are online.
 2 While doing bulk assignment, if the master again goes down  restart(or 
 backup comes up) all the node created in the ZK will now be tried to reassign 
 to the new RSs. This is leading to double assignment.
 we had 2800 regions, among this 1900 region got double assignment, taking the 
 region count to 4700. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics


 [ 
https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu updated HBASE-6131:
--

Fix Version/s: 0.96.0
   0.92.2

 Add attribution for code added by HBASE-5533 metrics
 

 Key: HBASE-6131
 URL: https://issues.apache.org/jira/browse/HBASE-6131
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.92.2, 0.96.0, 0.94.1

 Attachments: 6131.txt, 6131_092.txt, 6131_094.txt


 See the comment over in 
 https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920
 The metrics histogram code was copied w/o attribution.  Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name

Aditya Kishore created HBASE-6158:
-

 Summary: Data loss if the words 'merges' or 'splits' are used as 
Column Family name
 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore


If a table is creates with either 'merges' or 'splits' as one of the Column 
Family name it can never be flushed to the disk even though the table creation 
(and data population) succeeds.

The reason for this is that these two are used as temporary directory names 
inside the region folder or merge and splits respectively and hence conflicts 
with the directories created for CF with same name.

A simple fix would be to uses .merges' and .splits as the working folder 
(patch attached). This will also be consistent with other work folder names. An 
alternate fix would be to declare these words (and other similar) as reserve 
words and throw exception when they are used. However, I do find the alternate 
approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


 [ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aditya Kishore updated HBASE-6158:
--

Attachment: HBASE-6158.patch

Rename the merge and split work folder names by prefixing '.'

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
 Attachments: HBASE-6158.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


 [ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu reassigned HBASE-6158:
-

Assignee: Aditya Kishore

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Attachments: HBASE-6158.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


[ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288769#comment-13288769
 ] 

Zhihong Ted Yu commented on HBASE-6158:
---

Can you generate patch for trunk ?
{code}
-  static final String MERGEDIR = merges;
+  static final String MERGEDIR = .merges;
{code}
The above constant is only used in HRegion. We can make it private, right ?

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-6158.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


 [ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu updated HBASE-6158:
--

Fix Version/s: 0.94.1
   0.96.0

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-6158.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6159) truncate leaks ZK connection in HBase shell

2012-06-04 Thread Richard Ding (JIRA)

Richard Ding created HBASE-6159:
---

 Summary: truncate leaks ZK connection in HBase shell
 Key: HBASE-6159
 URL: https://issues.apache.org/jira/browse/HBASE-6159
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 0.90.6
Reporter: Richard Ding
Assignee: Richard Ding
Priority: Minor


ZK connection count always goes up by one when running shell command:

{code}
hbase(main):001:0 truncate 'table'
{code}

This can be fixed with the change in admin.rb:

{code}
- h_table = org.apache.hadoop.hbase.client.HTable.new(table_name)
- table_description = h_table.getTableDescriptor()
+ table_description = @admin.getTableDescriptor(table_name.to_java_bytes)
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


 [ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aditya Kishore updated HBASE-6158:
--

Attachment: (was: HBASE-6158.patch)

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


 [ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu updated HBASE-6158:
--

Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


[ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288792#comment-13288792
 ] 

stack commented on HBASE-6158:
--

+1

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6000) Cleanup where we keep .proto files

2012-06-04 Thread Gregory Chanan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288798#comment-13288798
 ] 

Gregory Chanan commented on HBASE-6000:
---

It looks like we have things under src/protobuf and src/main/protobuf, in 
addition to the definitions under resources.

 Cleanup where we keep .proto files
 --

 Key: HBASE-6000
 URL: https://issues.apache.org/jira/browse/HBASE-6000
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: stack
Assignee: stack
 Attachments: 6000.txt, 6000.txt


 I see Andrew for his pb work over in rest has .protos files under 
 src/main/resources.  We should unify where these files live.  The recently 
 added .protos place them under src/main/protobuf  Its confusing.
 The thift idl files are here under resources too.
 Seems like we should move src/main/protobuf under src/resources to be 
 consistent.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5533) Add more metrics to HBase


[ 
https://issues.apache.org/jira/browse/HBASE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288799#comment-13288799
 ] 

Hudson commented on HBASE-5533:
---

Integrated in HBase-0.92 #444 (See 
[https://builds.apache.org/job/HBase-0.92/444/])
HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 
1346033)

 Result = FAILURE
stack : 
Files : 
* /hbase/branches/0.92/pom.xml
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/MetricsHistogram.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/metrics/TestExponentiallyDecayingSample.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/metrics/TestMetricsHistogram.java


 Add more metrics to HBase
 -

 Key: HBASE-5533
 URL: https://issues.apache.org/jira/browse/HBASE-5533
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.2, 0.94.0
Reporter: Shaneal Manek
Assignee: Shaneal Manek
Priority: Minor
 Fix For: 0.92.2, 0.94.0, 0.96.0

 Attachments: BlockingQueueContention.java, HBASE-5533-0.92-v4.patch, 
 HBASE-5533-TRUNK-v6.patch, HBASE-5533-TRUNK-v6.patch, 
 HBASE-5533-v7-0.92.patch, TimingOverhead.java, hbase-5533-0.92.patch, 
 hbase5533-0.92-v2.patch, hbase5533-0.92-v3.patch, hbase5533-0.92-v5.patch, 
 histogram_web_ui.png


 To debug/monitor production clusters, there are some more metrics I wish I 
 had available.
 In particular:
 - Although the average FS latencies are useful, a 'histogram' of recent 
 latencies (90% of reads completed in under 100ms, 99% in under 200ms, etc) 
 would be more useful
 - Similar histograms of latencies on common operations (GET, PUT, DELETE) 
 would be useful
 - Counting the number of accesses to each region to detect hotspotting
 - Exposing the current number of HLog files

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6159) truncate leaks ZK connection in HBase shell


 [ 
https://issues.apache.org/jira/browse/HBASE-6159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6159:
-

Attachment: 6159.txt

Here is Richards comment as a patch.  +1 on commit.

 truncate leaks ZK connection in HBase shell
 ---

 Key: HBASE-6159
 URL: https://issues.apache.org/jira/browse/HBASE-6159
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 0.90.6
Reporter: Richard Ding
Assignee: Richard Ding
Priority: Minor
 Attachments: 6159.txt


 ZK connection count always goes up by one when running shell command:
 {code}
 hbase(main):001:0 truncate 'table'
 {code}
 This can be fixed with the change in admin.rb:
 {code}
 - h_table = org.apache.hadoop.hbase.client.HTable.new(table_name)
 - table_description = h_table.getTableDescriptor()
 + table_description = @admin.getTableDescriptor(table_name.to_java_bytes)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288803#comment-13288803
 ] 

Hudson commented on HBASE-6067:
---

Integrated in HBase-TRUNK #2985 (See 
[https://builds.apache.org/job/HBase-TRUNK/2985/])
HBASE-6067 Addendum adopts suggestions from Daryn (Revision 1346057)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java


 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6159) truncate leaks ZK connection in HBase shell


 [ 
https://issues.apache.org/jira/browse/HBASE-6159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6159:
-

Status: Patch Available  (was: Open)

 truncate leaks ZK connection in HBase shell
 ---

 Key: HBASE-6159
 URL: https://issues.apache.org/jira/browse/HBASE-6159
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 0.90.6
Reporter: Richard Ding
Assignee: Richard Ding
Priority: Minor
 Attachments: 6159.txt


 ZK connection count always goes up by one when running shell command:
 {code}
 hbase(main):001:0 truncate 'table'
 {code}
 This can be fixed with the change in admin.rb:
 {code}
 - h_table = org.apache.hadoop.hbase.client.HTable.new(table_name)
 - table_description = h_table.getTableDescriptor()
 + table_description = @admin.getTableDescriptor(table_name.to_java_bytes)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics


[ 
https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288800#comment-13288800
 ] 

Hudson commented on HBASE-6131:
---

Integrated in HBase-0.92 #444 (See 
[https://builds.apache.org/job/HBase-0.92/444/])
HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 
1346033)

 Result = FAILURE
stack : 
Files : 
* /hbase/branches/0.92/pom.xml
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/MetricsHistogram.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/metrics/TestExponentiallyDecayingSample.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/metrics/TestMetricsHistogram.java


 Add attribution for code added by HBASE-5533 metrics
 

 Key: HBASE-6131
 URL: https://issues.apache.org/jira/browse/HBASE-6131
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.92.2, 0.96.0, 0.94.1

 Attachments: 6131.txt, 6131_092.txt, 6131_094.txt


 See the comment over in 
 https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920
 The metrics histogram code was copied w/o attribution.  Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288825#comment-13288825
 ] 

stack commented on HBASE-6067:
--

+1 on getting it into 0.92.2.  I'll wait on it before cutting next RC

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


[ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288828#comment-13288828
 ] 

Hudson commented on HBASE-6067:
---

Integrated in HBase-0.94 #245 (See 
[https://builds.apache.org/job/HBase-0.94/245/])
HBASE-6067 Addendum adopts suggestions from Daryn (Revision 1346055)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java


 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.96.0, 0.94.1, 0.92.3

 Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name

[
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-6158:
-

Resolution: Fixed
Status: Resolved (was: Patch Available)

Applied trunk and 0.94 branch. Thanks for the patch Aditya.

Data loss if the words 'merges' or 'splits' are used as Column Family name
--

Key: HBASE-6158
URL: https://issues.apache.org/jira/browse/HBASE-6158
Project: HBase
Issue Type: Bug
Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Fix For: 0.96.0, 0.94.1

Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch

If a table is creates with either 'merges' or 'splits' as one of the Column
Family name it can never be flushed to the disk even though the table
creation (and data population) succeeds.
The reason for this is that these two are used as temporary directory names
inside the region folder or merge and splits respectively and hence conflicts
with the directories created for CF with same name.
A simple fix would be to uses .merges' and .splits as the working folder
(patch attached). This will also be consistent with other work folder names.
An alternate fix would be to declare these words (and other similar) as
reserve words and throw exception when they are used. However, I do find the
alternate approach as unnecessarily restrictive.

[jira] [Updated] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


 [ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6158:
-

Fix Version/s: (was: 0.96.0)
   0.92.2

Applied to 0.92 too (data loss)

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6046) Master retry on ZK session expiry causes inconsistent region assignments.

[
https://issues.apache.org/jira/browse/HBASE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288835#comment-13288835
]

stack commented on HBASE-6046:
--

+1 on patch. I'm surprised it works for all startup paths but if there is a
code path that has a master coming up only part-initialized, I'll blame bad
code coverage rather than this patch so go commit Ram.

Master retry on ZK session expiry causes inconsistent region assignments.
-

Key: HBASE-6046
URL: https://issues.apache.org/jira/browse/HBASE-6046
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.92.1, 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE_6046-trunk.patch, HBASE_6046_0.94.patch,
HBASE_6046_0.94_1.patch, HBASE_6046_0.94_2.patch, HBASE_6046_0.94_3.patch

1 ZK Session timeout in the hmaster leads to bulk assignment though all the
RSs are online.
2 While doing bulk assignment, if the master again goes down restart(or
backup comes up) all the node created in the ZK will now be tried to reassign
to the new RSs. This is leading to double assignment.
we had 2800 regions, among this 1900 region got double assignment, taking the
region count to 4700.

[jira] [Commented] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name

[
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288836#comment-13288836
]

Hadoop QA commented on HBASE-6158:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12530831/HBASE-6158_trunk.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestClassLoading

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2101//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2101//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2101//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2101//console

This message is automatically generated.

Data loss if the words 'merges' or 'splits' are used as Column Family name
--

Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch

[jira] [Updated] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


 [ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu updated HBASE-6067:
--

Fix Version/s: (was: 0.92.3)
   0.92.2

Integrated to 0.92 branch.

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.92.2, 0.96.0, 0.94.1

 Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem


 [ 
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu updated HBASE-6067:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 HBase won't start when hbase.rootdir uses ViewFileSystem
 

 Key: HBASE-6067
 URL: https://issues.apache.org/jira/browse/HBASE-6067
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Eli Collins
Assignee: Zhihong Ted Yu
 Fix For: 0.92.2, 0.96.0, 0.94.1

 Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt


 HBase currently doesn't work with HDFS federation (hbase.rootdir with a 
 client that uses viewfs) because HLog#init uses 
 FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an 
 exception because there is no default filesystem in a viewfs client so 
 there's no way to determine a default block size or replication factor. They 
 could use the versions of these methods that take a path, however these were 
 introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


[ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288843#comment-13288843
 ] 

Aditya Kishore commented on HBASE-6158:
---

The warnings from QA are unrelated to the changes made for this bug.

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6046) Master retry on ZK session expiry causes inconsistent region assignments.

[
https://issues.apache.org/jira/browse/HBASE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288844#comment-13288844
]

Hadoop QA commented on HBASE-6046:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12530823/HBASE_6046-trunk.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 6 new or modified tests.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint

org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.replication.TestReplication

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2099//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2099//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2099//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2099//console

This message is automatically generated.

Master retry on ZK session expiry causes inconsistent region assignments.
-

[jira] [Commented] (HBASE-5533) Add more metrics to HBase


[ 
https://issues.apache.org/jira/browse/HBASE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288849#comment-13288849
 ] 

Hudson commented on HBASE-5533:
---

Integrated in HBase-0.92-security #110 (See 
[https://builds.apache.org/job/HBase-0.92-security/110/])
HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 
1346033)

 Result = FAILURE
stack : 
Files : 
* /hbase/branches/0.92/pom.xml
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/MetricsHistogram.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/metrics/TestExponentiallyDecayingSample.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/metrics/TestMetricsHistogram.java


 Add more metrics to HBase
 -

 Key: HBASE-5533
 URL: https://issues.apache.org/jira/browse/HBASE-5533
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.2, 0.94.0
Reporter: Shaneal Manek
Assignee: Shaneal Manek
Priority: Minor
 Fix For: 0.92.2, 0.94.0, 0.96.0

 Attachments: BlockingQueueContention.java, HBASE-5533-0.92-v4.patch, 
 HBASE-5533-TRUNK-v6.patch, HBASE-5533-TRUNK-v6.patch, 
 HBASE-5533-v7-0.92.patch, TimingOverhead.java, hbase-5533-0.92.patch, 
 hbase5533-0.92-v2.patch, hbase5533-0.92-v3.patch, hbase5533-0.92-v5.patch, 
 histogram_web_ui.png


 To debug/monitor production clusters, there are some more metrics I wish I 
 had available.
 In particular:
 - Although the average FS latencies are useful, a 'histogram' of recent 
 latencies (90% of reads completed in under 100ms, 99% in under 200ms, etc) 
 would be more useful
 - Similar histograms of latencies on common operations (GET, PUT, DELETE) 
 would be useful
 - Counting the number of accesses to each region to detect hotspotting
 - Exposing the current number of HLog files

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6160) META entries from daughters can be deleted before parent entries

2012-06-04 Thread Enis Soztutar (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288875#comment-13288875
 ] 

Enis Soztutar commented on HBASE-6160:
--

One option for a fix is to ensure that META entry for the parent region is 
deleted before deleting the META entry, and do the META entry deletion 
recursively.

 META entries from daughters can be deleted before parent entries
 

 Key: HBASE-6160
 URL: https://issues.apache.org/jira/browse/HBASE-6160
 Project: HBase
  Issue Type: Bug
  Components: client, regionserver
Affects Versions: 0.92.2, 0.94.0, 0.96.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar

 HBASE-5986 fixed and issue, where the client sees the META entry for the 
 parent, but not the children. However, after the fix, we have seen the 
 following issue in tests: 
 Region A is split to - B, C
 Region B is split to - D, E
 After some time, META entry for B is deleted since it is not needed anymore, 
 but META entry for Region A stays in META (C still refers it). In this case, 
 the client throws RegionOfflineException for B. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


[ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288890#comment-13288890
 ] 

Aditya Kishore commented on HBASE-6158:
---

 Is this change incompatible for rolling upgrade?

I don't think so. These are temporary work folders which get cleaned up 
routinely and particularly at Region Server (re)start for each region through 
HRegion.initialize().

However, a possible side effect could be that merges and splits folders 
created by pre-upgrade code (which somehow did not get cleaned up during a 
shutdown) may continue to exist on the file system as the cleanup code would no 
longer be looking for them. But this is better fixed in the upgrade scripts by 
deleting these folders if they are found to be a work folder instead of a CF 
container folder.

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6160) META entries from daughters can be deleted before parent entries


[ 
https://issues.apache.org/jira/browse/HBASE-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288892#comment-13288892
 ] 

Zhihong Ted Yu commented on HBASE-6160:
---

@Enis:
Can you attach log snippets to show the problem ?
e.g. what was the duration between the two splits.

 META entries from daughters can be deleted before parent entries
 

 Key: HBASE-6160
 URL: https://issues.apache.org/jira/browse/HBASE-6160
 Project: HBase
  Issue Type: Bug
  Components: client, regionserver
Affects Versions: 0.92.2, 0.94.0, 0.96.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar

 HBASE-5986 fixed and issue, where the client sees the META entry for the 
 parent, but not the children. However, after the fix, we have seen the 
 following issue in tests: 
 Region A is split to - B, C
 Region B is split to - D, E
 After some time, META entry for B is deleted since it is not needed anymore, 
 but META entry for Region A stays in META (C still refers it). In this case, 
 the client throws RegionOfflineException for B. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HBASE-6160) META entries from daughters can be deleted before parent entries

[
https://issues.apache.org/jira/browse/HBASE-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288892#comment-13288892
]

Zhihong Ted Yu edited comment on HBASE-6160 at 6/4/12 8:47 PM:
---

@Enis:
Can you attach log snippets to show the problem ?
e.g. what was the interval between the two splits.

was (Author: zhi...@ebaysf.com):
@Enis:
Can you attach log snippets to show the problem ?
e.g. what was the duration between the two splits.

META entries from daughters can be deleted before parent entries

Key: HBASE-6160
URL: https://issues.apache.org/jira/browse/HBASE-6160
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.2, 0.94.0, 0.96.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar

HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.

[jira] [Commented] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name


[ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288904#comment-13288904
 ] 

Hudson commented on HBASE-6158:
---

Integrated in HBase-0.94 #246 (See 
[https://builds.apache.org/job/HBase-0.94/246/])
HBASE-6158 Data loss if the words 'merges' or 'splits' are used as Column 
Family name (Revision 1346096)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java


 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6110) Fix TestInfoServers


 [ 
https://issues.apache.org/jira/browse/HBASE-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates updated HBASE-6110:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk.

 Fix TestInfoServers
 ---

 Key: HBASE-6110
 URL: https://issues.apache.org/jira/browse/HBASE-6110
 Project: HBase
  Issue Type: Bug
  Components: build, master, regionserver
Affects Versions: 0.96.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Attachments: hbase-6110-v0.patch, hbase-6110-v1.patch, 
 hbase-6110-v2.patch, hbase-6110-v3.patch


 With the recent port to modules, we broke a couple of tests, including this 
 one. The fix needs to ensure that the webapp still works from the in-situ and 
 packaged running of HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6112) Fix hadoop-2.0 build


 [ 
https://issues.apache.org/jira/browse/HBASE-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates updated HBASE-6112:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 Fix hadoop-2.0 build
 

 Key: HBASE-6112
 URL: https://issues.apache.org/jira/browse/HBASE-6112
 Project: HBase
  Issue Type: Bug
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.96.0

 Attachments: hbase-6112-revert-and-docs.patch, hbase-6112-v0.patch


 Some of the pom definitions are broken for the Hadoop 2.0 build. Its breaking 
 the build for that version

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4336) Convert source tree into maven modules


 [ 
https://issues.apache.org/jira/browse/HBASE-4336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates resolved HBASE-4336.


Resolution: Fixed

Closing this issue - lets open new ones when we want to add new modules.

 Convert source tree into maven modules
 --

 Key: HBASE-4336
 URL: https://issues.apache.org/jira/browse/HBASE-4336
 Project: HBase
  Issue Type: Task
  Components: build
Reporter: Gary Helmling
Assignee: Jesse Yates
Priority: Critical
 Fix For: 0.96.0

 Attachments: 4336-addendum-2.txt, 4336-addendum-3.txt, 4336v2.txt.gz, 
 4336v2_part2.txt, hbase-4336-addendum.patch, hbase-4336-v1.patch.gz, 
 hbase-4336-v2.patch.gz, refactor.txt.gz


 When we originally converted the build to maven we had a single core module 
 defined, but later reverted this to a module-less build for the sake of 
 simplicity.
 It now looks like it's time to re-address this, as we have an actual need for 
 modules to:
 * provide a trimmed down client library that applications can make use of
 * more cleanly support building against different versions of Hadoop, in 
 place of some of the reflection machinations currently required
 * incorporate the secure RPC engine that depends on some secure Hadoop classes
 I propose we start simply by refactoring into two initial modules:
 * core - common classes and utilities, and client-side code and interfaces
 * server - master and region server implementations and supporting code
 This would also lay the groundwork for incorporating the HBase security 
 features that have been developed.  Once the module structure is in place, 
 security-related features could then be incorporated into a third module -- 
 security -- after normal review and approval.  The security module could 
 then depend on secure Hadoop, without modifying the dependencies of the rest 
 of the HBase code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase


[ 
https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288916#comment-13288916
 ] 

Lars Hofhansl commented on HBASE-5954:
--

HDFS-744 is in Hadoop-2.0 now.
I'd like to make a clean patch with only the cluster wide option first. That 
way, this can be enabled and allow HBase to still be a nice HDFS citizen.

Then we can add other options (per CF, per Put, etc).


 Allow proper fsync support for HBase
 

 Key: HBASE-5954
 URL: https://issues.apache.org/jira/browse/HBASE-5954
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5954-trunk-hdfs-trunk-v2.txt, 
 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, 
 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, 
 5954-trunk-hdfs-trunk.txt, hbase-hdfs-744.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6160) META entries from daughters can be deleted before parent entries

2012-06-04 Thread Enis Soztutar (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288941#comment-13288941
 ] 

Enis Soztutar commented on HBASE-6160:
--

The exception: 
{code}
12/06/04 06:50:41 ERROR security.UserGroupInformation: 
PriviledgedActionException as: 
cause:org.apache.hadoop.hbase.client.RegionOfflineException: Split daughter 
region 
TestLoadAndVerify_1338798130970,\\xA2\x04\x00\x00\x00\x00\x00/48_0,1338800158687.50a4617eead34cad335a8dfa727d177d.
 cannot be found in META.
Exception in thread main 
org.apache.hadoop.hbase.client.RegionOfflineException: Split daughter region 
TestLoadAndVerify_1338798130970,\\xA2\x04\x00\x00\x00\x00\x00/48_0,1338800158687.50a4617eead34cad335a8dfa727d177d.
 cannot be found in META.
at 
org.apache.hadoop.hbase.client.MetaScanner$BlockingMetaScannerVisitor.processRow(MetaScanner.java:433)
at 
org.apache.hadoop.hbase.client.MetaScanner$TableMetaScannerVisitor.processRow(MetaScanner.java:490)
at 
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:227)
at 
org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:57)
at 
org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:136)
at 
org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
at 
org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:361)
at 
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:133)
at 
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:108)
at 
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:86)
at 
org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:326)
at 
org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:499)
at 
org.apache.hadoop.hbase.client.HTable.getStartEndKeys(HTable.java:452)
at 
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:132)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)

{code}

So the region in question is 
{code}
50a4617eead34cad335a8dfa727d177d
{code}
and from the logs we see that {{25d9c4ff574a37bd95bf5e5be6d618dd}} is split 
into {{1dc74065583c67b3916c4ed158cb53fa}} and 
{{50a4617eead34cad335a8dfa727d177d}}

{code}
./hbase-hbase-regionserver-ip-10-226-65-102.log:2012-06-04 04:56:02,855 INFO 
org.apache.hadoop.hbase.regionserver.SplitRequest: Region split, META updated, 
and report to master. 
Parent=TestLoadAndVerify_1338798130970,[\x02\x01\x00\x00\x00\x00\x00/71_0,1338799021182.25d9c4ff574a37bd95bf5e5be6d618dd.,
 new regions: 
TestLoadAndVerify_1338798130970,[\x02\x01\x00\x00\x00\x00\x00/71_0,1338800158687.1dc74065583c67b3916c4ed158cb53fa.,
 
TestLoadAndVerify_1338798130970,\\xA2\x04\x00\x00\x00\x00\x00/48_0,1338800158687.50a4617eead34cad335a8dfa727d177d..
 Split took 4sec
{code}

After some time, {{50a4617eead34cad335a8dfa727d177d}} is further split into 
two: 

{code}
./hbase-hbase-regionserver-ip-10-226-65-102.log:2012-06-04 05:41:13,488 INFO 
org.apache.hadoop.hbase.regionserver.SplitRequest: Region split, META updated, 
and report to master. Parent=TestLoadAndVerify_1
338798130970,\\xA2\x04\x00\x00\x00\x00\x00/48_0,1338800158687.50a4617eead34cad335a8dfa727d177d.,
 new regions: 
TestLoadAndVerify_1338798130970,\\xA2\x04\x00\x00\x00\x00\x00/48_0,1338802866393.16288
65d7fa8e9eec3a7d8073465296e., 
TestLoadAndVerify_1338798130970,]y\x04\x00\x00\x00\x00\x00/47_0,1338802866393.413cafe6c61426e26254c197e8c0a6ba..
 Split took 7sec
{code}

Further time passes, and CatalogJanitor deletes the META entry for that region:
{code}
./hbase-hbase-master-ip-10-144-69-91.log:2012-06-04 05:47:16,688 DEBUG 
org.apache.hadoop.hbase.master.CatalogJanitor: Deleting region 
TestLoadAndVerify_1338798130970,\\xA2\x04\x00\x00\x00\x00\x00/48_0,1338800158687.50a4617eead34cad335a8dfa727d177d.
 because daughter splits no longer hold references
./hbase-hbase-master-ip-10-144-69-91.log:2012-06-04 05:47:18,103 INFO 
org.apache.hadoop.hbase.catalog.MetaEditor: Deleted daughters references, 
qualifier=splitA and qualifier=splitB, from parent 
TestLoadAndVerify_1338798130970,\\xA2\x04\x00\x00\x00\x00\x00/48_0,1338800158687.50a4617eead34cad335a8dfa727d177d.
./hbase-hbase-master-ip-10-144-69-91.log:2012-06-04 05:47:18,103 DEBUG 
org.apache.hadoop.hbase.regionserver.HRegion: DELETING region 
hdfs://ip-10-10-50-98.ec2.internal:8020/apps/hbase/data/TestLoadAndVerify_1338798130970/50a4617eead34cad335a8dfa727d177d
./hbase-hbase-master-ip-10-144-69-91.log:2012-06-04 05:47:18,145 INFO 
org.apache.hadoop.hbase.catalog.MetaEditor: Deleted region

[jira] [Commented] (HBASE-6159) truncate leaks ZK connection in HBase shell