[jira] [Commented] (HBASE-5143) Fix config typo in pluggable load balancer factory
[ https://issues.apache.org/jira/browse/HBASE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182214#comment-13182214 ] Hadoop QA commented on HBASE-5143: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509839/HBASE-5143.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -151 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/701//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/701//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/701//console This message is automatically generated. Fix config typo in pluggable load balancer factory -- Key: HBASE-5143 URL: https://issues.apache.org/jira/browse/HBASE-5143 Project: HBase Issue Type: Sub-task Components: master Reporter: Harsh J Priority: Critical Fix For: 0.92.0, 0.94.0 Attachments: HBASE-5143.patch HBASE-4240 made LoadBalancer pluggable. Configuration it loads seems to be wrongly named and carries a typo: hbase.maser.loadBalancer.class Could rather be hbase.master.loadbalancer.class Luckily 0.92 is not out yet and we should fix it asap, before folks start using it. Attaching patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5052) The path where a dynamically loaded coprocessor jar is copied on the local file system depends on the region name (and implicitly, the start key)
[ https://issues.apache.org/jira/browse/HBASE-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182216#comment-13182216 ] Hudson commented on HBASE-5052: --- Integrated in HBase-0.92-security #67 (See [https://builds.apache.org/job/HBase-0.92-security/67/]) HBASE-5052 The path where a dynamically loaded coprocessor jar is copied on the local file system depends on the region name (and implicitly, the start key) stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java The path where a dynamically loaded coprocessor jar is copied on the local file system depends on the region name (and implicitly, the start key) - Key: HBASE-5052 URL: https://issues.apache.org/jira/browse/HBASE-5052 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.92.0 Reporter: Andrei Dragomir Assignee: Andrei Dragomir Fix For: 0.92.0 Attachments: HBASE-5052.patch When loading a coprocessor from hdfs, the jar file gets copied to a path on the local filesystem, which depends on the region name, and the region start key. The name is cleaned, but not enough, so when you have filesystem unfriendly characters (/?:, etc), the coprocessor is not loaded, and an error is thrown -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5141) Memory leak in MonitoredRPCHandlerImpl
[ https://issues.apache.org/jira/browse/HBASE-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182215#comment-13182215 ] Hudson commented on HBASE-5141: --- Integrated in HBase-0.92-security #67 (See [https://builds.apache.org/job/HBase-0.92-security/67/]) HBASE-5141 Memory leak in MonitoredRPCHandlerImpl stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredRPCHandlerImpl.java Memory leak in MonitoredRPCHandlerImpl -- Key: HBASE-5141 URL: https://issues.apache.org/jira/browse/HBASE-5141 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Priority: Blocker Fix For: 0.92.0, 0.94.0 Attachments: HBASE-5141-v2.patch, HBASE-5141.patch, Screen Shot 2012-01-06 at 3.03.09 PM.png I got a pretty reliable way of OOME'ing my region servers. Using a big payload (64MB in my case), a default heap and default number of handlers, it's not too long that all the MonitoredRPCHandlerImpl hold on a 64MB reference and once a compaction kicks in it kills everything. The issue is that even after the RPC call is done, the packet still lives in MonitoredRPCHandlerImpl. Will attach a screen shot of jprofiler's analysis in a moment. This is a blocker for 0.92.0, anyone using a high number of handlers and bigish values will kill themselves. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on SoftValueSortedMap
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182217#comment-13182217 ] Hudson commented on HBASE-5088: --- Integrated in HBase-0.92-security #67 (See [https://builds.apache.org/job/HBase-0.92-security/67/]) HBASE-5088 addendum larsh : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java A concurrency issue on SoftValueSortedMap - Key: HBASE-5088 URL: https://issues.apache.org/jira/browse/HBASE-5088 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.90.4, 0.94.0 Reporter: Jieshan Bean Assignee: Lars Hofhansl Priority: Critical Fix For: 0.92.0, 0.94.0, 0.90.6 Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok. But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate on this view map(like add/delete) in other threads, a concurrency issue may occur. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5143) Fix config typo in pluggable load balancer factory
[
https://issues.apache.org/jira/browse/HBASE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182219#comment-13182219
]
Harsh J commented on HBASE-5143:
Test failures appear unrelated. Also, an {{ack/grep}} of the old config string
yields no result across code base.
Fix config typo in pluggable load balancer factory
--
Key: HBASE-5143
URL: https://issues.apache.org/jira/browse/HBASE-5143
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Harsh J
Priority: Critical
Fix For: 0.92.0, 0.94.0
Attachments: HBASE-5143.patch
HBASE-4240 made LoadBalancer pluggable.
Configuration it loads seems to be wrongly named and carries a typo:
hbase.maser.loadBalancer.class
Could rather be hbase.master.loadbalancer.class
Luckily 0.92 is not out yet and we should fix it asap, before folks start
using it. Attaching patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5143) Fix config typo in pluggable load balancer factory
[ https://issues.apache.org/jira/browse/HBASE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HBASE-5143: --- Priority: Blocker (was: Critical) Fix config typo in pluggable load balancer factory -- Key: HBASE-5143 URL: https://issues.apache.org/jira/browse/HBASE-5143 Project: HBase Issue Type: Sub-task Components: master Reporter: Harsh J Priority: Blocker Fix For: 0.92.0, 0.94.0 Attachments: HBASE-5143.patch HBASE-4240 made LoadBalancer pluggable. Configuration it loads seems to be wrongly named and carries a typo: hbase.maser.loadBalancer.class Could rather be hbase.master.loadbalancer.class Luckily 0.92 is not out yet and we should fix it asap, before folks start using it. Attaching patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4224) Need a flush by regionserver rather than by table option
[ https://issues.apache.org/jira/browse/HBASE-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182231#comment-13182231 ] jirapos...@reviews.apache.org commented on HBASE-4224: -- bq. On 2012-01-07 17:27:49, Ted Yu wrote: bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 2794 bq. https://reviews.apache.org/r/3308/diff/3/?file=66890#file66890line2794 bq. bq. Do we need to place a try/catch block around line 2795 ? bq. Currently the first failure would stop subsequent flushes. Ted this sounds good and we'd want to surround it with a try catch. But then we'd be masking those errors right? If I were the user and called flush through the shell, I'd not only wanna know that all flushes didn't go successful but I'd also want to know how many of them were unsuccessful and which region flushes failed due to what reason. So do you think it would be a good idea to return a status object which gives the client more information regarding the flushes ? bq. On 2012-01-07 17:27:49, Ted Yu wrote: bq. /src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java, line 486 bq. https://reviews.apache.org/r/3308/diff/3/?file=66889#file66889line486 bq. bq. This and flushAllRegions() are similar. bq. Can we introduce just one new method which checks whether the list is empty to decide what to do ? bq. i.e. move the check @ line 1403 of HBaseAdmin to the implementation of the new method. This is rather a flag defined on the client which means when the regions == null then u flush the entire RegionServer. IF we move this to the region server side then we'd be making this an API level definition. I think flushAllRegions is more intuitive as a RegionServer API than an API which flushes the Region Server when the regions parameter is null. Assume a new person wants to flushAllRegions. flush(ListHRegionInfo regions) tells him that hey I can flush a list of regions and unless he looks at the implementation or the javadoc he'd miss the fact that this API can also flush all the regions. On the other hand flushAllRegions invites the person with open arms saying that hey i'll flush all the regions for you. bq. On 2012-01-07 17:27:49, Ted Yu wrote: bq. /src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 1409 bq. https://reviews.apache.org/r/3308/diff/3/?file=66888#file66888line1409 bq. bq. regions could be empty, right ? The only condition when regions could be empty is when the name to be flushed is a RegionServer and not a single Region or table. This is based on the presumption that MetaReader.getTableRegionsAndLocations doesn't return null regions and isRegionsName is null then it never executes the flush. So this is fine ? - Akash --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3308/#review4236 --- On 2012-01-06 18:48:11, Akash Ashok wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3308/ bq. --- bq. bq. (Updated 2012-01-06 18:48:11) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. Flush by RegionServer bq. bq. bq. This addresses bug HBase-4224. bq. https://issues.apache.org/jira/browse/HBase-4224 bq. bq. bq. Diffs bq. - bq. bq./src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 1226330 bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1226330 bq./src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1226330 bq./src/main/java/org/apache/hadoop/hbase/ServerName.java 1226330 bq. bq. Diff: https://reviews.apache.org/r/3308/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Akash bq. bq. Need a flush by regionserver rather than by table option Key: HBASE-4224 URL: https://issues.apache.org/jira/browse/HBASE-4224 Project: HBase Issue Type: Bug Components: shell Reporter: stack Assignee: Akash Ashok Attachments: HBase-4224-v2.patch, HBase-4224.patch This evening needed to clean out logs on the cluster. logs are by regionserver. to let go of logs, we need to have all edits emptied from memory. only flush is by table or region. We need to be able to flush the regionserver. Need to add this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your
[jira] [Commented] (HBASE-4224) Need a flush by regionserver rather than by table option
[
https://issues.apache.org/jira/browse/HBASE-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182234#comment-13182234
]
Zhihong Yu commented on HBASE-4224:
---
w.r.t. whether flushAllRegions() should be introduced into HRegionInterface. In
your patch HBaseAdmin has this method:
{ocde}
private boolean flush(ServerName serverName, ListHRegionInfo regions)
{ocde}
which decides the scope of flushing by checking regions. I think this check can
be performed on region server side.
Your argument about javadoc makes sense. We should clearly document the
functionality of the (single) new method added to HRegionInterface.
bq. But then we'd be masking those errors right?
Please refer to HRegion.bulkLoadHFiles() for example of buffering multiple
IOE's:
{code}
if (ioes.size() != 0) {
LOG.error(There were IO errors when checking if bulk load is ok. +
throwing exception!);
throw MultipleIOException.createIOException(ioes);
}
{code}
Need a flush by regionserver rather than by table option
Key: HBASE-4224
URL: https://issues.apache.org/jira/browse/HBASE-4224
Project: HBase
Issue Type: Bug
Components: shell
Reporter: stack
Assignee: Akash Ashok
Attachments: HBase-4224-v2.patch, HBase-4224.patch
This evening needed to clean out logs on the cluster. logs are by
regionserver. to let go of logs, we need to have all edits emptied from
memory. only flush is by table or region. We need to be able to flush the
regionserver. Need to add this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4224) Need a flush by regionserver rather than by table option
[ https://issues.apache.org/jira/browse/HBASE-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182262#comment-13182262 ] Akash Ashok commented on HBASE-4224: Your argument about javadoc makes sense. We should clearly document the functionality of the (single) new method added to HRegionInterface. What I meant to say was that it's easy enough to over look this functionality even with good amount of javadoc. I think it would make a lot of difference if we have this extra new method flushAllRegions(). I am just trying to understand if you see any issue with this new extra API being added on the Region Server ? +1 on the MultipleIOException. Thanks :). I like that class :) Need a flush by regionserver rather than by table option Key: HBASE-4224 URL: https://issues.apache.org/jira/browse/HBASE-4224 Project: HBase Issue Type: Bug Components: shell Reporter: stack Assignee: Akash Ashok Attachments: HBase-4224-v2.patch, HBase-4224.patch This evening needed to clean out logs on the cluster. logs are by regionserver. to let go of logs, we need to have all edits emptied from memory. only flush is by table or region. We need to be able to flush the regionserver. Need to add this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4224) Need a flush by regionserver rather than by table option
[ https://issues.apache.org/jira/browse/HBASE-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182266#comment-13182266 ] Zhihong Yu commented on HBASE-4224: --- I am not strongly opposed to adding a new method. Comment from other committers is welcome. Need a flush by regionserver rather than by table option Key: HBASE-4224 URL: https://issues.apache.org/jira/browse/HBASE-4224 Project: HBase Issue Type: Bug Components: shell Reporter: stack Assignee: Akash Ashok Attachments: HBase-4224-v2.patch, HBase-4224.patch This evening needed to clean out logs on the cluster. logs are by regionserver. to let go of logs, we need to have all edits emptied from memory. only flush is by table or region. We need to be able to flush the regionserver. Need to add this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5146) Hbase Shell - allow setting config properties
Hbase Shell - allow setting config properties - Key: HBASE-5146 URL: https://issues.apache.org/jira/browse/HBASE-5146 Project: HBase Issue Type: New Feature Components: shell Affects Versions: 0.90.2, 0.90.1 Reporter: Mikael Sitruk It should be possible to set config properties of the cluster/table/column family via the shell and not only via API For example the 'hbase.hregion.majorcompaction' property cannot be set via the shell. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5146) Hbase Shell - allow setting config properties
[ https://issues.apache.org/jira/browse/HBASE-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikael Sitruk updated HBASE-5146: - Affects Version/s: 0.92.0 Hbase Shell - allow setting config properties - Key: HBASE-5146 URL: https://issues.apache.org/jira/browse/HBASE-5146 Project: HBase Issue Type: New Feature Components: shell Affects Versions: 0.90.1, 0.90.2, 0.92.0 Reporter: Mikael Sitruk It should be possible to set config properties of the cluster/table/column family via the shell and not only via API For example the 'hbase.hregion.majorcompaction' property cannot be set via the shell. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5146) Hbase Shell - allow setting config properties
[ https://issues.apache.org/jira/browse/HBASE-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182301#comment-13182301 ] Zhihong Yu commented on HBASE-5146: --- Table / column family attribute change is related to HBASE-4213. Cluster wide property update depends on HBASE-3909. Hbase Shell - allow setting config properties - Key: HBASE-5146 URL: https://issues.apache.org/jira/browse/HBASE-5146 Project: HBase Issue Type: New Feature Components: shell Affects Versions: 0.90.1, 0.90.2, 0.92.0 Reporter: Mikael Sitruk It should be possible to set config properties of the cluster/table/column family via the shell and not only via API For example the 'hbase.hregion.majorcompaction' property cannot be set via the shell. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5147) Compaction/Major compaction operation from shell/API/JMX
Compaction/Major compaction operation from shell/API/JMX Key: HBASE-5147 URL: https://issues.apache.org/jira/browse/HBASE-5147 Project: HBase Issue Type: New Feature Components: master, regionserver, shell Affects Versions: 0.90.2, 0.90.1, 0.92.0 Reporter: Mikael Sitruk Compaction handling is important internal process nevertheless when it occurs it can have dramatic impact on cluster performance (especially when the cluster growth uniformly). Secondly current operations enabled are just to trigger a compaction process without possibility to know what is its status. Making very unconvenient to manage compaction manually. This JIRA suggests to allow management of compaction from different interfaces - Shell, API, JMX Following is the list of requested operations 1. getting compaction status - which server/region is under compaction and what is compacted (currently this info is only via log) 2. % of uncompacted files 3. Possibility to stop major compaction 4. Starting compaction from JMX (other interface already support this) Note - part of these requirements are also suggested in HBASE-3965 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5142) Compute standard deviation using AggregateProtocol
[ https://issues.apache.org/jira/browse/HBASE-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182308#comment-13182308 ] Zhihong Yu commented on HBASE-5142: --- AggregationClient.std() already provides this capability. Compute standard deviation using AggregateProtocol -- Key: HBASE-5142 URL: https://issues.apache.org/jira/browse/HBASE-5142 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu Looking at http://en.wikipedia.org/wiki/Standard_deviation, I think two passes are involved in computing std: 1. compute average, u 2. tuple of (sum of (xi-u)*(xi-u), number of rows) is returned from each region 3. client computes std based on results from step 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5149) getConfiguration() implementation is misleading
getConfiguration() implementation is misleading
---
Key: HBASE-5149
URL: https://issues.apache.org/jira/browse/HBASE-5149
Project: HBase
Issue Type: Improvement
Components: master, regionserver
Affects Versions: 0.90.4, 0.90.2, 0.90.1
Reporter: Mikael Sitruk
The following code will not return the cluster configuration but the local one
which is somewhat misleading.
{code}
conn = (HConnection) HConnectionManager.getConnection(m_hbConfig); //here the
configuration is local
Configuration conf = conn.getConfiguration()
conf.getString(hbase.hregion.majorcompaction); // will return the parameter
from the local config instead of the cluster the connection is connected to.
{code}
It is suggested that once a connection has been acquired the configuration
object should be the one of the cluster.
As a general observation it is not possible to retrieve the used configuration
on the cluster
It is suggested to add API at {{HRegionServerInterface}}, {{HMasterInterface}}
to get the configuration used by the component appropriately (note in 0.90.4
the getConfiguration exist on the Server interface implemented by HRegionServer
and HMaster classes) but this interface is not visible/extended by
HRegionServerInterface/HMasterInterface, therefore not accessible from client
code.
Also an API like {{HashMapHserverInfo,Configuration
getClusterConfigurations()}} can be added on the HConnection object.
Additional notes:
Since servers can have different properties values (like disk, tmp dir,...) it
can be acceptable that the configuration object returned by the connection
returns special value to indicate - conflict between value or that multiple
values exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3584) We need to atomically put/delete/increment in one call
[ https://issues.apache.org/jira/browse/HBASE-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182327#comment-13182327 ] Lars Hofhansl commented on HBASE-3584: -- Should we do this before client 2.0? What should the client side API look like? Something like: AtomicRowOperation (which implements Row, so can itself be batched), which wraps a set Put/Delete/Increment/Append (all for the same rowkey)? Compared to a straight put(ListPut), this would probably be slower as we cannot do the same mini batching (not with a *lot* of refactoring). Also it is not entirely clear that this could even work with Increment/Append of the special immediate-visibility that those need. So maybe start with Put/Delete. We need to atomically put/delete/increment in one call -- Key: HBASE-3584 URL: https://issues.apache.org/jira/browse/HBASE-3584 Project: HBase Issue Type: Bug Reporter: ryan rawson Right now we have the following calls: put(Put) delete(Delete) increment(Increments) But we cannot combine all of the above in a single call, complete with a single row lock. It would be nice to do that. It would also allow us to do a CAS where we could do a put/increment if the check succeeded. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5142) Compute standard deviation using AggregateProtocol
[ https://issues.apache.org/jira/browse/HBASE-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5142: -- Comment: was deleted (was: AggregationClient.std() already provides this capability.) Compute standard deviation using AggregateProtocol -- Key: HBASE-5142 URL: https://issues.apache.org/jira/browse/HBASE-5142 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu Looking at http://en.wikipedia.org/wiki/Standard_deviation, I think two passes are involved in computing std: 1. compute average, u 2. tuple of (sum of (xi-u)*(xi-u), number of rows) is returned from each region 3. client computes std based on results from step 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5123) Provide more aggregate functions for Aggregations Protocol
[ https://issues.apache.org/jira/browse/HBASE-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182337#comment-13182337 ] Zhihong Yu edited comment on HBASE-5123 at 1/9/12 12:32 AM: AggregationClient.std() already provides support for standard deviation. was (Author: zhi...@ebaysf.com): AggregationClient.std() already provides this capability. Provide more aggregate functions for Aggregations Protocol -- Key: HBASE-5123 URL: https://issues.apache.org/jira/browse/HBASE-5123 Project: HBase Issue Type: Improvement Reporter: Zhihong Yu Royston requested the following aggregates on top of what we already have: Median, Weighted Median, Mult See discussion entitled 'AggregateProtocol Help' on user list -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5123) Provide more aggregate functions for Aggregations Protocol
[ https://issues.apache.org/jira/browse/HBASE-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182337#comment-13182337 ] Zhihong Yu commented on HBASE-5123: --- AggregationClient.std() already provides this capability. Provide more aggregate functions for Aggregations Protocol -- Key: HBASE-5123 URL: https://issues.apache.org/jira/browse/HBASE-5123 Project: HBase Issue Type: Improvement Reporter: Zhihong Yu Royston requested the following aggregates on top of what we already have: Median, Weighted Median, Mult See discussion entitled 'AggregateProtocol Help' on user list -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5142) Compute quantile/ntile using AggregateProtocol
[ https://issues.apache.org/jira/browse/HBASE-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5142: -- Description: (was: Looking at http://en.wikipedia.org/wiki/Standard_deviation, I think two passes are involved in computing std: 1. compute average, u 2. tuple of (sum of (xi-u)*(xi-u), number of rows) is returned from each region 3. client computes std based on results from step 2) Summary: Compute quantile/ntile using AggregateProtocol (was: Compute standard deviation using AggregateProtocol) Compute quantile/ntile using AggregateProtocol -- Key: HBASE-5142 URL: https://issues.apache.org/jira/browse/HBASE-5142 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182342#comment-13182342 ] jirapos...@reviews.apache.org commented on HBASE-5150: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3433/ --- Review request for hbase, Todd Lipcon and Michael Stack. Summary --- Cleaned up the tests introduced in HBASE-5081, added some documentation. This addresses bug HBASE-5150. https://issues.apache.org/jira/browse/HBASE-5150 Diffs - src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 0ef0e33 src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java b0487f1 src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0974b56 Diff: https://reviews.apache.org/r/3433/diff Testing --- Affected tests: TestDistributedLogSplitting and TestSplitLogManager, are passed Thanks, Jimmy Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5150: --- Attachment: hbase-5150.txt Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-5150.txt This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5150: --- Status: Patch Available (was: Open) Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-5150.txt This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5121) MajorCompaction may affect scan's correctness
[ https://issues.apache.org/jira/browse/HBASE-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182345#comment-13182345 ] chunhui shen commented on HBASE-5121: - @Lars I agree with the suggested patch which is simpler and available. MajorCompaction may affect scan's correctness - Key: HBASE-5121 URL: https://issues.apache.org/jira/browse/HBASE-5121 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.90.4 Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.94.0, 0.92.1, 0.90.6 Attachments: 5121-suggest.txt, 5121-trunk-combined.txt, 5121.90, hbase-5121-testcase.patch, hbase-5121.patch, hbase-5121v2.patch In our test, there are two families' keyvalue for one row. But we could find a infrequent problem when doing scan's next if majorCompaction happens concurrently. In the client's two continuous doing scan.next(): 1.First time, scan's next returns the result where family A is null. 2.Second time, scan's next returns the result where family B is null. The two next()'s result have the same row. If there are more families, I think the scenario will be more strange... We find the reason is that storescanner.peek() is changed after majorCompaction if there are delete type KeyValue. This change causes the PriorityQueueKeyValueScanner of RegionScanner's heap is not sure to be sorted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182346#comment-13182346 ] jirapos...@reviews.apache.org commented on HBASE-5150: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3433/#review4256 --- src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java https://reviews.apache.org/r/3433/#comment9614 Should this read ', or until it is ...' ? src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java https://reviews.apache.org/r/3433/#comment9613 Typo: exception src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java https://reviews.apache.org/r/3433/#comment9615 So this is no longer a failure case ? - Ted On 2012-01-09 01:32:36, Jimmy Xiang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3433/ bq. --- bq. bq. (Updated 2012-01-09 01:32:36) bq. bq. bq. Review request for hbase, Todd Lipcon and Michael Stack. bq. bq. bq. Summary bq. --- bq. bq. Cleaned up the tests introduced in HBASE-5081, added some documentation. bq. bq. bq. This addresses bug HBASE-5150. bq. https://issues.apache.org/jira/browse/HBASE-5150 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 0ef0e33 bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java b0487f1 bq.src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0974b56 bq. bq. Diff: https://reviews.apache.org/r/3433/diff bq. bq. bq. Testing bq. --- bq. bq. Affected tests: TestDistributedLogSplitting and TestSplitLogManager, are passed bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-5150.txt This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182347#comment-13182347 ] Hadoop QA commented on HBASE-5150: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509870/hbase-5150.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -151 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/702//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/702//console This message is automatically generated. Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-5150.txt This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182350#comment-13182350 ] jirapos...@reviews.apache.org commented on HBASE-5150: -- bq. On 2012-01-09 01:56:13, Ted Yu wrote: bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java, line 302 bq. https://reviews.apache.org/r/3433/diff/1/?file=67167#file67167line302 bq. bq. Should this read ', or until it is ...' ? It will keep waiting until it is interrupted. The task won't be removed from the task map in this test case actually because slm.ignoreZKDeleteForTesting is set to true. I will add this to the doc. bq. On 2012-01-09 01:56:13, Ted Yu wrote: bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java, line 322 bq. https://reviews.apache.org/r/3433/diff/1/?file=67167#file67167line322 bq. bq. Typo: exception right. bq. On 2012-01-09 01:56:13, Ted Yu wrote: bq. src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java, line 481 bq. https://reviews.apache.org/r/3433/diff/1/?file=67168#file67168line481 bq. bq. So this is no longer a failure case ? It doesn't fail here. It tests a scenario when a znode is removed by others. - Jimmy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3433/#review4256 --- On 2012-01-09 01:32:36, Jimmy Xiang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3433/ bq. --- bq. bq. (Updated 2012-01-09 01:32:36) bq. bq. bq. Review request for hbase, Todd Lipcon and Michael Stack. bq. bq. bq. Summary bq. --- bq. bq. Cleaned up the tests introduced in HBASE-5081, added some documentation. bq. bq. bq. This addresses bug HBASE-5150. bq. https://issues.apache.org/jira/browse/HBASE-5150 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 0ef0e33 bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java b0487f1 bq.src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0974b56 bq. bq. Diff: https://reviews.apache.org/r/3433/diff bq. bq. bq. Testing bq. --- bq. bq. Affected tests: TestDistributedLogSplitting and TestSplitLogManager, are passed bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-5150.txt This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182351#comment-13182351 ] jirapos...@reviews.apache.org commented on HBASE-5150: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3433/ --- (Updated 2012-01-09 02:39:28.394063) Review request for hbase, Todd Lipcon and Michael Stack. Changes --- Updated per comments. Summary --- Cleaned up the tests introduced in HBASE-5081, added some documentation. This addresses bug HBASE-5150. https://issues.apache.org/jira/browse/HBASE-5150 Diffs (updated) - src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 0ef0e33 src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java b0487f1 src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0974b56 Diff: https://reviews.apache.org/r/3433/diff Testing --- Affected tests: TestDistributedLogSplitting and TestSplitLogManager, are passed Thanks, Jimmy Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-5150.txt This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182355#comment-13182355 ] jirapos...@reviews.apache.org commented on HBASE-5150: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3433/#review4259 --- src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java https://reviews.apache.org/r/3433/#comment9620 There is no reason to wait for 3 seconds over here src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java https://reviews.apache.org/r/3433/#comment9621 Since fs is from master.getMasterFileSystem().getFileSystem(), I am pretty sure it gets cleaned up at the end of the test when the cluster is stopped. Did you see the files hanging around? src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java https://reviews.apache.org/r/3433/#comment9622 The test should fail if there is an exception. - Prakash On 2012-01-09 02:39:28, Jimmy Xiang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3433/ bq. --- bq. bq. (Updated 2012-01-09 02:39:28) bq. bq. bq. Review request for hbase, Todd Lipcon and Michael Stack. bq. bq. bq. Summary bq. --- bq. bq. Cleaned up the tests introduced in HBASE-5081, added some documentation. bq. bq. bq. This addresses bug HBASE-5150. bq. https://issues.apache.org/jira/browse/HBASE-5150 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 0ef0e33 bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java b0487f1 bq.src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0974b56 bq. bq. Diff: https://reviews.apache.org/r/3433/diff bq. bq. bq. Testing bq. --- bq. bq. Affected tests: TestDistributedLogSplitting and TestSplitLogManager, are passed bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-5150.txt This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5150) Fail in a thread may not fail a test, clean up log splitting test
[ https://issues.apache.org/jira/browse/HBASE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13182359#comment-13182359 ] jirapos...@reviews.apache.org commented on HBASE-5150: -- bq. On 2012-01-09 03:01:16, Prakash Khemani wrote: bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java, line 316 bq. https://reviews.apache.org/r/3433/diff/2/?file=67170#file67170line316 bq. bq. There is no reason to wait for 3 seconds over here If everything works as expected, we won't have to wait here, since those two fail() calls won't be called. I'd like to give it some time to make sure the second splitLogDistributed() is really blocked there. bq. On 2012-01-09 03:01:16, Prakash Khemani wrote: bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java, line 326 bq. https://reviews.apache.org/r/3433/diff/2/?file=67170#file67170line326 bq. bq. Since fs is from master.getMasterFileSystem().getFileSystem(), I am pretty sure it gets cleaned up at the end of the test when the cluster is stopped. Did you see the files hanging around? ShutdownMiniCluster will try to clean up the test dir. Just want to be safe. bq. On 2012-01-09 03:01:16, Prakash Khemani wrote: bq. src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java, line 481 bq. https://reviews.apache.org/r/3433/diff/2/?file=67171#file67171line481 bq. bq. The test should fail if there is an exception. That fail() doesn't fail the test. If there is an exception, those waitForCounter() will actually fail the test. - Jimmy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3433/#review4259 --- On 2012-01-09 02:39:28, Jimmy Xiang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3433/ bq. --- bq. bq. (Updated 2012-01-09 02:39:28) bq. bq. bq. Review request for hbase, Todd Lipcon and Michael Stack. bq. bq. bq. Summary bq. --- bq. bq. Cleaned up the tests introduced in HBASE-5081, added some documentation. bq. bq. bq. This addresses bug HBASE-5150. bq. https://issues.apache.org/jira/browse/HBASE-5150 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 0ef0e33 bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java b0487f1 bq.src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0974b56 bq. bq. Diff: https://reviews.apache.org/r/3433/diff bq. bq. bq. Testing bq. --- bq. bq. Affected tests: TestDistributedLogSplitting and TestSplitLogManager, are passed bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. Fail in a thread may not fail a test, clean up log splitting test - Key: HBASE-5150 URL: https://issues.apache.org/jira/browse/HBASE-5150 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-5150.txt This is to clean up some tests for HBASE-5081. The Assert.fail method in a separate thread will terminate the thread, but may not fail the test. We can use callable, so that we can get the error in getting the result. Some documentation to explain the test will be helpful too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
