[jira] [Assigned] (HBASE-5068) RC1 can not build its hadoop-0.23 profile
[ https://issues.apache.org/jira/browse/HBASE-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5068: - Assignee: Roman Shaposhnik > RC1 can not build its hadoop-0.23 profile > - > > Key: HBASE-5068 > URL: https://issues.apache.org/jira/browse/HBASE-5068 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.92.0 >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Fix For: 0.92.0, 0.94.0 > > Attachments: HBASE-5068.patch.txt > > > The hadoop .23 version needs to be bumped to 0.23.1-SNAPSHOT -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-4720: - Assignee: Mubarak Seyed > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.94.0 > > Attachments: HBASE-4720.v1.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5041) Major compaction on non existing table does not throw error
[ https://issues.apache.org/jira/browse/HBASE-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5041: - Assignee: Shrijeet Paliwal > Major compaction on non existing table does not throw error > > > Key: HBASE-5041 > URL: https://issues.apache.org/jira/browse/HBASE-5041 > Project: HBase > Issue Type: Bug > Components: regionserver, shell >Affects Versions: 0.90.3 >Reporter: Shrijeet Paliwal >Assignee: Shrijeet Paliwal > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 0001-HBASE-5041-Throw-error-if-table-does-not-exist.patch > > > Following will not complain even if fubar does not exist > {code} > echo "major_compact 'fubar'" | $HBASE_HOME/bin/hbase shell > {code} > The downside for this defect is that major compaction may be skipped due to > a typo by Ops. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-4720: - Assignee: Mubarak Seyed (was: Seyed) > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.94.0 > > Attachments: HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v2.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5070) Constraints implementation and javadoc changes
[ https://issues.apache.org/jira/browse/HBASE-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5070: - Assignee: Jesse Yates > Constraints implementation and javadoc changes > -- > > Key: HBASE-5070 > URL: https://issues.apache.org/jira/browse/HBASE-5070 > Project: HBase > Issue Type: Task >Reporter: Zhihong Yu >Assignee: Jesse Yates > Attachments: java_HBASE-5070-v2.patch, java_HBASE-5070-v3.patch > > > This is continuation of HBASE-4605 > See Stack's comments https://reviews.apache.org/r/2579/#review3980 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5010) Filter HFiles based on TTL
[ https://issues.apache.org/jira/browse/HBASE-5010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5010: - Assignee: Zhihong Yu (was: Mikhail Bautin) > Filter HFiles based on TTL > -- > > Key: HBASE-5010 > URL: https://issues.apache.org/jira/browse/HBASE-5010 > Project: HBase > Issue Type: Bug >Reporter: Mikhail Bautin >Assignee: Zhihong Yu > Attachments: 5010.patch, D1017.1.patch, D1017.2.patch, D909.1.patch, > D909.2.patch > > > In ScanWildcardColumnTracker we have > {code:java} > > this.oldestStamp = EnvironmentEdgeManager.currentTimeMillis() - ttl; > ... > private boolean isExpired(long timestamp) { > return timestamp < oldestStamp; > } > {code} > but this time range filtering does not participate in HFile selection. In one > real case this caused next() calls to time out because all KVs in a table got > expired, but next() had to iterate over the whole table to find that out. We > should be able to filter out those HFiles right away. I think a reasonable > approach is to add a "default timerange filter" to every scan for a CF with a > finite TTL and utilize existing filtering in > StoreFile.Reader.passesTimerangeFilter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5052) The path where a dynamically loaded coprocessor jar is copied on the local file system depends on the region name (and implicitly, the start key)
[ https://issues.apache.org/jira/browse/HBASE-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5052: - Assignee: Andrei Dragomir > The path where a dynamically loaded coprocessor jar is copied on the local > file system depends on the region name (and implicitly, the start key) > - > > Key: HBASE-5052 > URL: https://issues.apache.org/jira/browse/HBASE-5052 > Project: HBase > Issue Type: Bug > Components: coprocessors >Affects Versions: 0.92.0 >Reporter: Andrei Dragomir >Assignee: Andrei Dragomir > Attachments: HBASE-5052.patch > > > When loading a coprocessor from hdfs, the jar file gets copied to a path on > the local filesystem, which depends on the region name, and the region start > key. The name is "cleaned", but not enough, so when you have filesystem > unfriendly characters (/?:, etc), the coprocessor is not loaded, and an error > is thrown -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5124) Backport LoadTestTool to 0.92
[ https://issues.apache.org/jira/browse/HBASE-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5124: - Assignee: Zhihong Yu > Backport LoadTestTool to 0.92 > - > > Key: HBASE-5124 > URL: https://issues.apache.org/jira/browse/HBASE-5124 > Project: HBase > Issue Type: Task >Reporter: Zhihong Yu >Assignee: Zhihong Yu > Fix For: 0.92.0 > > Attachments: hbase-5124.txt > > > LoadTestTool is very useful. > This JIRA backports LoadTestTool to 0.92 so that users don't have to build > TRUNK in order to use it against 0.92 cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5136) Redundant MonitoredTask instances in case of distributed log splitting retry
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5136: - Assignee: Zhihong Yu > Redundant MonitoredTask instances in case of distributed log splitting retry > > > Key: HBASE-5136 > URL: https://issues.apache.org/jira/browse/HBASE-5136 > Project: HBase > Issue Type: Task >Reporter: Zhihong Yu >Assignee: Zhihong Yu > > In case of log splitting retry, the following code would be executed multiple > times: > {code} > public long splitLogDistributed(final List logDirs) throws > IOException { > MonitoredTask status = TaskMonitor.get().createStatus( > "Doing distributed log split in " + logDirs); > {code} > leading to multiple MonitoredTask instances. > User may get confused by multiple distributed log splitting entries for the > same region server on master UI -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5139) Compute (weighted) median using AggregateProtocol
[ https://issues.apache.org/jira/browse/HBASE-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5139: - Assignee: Zhihong Yu > Compute (weighted) median using AggregateProtocol > - > > Key: HBASE-5139 > URL: https://issues.apache.org/jira/browse/HBASE-5139 > Project: HBase > Issue Type: Sub-task >Reporter: Zhihong Yu >Assignee: Zhihong Yu > > Suppose cf:cq1 stores numeric values and optionally cf:cq2 stores weights. > This task finds out the median value among the values of cf:cq1 (See > http://www.stat.ucl.ac.be/ISdidactique/Rhelp/library/R.basic/html/weighted.median.html) > This can be done in two passes. > The first pass utilizes AggregateProtocol where the following tuple is > returned from each region: > (partial-sum-of-values, partial-sum-of-weights) > The start rowkey (supplied by coprocessor framework) would be used to sort > the tuples. This way we can determine which region (called R) contains the > (weighted) median. partial-sum-of-weights can be 0 if unweighted median is > sought > The second pass involves scanning the table, beginning with startrow of > region R and computing partial (weighted) sum until the threshold of S/2 is > crossed. The (weighted) median is returned. > However, this approach wouldn't work if there is mutation in the underlying > table between pass one and pass two. > In that case, sequential scanning seems to be the solution which is slower > than the above approach. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3565) Add a metric to keep track of slow HLog appends
[ https://issues.apache.org/jira/browse/HBASE-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-3565: - Assignee: Zhihong Yu > Add a metric to keep track of slow HLog appends > --- > > Key: HBASE-3565 > URL: https://issues.apache.org/jira/browse/HBASE-3565 > Project: HBase > Issue Type: Improvement > Components: metrics, regionserver >Reporter: Benoit Sigoure >Assignee: Zhihong Yu > Labels: monitoring > Fix For: 0.94.0 > > Attachments: HBASE-3565.trunk.v1.patch > > > Whenever an edit takes too long to be written to an HLog, HBase logs a > warning such as this one: > {code} > 2011-02-23 20:03:14,703 WARN org.apache.hadoop.hbase.regionserver.wal.HLog: > IPC Server handler 21 on 60020 took 15065ms appending an edit to hlog; > editcount=126050 > {code} > I would like to have a counter incremented each time this happens and this > counter exposed via the metrics stuff in HBase so I can collect it in my > monitoring system. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5182) TBoundedThreadPoolServer threadKeepAliveTimeSec is not configured properly
[ https://issues.apache.org/jira/browse/HBASE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5182: - Assignee: Scott Chen > TBoundedThreadPoolServer threadKeepAliveTimeSec is not configured properly > -- > > Key: HBASE-5182 > URL: https://issues.apache.org/jira/browse/HBASE-5182 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Scott Chen >Assignee: Scott Chen >Priority: Minor > Fix For: 0.94.0 > > Attachments: hbase-5182.txt > > > TBoundedThreadPoolServer does not take the configured threadKeepAliveTimeSec. > It uses the default value instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5181) Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)
[ https://issues.apache.org/jira/browse/HBASE-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5181: - Assignee: Mubarak Seyed > Improve error message when Master fail-over happens and ZK unassigned node > contains stale znode(s) > -- > > Key: HBASE-5181 > URL: https://issues.apache.org/jira/browse/HBASE-5181 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.92.0, 0.90.5 >Reporter: Mubarak Seyed >Assignee: Mubarak Seyed >Priority: Minor > Labels: noob > > When master fail-over happens, if we have number of RITs under > /hbase/unassigned and if we have stale znode(s) (encoded region names) under > /hbase/unassigned, we are getting > {code} > 2011-12-30 10:27:35,623 INFO org.apache.hadoop.hbase.master.HMaster: Master > startup proceeding: master failover > 2011-12-30 10:27:36,002 INFO > org.apache.hadoop.hbase.master.AssignmentManager: Failed-over master needs to > process 1717 regions in transition > 2011-12-30 10:27:36,004 FATAL org.apache.hadoop.hbase.master.HMaster: > Unhandled exception. Starting shutdown. > java.lang.ArrayIndexOutOfBoundsException: -256 > at > org.apache.hadoop.hbase.executor.RegionTransitionData.readFields(RegionTransitionData.java:148) > > at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:105) > at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) > at > org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198) > > at org.apache.hadoop.hbase.zookeeper.ZKAssign.getData(ZKAssign.java:743) > at > org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:262) > > at > org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:223) > > at > org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:401) > at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283) > {code} > and there is no clue on how to clean-up the stale znode(s) from unassigned > using zkCli.sh (del /hbase/unassigned/). It would be good if > we include the bad region name in IOException from > RegionTransitionData.readFields(). > {code} > @Override > public void readFields(DataInput in) throws IOException { > // the event type byte > eventType = EventType.values()[in.readShort()]; > // the timestamp > stamp = in.readLong(); > // the encoded name of the region being transitioned > regionName = Bytes.readByteArray(in); > // remaining fields are optional so prefixed with boolean > // the name of the regionserver sending the data > if (in.readBoolean()) { > byte [] versionedBytes = Bytes.readByteArray(in); > this.origin = ServerName.parseVersionedServerName(versionedBytes); > } > if (in.readBoolean()) { > this.payload = Bytes.readByteArray(in); > } > } > {code} > If the code execution has survived until regionName then we can include the > regionName in IOException with error message to clean-up the stale znode(s) > under /hbase/unassigned. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5143) Fix config typo in pluggable load balancer factory
[ https://issues.apache.org/jira/browse/HBASE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5143: - Assignee: Harsh J > Fix config typo in pluggable load balancer factory > -- > > Key: HBASE-5143 > URL: https://issues.apache.org/jira/browse/HBASE-5143 > Project: HBase > Issue Type: Sub-task > Components: master >Reporter: Harsh J >Assignee: Harsh J >Priority: Blocker > Fix For: 0.92.0, 0.94.0 > > Attachments: HBASE-5143.patch > > > HBASE-4240 made LoadBalancer pluggable. > Configuration it loads seems to be wrongly named and carries a typo: > "hbase.maser.loadBalancer.class" > Could rather be "hbase.master.loadbalancer.class" > Luckily 0.92 is not out yet and we should fix it asap, before folks start > using it. Attaching patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5176) AssignmentManager: getRegion: logging nit adds a redundant '+'
[ https://issues.apache.org/jira/browse/HBASE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5176: - Assignee: Karthik K > AssignmentManager: getRegion: logging nit adds a redundant '+' > - > > Key: HBASE-5176 > URL: https://issues.apache.org/jira/browse/HBASE-5176 > Project: HBase > Issue Type: Bug > Environment: hadoop 1.0.0 , zk 3.4.2 , hbase 0.92.0 rc3 >Reporter: Karthik K >Assignee: Karthik K >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-5176.patch > > > From the logs of HMaster: > 2012-01-10 17:28:24,370 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for > -ROOT-,,0.70236052 destination server is + localhost,60020,1326242475275 > Was the '+' intended to be there , as part of some token for log verification > or just being redundant , w.r.t the following string append ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3373) Allow regions to be load-balanced by table
[ https://issues.apache.org/jira/browse/HBASE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-3373: - Assignee: Zhihong Yu > Allow regions to be load-balanced by table > -- > > Key: HBASE-3373 > URL: https://issues.apache.org/jira/browse/HBASE-3373 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.20.6 >Reporter: Ted Yu >Assignee: Zhihong Yu > Fix For: 0.94.0 > > Attachments: 3373.txt, HbaseBalancerTest2.java > > > From our experience, cluster can be well balanced and yet, one table's > regions may be badly concentrated on few region servers. > For example, one table has 839 regions (380 regions at time of table > creation) out of which 202 are on one server. > It would be desirable for load balancer to distribute regions for specified > tables evenly across the cluster. Each of such tables has number of regions > many times the cluster size. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4913) Per-CF compaction Via the Shell
[ https://issues.apache.org/jira/browse/HBASE-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-4913: - Assignee: Mubarak Seyed > Per-CF compaction Via the Shell > --- > > Key: HBASE-4913 > URL: https://issues.apache.org/jira/browse/HBASE-4913 > Project: HBase > Issue Type: Sub-task > Components: client, regionserver >Reporter: Nicolas Spiegelberg >Assignee: Mubarak Seyed > Fix For: 0.94.0 > > Attachments: HBASE-4913.trunk.v1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5271) Result.getValue and Result.getColumnLatest return the wrong column.
[ https://issues.apache.org/jira/browse/HBASE-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5271: - Assignee: Ghais Issa > Result.getValue and Result.getColumnLatest return the wrong column. > --- > > Key: HBASE-5271 > URL: https://issues.apache.org/jira/browse/HBASE-5271 > Project: HBase > Issue Type: Bug > Components: client >Affects Versions: 0.90.5 >Reporter: Ghais Issa >Assignee: Ghais Issa > Fix For: 0.94.0, 0.90.6, 0.92.1 > > Attachments: 5271-90.txt, 5271-v2.txt, > fixKeyValueMatchingColumn.diff, testGetValue.diff > > > In the following example result.getValue returns the wrong column > KeyValue kv = new KeyValue(Bytes.toBytes("r"), Bytes.toBytes("24"), > Bytes.toBytes("2"), Bytes.toBytes(7L)); > Result result = new Result(new KeyValue[] { kv }); > System.out.println(Bytes.toLong(result.getValue(Bytes.toBytes("2"), > Bytes.toBytes("2"; //prints 7. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3134) [replication] Add the ability to enable/disable streams
[ https://issues.apache.org/jira/browse/HBASE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-3134: - Assignee: Teruyoshi Zenmyo > [replication] Add the ability to enable/disable streams > --- > > Key: HBASE-3134 > URL: https://issues.apache.org/jira/browse/HBASE-3134 > Project: HBase > Issue Type: New Feature > Components: replication >Reporter: Jean-Daniel Cryans >Assignee: Teruyoshi Zenmyo >Priority: Minor > Labels: replication > Attachments: HBASE-3134.patch > > > This jira was initially in the scope of HBASE-2201, but was pushed out since > it has low value compared to the required effort (and when want to ship > 0.90.0 rather soonish). > We need to design a way to enable/disable replication streams in a > determinate fashion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[ https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5290: - Assignee: Ben West (was: Liyin Tang) > [FindBugs] Synchronization on boxed primitive > - > > Key: HBASE-5290 > URL: https://issues.apache.org/jira/browse/HBASE-5290 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0 >Reporter: Liyin Tang >Assignee: Ben West >Priority: Minor > Fix For: 0.94.0 > > Attachments: 5290-v3.txt, 5290-v4.txt, HBASE-5290.patch, > HBASE-5290.patch, HBASE-5290v2.patch > > > This bug is reported by the findBugs tool, which is a static analysis tool. > Bug: Synchronization on Integer in > org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList() > The code synchronizes on a boxed primitive constant, such as an Integer. > {code} > private static Integer count = 0; > ... > synchronized(count) { > count++; > } > ... > {code} > Since Integer objects can be cached and shared, this code could be > synchronizing on the same object as other, unrelated code, leading to > unresponsiveness and possible deadlock > See CERT CON08-J. Do not synchronize on objects that may be reused for more > information. > Confidence: Normal, Rank: Troubling (14) > Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE > Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5212) Fix test TestTableMapReduce against 0.23.
[ https://issues.apache.org/jira/browse/HBASE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5212: - Assignee: Gregory Chanan > Fix test TestTableMapReduce against 0.23. > - > > Key: HBASE-5212 > URL: https://issues.apache.org/jira/browse/HBASE-5212 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Mahadev konar >Assignee: Gregory Chanan > Fix For: 0.92.1 > > Attachments: 5212-v2.txt, HBASE-5212-v3.patch, HBASE-5212.patch > > > As reported by Andrew on the hadoop mailing list, mvn -Dhadoop.profile=23 > clean test -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce fails > on 0.92 branch. There are minor changes to HBase poms required to fix that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3850) Log more details when a scanner lease expires
[ https://issues.apache.org/jira/browse/HBASE-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-3850: - Assignee: Darren Haas > Log more details when a scanner lease expires > - > > Key: HBASE-3850 > URL: https://issues.apache.org/jira/browse/HBASE-3850 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Benoit Sigoure >Assignee: Darren Haas >Priority: Critical > Attachments: HBASE-3850.trunk.v1.patch > > > The message logged by the RegionServer when a Scanner lease expires isn't as > useful as it could be. {{Scanner 4765412385779771089 lease expired}} - most > clients don't log their scanner ID, so it's really hard to figure out what > was going on. I think it would be useful to at least log the name of the > region on which the Scanner was open, and it would be great to have the > ip:port of the client that had that lease too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4966) Put/Delete values cannot be tested with MRUnit
[ https://issues.apache.org/jira/browse/HBASE-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-4966: - Assignee: Nicholas Telford > Put/Delete values cannot be tested with MRUnit > -- > > Key: HBASE-4966 > URL: https://issues.apache.org/jira/browse/HBASE-4966 > Project: HBase > Issue Type: Bug > Components: client, mapreduce >Affects Versions: 0.90.4 >Reporter: Nicholas Telford >Assignee: Nicholas Telford >Priority: Minor > > When using the IdentityTableReducer, which expects input values of either a > Put or Delete object, testing with MRUnit the Mapper with MRUnit is not > possible because neither Put nor Delete implement equals(). > We should implement equals() on both such that equality means: > * Both objects are of the same class (in this case, Put or Delete) > * Both objects are for the same key. > * Both objects contain an equal set of KeyValues (applicable only to Put) > KeyValue.equals() appears to already be implemented, but only checks for > equality of row key, column family and column qualifier - two KeyValues can > be considered "equal" if they contain different values. This won't work for > testing. > Instead, the Put.equals() and Delete.equals() implementations should do a > "deep" equality check on their KeyValues, like this: > {code:java} > myKv.equals(theirKv) && Bytes.equals(myKv.getValue(), theirKv.getValue()); > {code} > NOTE: This would impact any code that relies on the existing "identity" > implementation of Put.equals() and Delete.equals(), therefore cannot be > guaranteed to be backwards-compatible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[ https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5329: - Assignee: Zhihong Yu > addRowLock() may allocate duplicate lock id, causing the client to be blocked > - > > Key: HBASE-5329 > URL: https://issues.apache.org/jira/browse/HBASE-5329 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.90.3 > Environment: Red Hat Enterprise Linux Server release 5.4 >Reporter: liaoxiangui >Assignee: Zhihong Yu >Priority: Critical > > {code} > protected long addRowLock(Integer r, HRegion region) throws > LeaseStillHeldException > { > long lockId = -1L; > lockId = rand.nextLong(); //!!!may generate duplicated > id,bug? > String lockName = String.valueOf(lockId); > rowlocks.put(lockName, r); > this.leases.createLease(lockName, new RowLockListener(lockName, > region)); > return lockId; > } > {code} > In addRowLock(),rand may generate duplicated lock id, it may cause > regionserver throw exception(Leases$LeaseStillHeldException).The client will > be blocked until old rowlock is released. > {code} > 2012-02-03 15:21:50,084 ERROR > org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock > (fsOk: true) > org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException > at > org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5325) Expose basic information about the master-status through jmx beans
[ https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5325: - Assignee: Hitesh Shah > Expose basic information about the master-status through jmx beans > --- > > Key: HBASE-5325 > URL: https://issues.apache.org/jira/browse/HBASE-5325 > Project: HBase > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.94.0 > > Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch > > > Similar to the Namenode and Jobtracker, it would be good if the hbase master > could expose some information through mbeans. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4991) Provide capability to delete named region
[ https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-4991: - Assignee: Mubarak Seyed > Provide capability to delete named region > - > > Key: HBASE-4991 > URL: https://issues.apache.org/jira/browse/HBASE-4991 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Mubarak Seyed > Fix For: 0.94.0 > > > See discussion titled 'Able to control routing to Solr shards or not' on > lily-discuss > User may want to quickly dispose of out of date records by deleting specific > regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5387) Reuse compression streams in HFileBlock.Writer
[ https://issues.apache.org/jira/browse/HBASE-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5387: - Assignee: Mikhail Bautin > Reuse compression streams in HFileBlock.Writer > -- > > Key: HBASE-5387 > URL: https://issues.apache.org/jira/browse/HBASE-5387 > Project: HBase > Issue Type: Bug >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: Fix-deflater-leak-2012-02-10_18_48_45.patch > > > We need to to reuse compression streams in HFileBlock.Writer instead of > allocating them every time. The motivation is that when using Java's built-in > implementation of Gzip, we allocate a new GZIPOutputStream object and an > associated native data structure any time. This is one suspected cause of > recent TestHFileBlock failures on Hadoop QA: > https://builds.apache.org/job/HBase-TRUNK/2658/testReport/org.apache.hadoop.hbase.io.hfile/TestHFileBlock/testPreviousOffset_1_/. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5388) tunning getCachedLocation method
[ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5388: - Assignee: ronghai.ma > tunning getCachedLocation method > > > Key: HBASE-5388 > URL: https://issues.apache.org/jira/browse/HBASE-5388 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.90.5 >Reporter: ronghai.ma >Assignee: ronghai.ma > Labels: patch > Fix For: 0.94.0 > > > About 75% improvement in execution time. > 1. Add a method in SoftValueSortedMap > {code} > public synchronized Entry lowerEntry(K key) { > return ((TreeMap) this.internalMap).lowerEntry(key); > } > {code} > 2. Modify getCachedLocation > {code} > Map.Entry tEntry = tableLocations.lowerEntry(row); > if (tEntry != null) { > HRegionLocation possibleRegion = tEntry.getValue(); > //other code > } > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5279) NPE in Master after upgrading to 0.92.0
[ https://issues.apache.org/jira/browse/HBASE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5279: - Assignee: Tobias Herbert > NPE in Master after upgrading to 0.92.0 > --- > > Key: HBASE-5279 > URL: https://issues.apache.org/jira/browse/HBASE-5279 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.92.0 >Reporter: Tobias Herbert >Assignee: Tobias Herbert >Priority: Critical > Fix For: 0.92.1 > > Attachments: HBASE-5279-v2.patch, HBASE-5279.patch > > > I have upgraded my environment from 0.90.4 to 0.92.0 > after the table migration I get the following error in the master (permanent) > {noformat} > 2012-01-25 18:23:48,648 FATAL master-namenode,6,1327512209588 > org.apache.hadoop.hbase.master.HMaster - Unhandled exception. Starting > shutdown. > java.lang.NullPointerException > at > org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2190) > at > org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:323) > at > org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:501) > at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326) > at java.lang.Thread.run(Thread.java:662) > 2012-01-25 18:23:48,650 INFO namenode,6,1327512209588 > org.apache.hadoop.hbase.master.HMaster - Aborting > {noformat} > I think that's because I had a hard crash in the cluster a while ago - and > the following WARN since then > {noformat} > 2012-01-25 21:20:47,121 WARN namenode,6,1327513078123-CatalogJanitor > org.apache.hadoop.hbase.master.CatalogJanitor - REGIONINFO_QUALIFIER is empty > in keyvalues={emails,,xxx./info:server/1314336400471/Put/vlen=38, > emails,,1314189353300.xxx./info:serverstartcode/1314336400471/Put/vlen=8} > {noformat} > my patch was simple to go around the NPE (as the other code around the lines) > but I don't know if that's correct -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5425) Punt on the timeout doesn't work in BulkEnabler#waitUntilDone (master's EnableTableHandler)
[ https://issues.apache.org/jira/browse/HBASE-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5425: - Assignee: terry zhang > Punt on the timeout doesn't work in BulkEnabler#waitUntilDone (master's > EnableTableHandler) > > > Key: HBASE-5425 > URL: https://issues.apache.org/jira/browse/HBASE-5425 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.5, 0.92.0 >Reporter: terry zhang >Assignee: terry zhang > Fix For: 0.94.0 > > Attachments: HBASE-5425.patch > > > please take a look at the code below in EnableTableHandler(hbase master): > {code:title=EnableTableHandler.java|borderStyle=solid} > protected boolean waitUntilDone(long timeout) > throws InterruptedException { > > . > int lastNumberOfRegions = this.countOfRegionsInTable; > while (!server.isStopped() && remaining > 0) { > Thread.sleep(waitingTimeForEvents); > regions = assignmentManager.getRegionsOfTable(tableName); > if (isDone(regions)) break; > // Punt on the timeout as long we make progress > if (regions.size() > lastNumberOfRegions) { > lastNumberOfRegions = regions.size(); > timeout += waitingTimeForEvents; > } > remaining = timeout - (System.currentTimeMillis() - startTime); > > } > private boolean isDone(final List regions) { > return regions != null && regions.size() >= this.countOfRegionsInTable; > } > {code} > We can easily find out if we let lastNumberOfRegions = > this.countOfRegionsInTable , the function of punt on timeout code will never > be executed. I think initlize lastNumberOfRegions = 0 can make it work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5466) Opening a table also opens the metatable and never closes it.
[ https://issues.apache.org/jira/browse/HBASE-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5466: - Assignee: Ashley Taylor > Opening a table also opens the metatable and never closes it. > - > > Key: HBASE-5466 > URL: https://issues.apache.org/jira/browse/HBASE-5466 > Project: HBase > Issue Type: Bug > Components: client >Affects Versions: 0.90.5, 0.92.0 >Reporter: Ashley Taylor >Assignee: Ashley Taylor > Fix For: 0.92.1 > > Attachments: MetaScanner_HBASE_5466(2).patch, > MetaScanner_HBASE_5466(3).patch, MetaScanner_HBASE_5466.patch > > > Having upgraded to CDH3U3 version of hbase we found we had a zookeeper > connection leak, tracking it down we found that closing the connection will > only close the zookeeper connection if all calls to get the connection have > been closed, there is incCount and decCount in the HConnection class, > When a table is opened it makes a call to the metascanner class which opens a > connection to the meta table, this table never gets closed. > This caused the count in the HConnection class to never return to zero > meaning that the zookeeper connection will not close when we close all the > tables or calling > HConnectionManager.deleteConnection(config, true); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5480) Fixups to MultithreadedTableMapper for Hadoop 0.23.2+
[ https://issues.apache.org/jira/browse/HBASE-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5480: - Assignee: Andrew Purtell Do you want to commit the patch Andy ? > Fixups to MultithreadedTableMapper for Hadoop 0.23.2+ > - > > Key: HBASE-5480 > URL: https://issues.apache.org/jira/browse/HBASE-5480 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Critical > Attachments: HBASE-5480.patch > > > There are two issues: > - StatusReporter has a new method getProgress() > - Mapper and reducer context objects can no longer be directly instantiated. > See attached patch. I'm not thrilled with the added reflection but it was the > minimally intrusive change. > Raised the priority to critical because compilation fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5510) Pass region info in LoadBalancer.randomAssignment(List servers)
[ https://issues.apache.org/jira/browse/HBASE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5510: - Assignee: Anoop Sam John (was: ramkrishna.s.vasudevan) Integrated to TRUNK. Thanks for the patch Anoop. > Pass region info in LoadBalancer.randomAssignment(List servers) > --- > > Key: HBASE-5510 > URL: https://issues.apache.org/jira/browse/HBASE-5510 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.96.0 > > Attachments: HBase-5010_3.patch, HBase-5510.patch, HBase-5510_2.patch > > > In LB there is randomAssignment(List) API which will be > used by AM to assign > a region from a down RS. [This will be also used in other cases like call to > assign() API from client] > I feel it would be better to pass the HRegionInfo also into this method. > When the LB making a choice for a region > assignment, when one RS is down, it would be nice that the LB knows for > which region it is doing this server selection. > +Scenario+ > While one RS down, we wanted the regions to get moved to other RSs but a set > of regions stay together. We are having custom load balancer but with the > current way of LB interface this is not possible. Another way is I can allow > a random assignment of the regions at the RS down time. Later with a cluster > balance I can balance the regions as I need. But this might make regions > assign 1st to one RS and then again move to another. Also for some time > period my business use case can not get satisfied. > Also I have seen some issue in JIRA which speaks about making sure that Root > and META regions always sit in some specific RSs. With the current LB API > this wont be possible in future. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4608) HLog Compression
[ https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-4608: - Assignee: Zhihong Yu (was: Li Pi) > HLog Compression > > > Key: HBASE-4608 > URL: https://issues.apache.org/jira/browse/HBASE-4608 > Project: HBase > Issue Type: New Feature >Reporter: Li Pi >Assignee: Zhihong Yu > Fix For: 0.94.0 > > Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, > 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, > 4608v18.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt > > > The current bottleneck to HBase write speed is replicating the WAL appends > across different datanodes. We can speed up this process by compressing the > HLog. Current plan involves using a dictionary to compress table name, region > id, cf name, and possibly other bits of repeated data. Also, HLog format may > be changed in other ways to produce a smaller HLog. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4608) HLog Compression
[ https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-4608: - Assignee: Li Pi (was: Zhihong Yu) > HLog Compression > > > Key: HBASE-4608 > URL: https://issues.apache.org/jira/browse/HBASE-4608 > Project: HBase > Issue Type: New Feature >Reporter: Li Pi >Assignee: Li Pi > Fix For: 0.94.0 > > Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, > 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, > 4608v18.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt > > > The current bottleneck to HBase write speed is replicating the WAL appends > across different datanodes. We can speed up this process by compressing the > HLog. Current plan involves using a dictionary to compress table name, region > id, cf name, and possibly other bits of repeated data. Also, HLog format may > be changed in other ways to produce a smaller HLog. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5206) Port HBASE-5155 to 0.92 and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5206: - Assignee: Ashutosh Jindal > Port HBASE-5155 to 0.92 and TRUNK > - > > Key: HBASE-5206 > URL: https://issues.apache.org/jira/browse/HBASE-5206 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.2, 0.96.0 >Reporter: Zhihong Yu >Assignee: Ashutosh Jindal > Attachments: 5206_92_1.patch, 5206_92_latest_1.patch, > 5206_92_latest_2.patch, 5206_trunk-v3.patch, 5206_trunk_1.patch, > 5206_trunk_latest_1.patch > > > This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should > not happen parallely leading to recreation of regions that were deleted) to > 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5579) A Delete Version could mask other values
[ https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5579: - Assignee: Daniel Gómez Ferro > A Delete Version could mask other values > > > Key: HBASE-5579 > URL: https://issues.apache.org/jira/browse/HBASE-5579 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Daniel Gómez Ferro >Assignee: Daniel Gómez Ferro > Fix For: 0.94.0, 0.96.0 > > Attachments: HBASE-5579.patch > > > A Delete Version operation mask values that have version = 0. The problem > happens at ScanDeleteTracker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-3996: - Assignee: Eran Kutner > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Eran Kutner > Fix For: 0.96.0 > > Attachments: 3996-v2.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5625) Avoid byte buffer allocations when reading a value from a Result object
[ https://issues.apache.org/jira/browse/HBASE-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5625: - Assignee: Tudor Scurtu > Avoid byte buffer allocations when reading a value from a Result object > --- > > Key: HBASE-5625 > URL: https://issues.apache.org/jira/browse/HBASE-5625 > Project: HBase > Issue Type: Improvement > Components: client >Affects Versions: 0.92.1 >Reporter: Tudor Scurtu >Assignee: Tudor Scurtu > Labels: patch > Attachments: 5625.txt, 5625v2.txt, 5625v3.txt > > > When calling Result.getValue(), an extra dummy KeyValue and its associated > underlying byte array are allocated, as well as a persistent buffer that will > contain the returned value. > These can be avoided by reusing a static array for the dummy object and by > passing a ByteBuffer object as a value destination buffer to the read method. > The current functionality is maintained, and we have added a separate method > call stack that employs the described changes. I will provide more details > with the patch. > Running tests with a profiler, the reduction of read time seems to be of up > to 40%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5669) AggregationClient fails validation for open stoprow scan
[ https://issues.apache.org/jira/browse/HBASE-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5669: - Assignee: Mubarak Seyed > AggregationClient fails validation for open stoprow scan > > > Key: HBASE-5669 > URL: https://issues.apache.org/jira/browse/HBASE-5669 > Project: HBase > Issue Type: Bug > Components: coprocessors >Affects Versions: 0.92.1 > Environment: n/a >Reporter: Brian Rogers >Assignee: Mubarak Seyed >Priority: Minor > Fix For: 0.92.2 > > Attachments: HBASE-5669.trunk.v1.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > AggregationClient.validateParameters throws an exception when the Scan has a > valid startrow but an unset endrow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags
[ https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5667: - Assignee: David Arthur > RegexStringComparator supports java.util.regex.Pattern flags > > > Key: HBASE-5667 > URL: https://issues.apache.org/jira/browse/HBASE-5667 > Project: HBase > Issue Type: Improvement > Components: filters >Reporter: David Arthur >Assignee: David Arthur >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff > > > * Add constructor that takes in a Pattern > * Add Pattern's flags to Writable fields, and actually use them when > recomposing the Filter -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5690) compression does not work in Store.java of 0.94
[ https://issues.apache.org/jira/browse/HBASE-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5690: - Assignee: honghua zhu > compression does not work in Store.java of 0.94 > --- > > Key: HBASE-5690 > URL: https://issues.apache.org/jira/browse/HBASE-5690 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.0 > Environment: all >Reporter: honghua zhu >Assignee: honghua zhu >Priority: Critical > Fix For: 0.94.1 > > Attachments: Store.patch > > > HBASE-5442 The store.createWriterInTmp method missing "compression" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5663) MultithreadedTableMapper doesn't work.
[ https://issues.apache.org/jira/browse/HBASE-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5663: - Assignee: Takuya Ueshin > MultithreadedTableMapper doesn't work. > -- > > Key: HBASE-5663 > URL: https://issues.apache.org/jira/browse/HBASE-5663 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Takuya Ueshin >Assignee: Takuya Ueshin > Fix For: 0.94.0, 0.96.0 > > Attachments: 5663+5636.txt, HBASE-5663.patch > > > MapReduce job using MultithreadedTableMapper goes down throwing the following > Exception: > {noformat} > java.io.IOException: java.lang.NoSuchMethodException: > org.apache.hadoop.mapreduce.Mapper$Context.(org.apache.hadoop.conf.Configuration, > org.apache.hadoop.mapred.TaskAttemptID, > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader, > > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter, > org.apache.hadoop.hbase.mapreduce.TableOutputCommitter, > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter, > org.apache.hadoop.hbase.mapreduce.TableSplit) > at > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.(MultithreadedTableMapper.java:260) > at > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper.run(MultithreadedTableMapper.java:133) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.NoSuchMethodException: > org.apache.hadoop.mapreduce.Mapper$Context.(org.apache.hadoop.conf.Configuration, > org.apache.hadoop.mapred.TaskAttemptID, > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader, > > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter, > org.apache.hadoop.hbase.mapreduce.TableOutputCommitter, > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter, > org.apache.hadoop.hbase.mapreduce.TableSplit) > at java.lang.Class.getConstructor0(Class.java:2706) > at java.lang.Class.getConstructor(Class.java:1657) > at > org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.(MultithreadedTableMapper.java:241) > ... 8 more > {noformat} > This occured when the tasks are creating MapRunner threads. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5636) TestTableMapReduce doesn't work properly.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5636: - Assignee: Takuya Ueshin > TestTableMapReduce doesn't work properly. > - > > Key: HBASE-5636 > URL: https://issues.apache.org/jira/browse/HBASE-5636 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.92.1, 0.94.0 >Reporter: Takuya Ueshin >Assignee: Takuya Ueshin > Attachments: HBASE-5636-v2.patch, HBASE-5636.patch > > > No map function is called because there are no test data put before test > starts. > The following three tests are in the same situation: > - org.apache.hadoop.hbase.mapred.TestTableMapReduce > - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce > - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5606) SplitLogManger async delete node hangs log splitting when ZK connection is lost
[ https://issues.apache.org/jira/browse/HBASE-5606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5606: - Assignee: Prakash Khemani > SplitLogManger async delete node hangs log splitting when ZK connection is > lost > > > Key: HBASE-5606 > URL: https://issues.apache.org/jira/browse/HBASE-5606 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.92.0 >Reporter: Gopinathan A >Assignee: Prakash Khemani >Priority: Critical > Fix For: 0.92.2 > > Attachments: > 0001-HBASE-5606-SplitLogManger-async-delete-node-hangs-lo.patch, > 0001-HBASE-5606-SplitLogManger-async-delete-node-hangs-lo.patch > > > 1. One rs died, the servershutdownhandler found it out and started the > distributed log splitting; > 2. All tasks are failed due to ZK connection lost, so the all the tasks were > deleted asynchronously; > 3. Servershutdownhandler retried the log splitting; > 4. The asynchronously deletion in step 2 finally happened for new task > 5. This made the SplitLogManger in hanging state. > This leads to .META. region not assigened for long time > {noformat} > hbase-root-master-HOST-192-168-47-204.log.2012-03-14"(55413,79):2012-03-14 > 19:28:47,932 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: put up > splitlog task at znode > /hbase/splitlog/hdfs%3A%2F%2F192.168.47.205%3A9000%2Fhbase%2F.logs%2Flinux-114.site%2C60020%2C1331720381665-splitting%2Flinux-114.site%252C60020%252C1331720381665.1331752316170 > hbase-root-master-HOST-192-168-47-204.log.2012-03-14"(89303,79):2012-03-14 > 19:34:32,387 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: put up > splitlog task at znode > /hbase/splitlog/hdfs%3A%2F%2F192.168.47.205%3A9000%2Fhbase%2F.logs%2Flinux-114.site%2C60020%2C1331720381665-splitting%2Flinux-114.site%252C60020%252C1331720381665.1331752316170 > {noformat} > {noformat} > hbase-root-master-HOST-192-168-47-204.log.2012-03-14"(80417,99):2012-03-14 > 19:34:31,196 DEBUG > org.apache.hadoop.hbase.master.SplitLogManager$DeleteAsyncCallback: deleted > /hbase/splitlog/hdfs%3A%2F%2F192.168.47.205%3A9000%2Fhbase%2F.logs%2Flinux-114.site%2C60020%2C1331720381665-splitting%2Flinux-114.site%252C60020%252C1331720381665.1331752316170 > hbase-root-master-HOST-192-168-47-204.log.2012-03-14"(89456,99):2012-03-14 > 19:34:32,497 DEBUG > org.apache.hadoop.hbase.master.SplitLogManager$DeleteAsyncCallback: deleted > /hbase/splitlog/hdfs%3A%2F%2F192.168.47.205%3A9000%2Fhbase%2F.logs%2Flinux-114.site%2C60020%2C1331720381665-splitting%2Flinux-114.site%252C60020%252C1331720381665.1331752316170 > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5724: - Assignee: Teruyoshi Zenmyo > Row cache of KeyValue should be cleared in readFields(). > > > Key: HBASE-5724 > URL: https://issues.apache.org/jira/browse/HBASE-5724 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1 >Reporter: Teruyoshi Zenmyo >Assignee: Teruyoshi Zenmyo > Attachments: HBASE-5724.txt > > > KeyValue does not clear its row cache in reading new values (readFields()). > Therefore, If a KeyValue (kv) which caches its row bytes reads another > KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5717) Scanner metrics are only reported if you get to the end of a scanner
[ https://issues.apache.org/jira/browse/HBASE-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5717: - Assignee: Ian Varley > Scanner metrics are only reported if you get to the end of a scanner > > > Key: HBASE-5717 > URL: https://issues.apache.org/jira/browse/HBASE-5717 > Project: HBase > Issue Type: Bug > Components: client, metrics >Reporter: Ian Varley >Assignee: Ian Varley >Priority: Minor > Fix For: 0.94.0, 0.96.0 > > Attachments: 5717-v4.patch, ClientScanner_HBASE_5717-v2.patch, > ClientScanner_HBASE_5717-v3.patch, ClientScanner_HBASE_5717.patch > > Original Estimate: 4h > Remaining Estimate: 4h > > When you turn on Scanner Metrics, the metrics are currently only made > available if you run over all records available in the scanner. If you stop > iterating before the end, the values are never flushed into the metrics > object (in the Scan attribute). > Will supply a patch with fix and test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5821) Incorrect handling of null value in Coprocessor aggregation function min()
[ https://issues.apache.org/jira/browse/HBASE-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5821: - Assignee: Maryann Xue > Incorrect handling of null value in Coprocessor aggregation function min() > -- > > Key: HBASE-5821 > URL: https://issues.apache.org/jira/browse/HBASE-5821 > Project: HBase > Issue Type: Bug > Components: coprocessors >Affects Versions: 0.92.1 >Reporter: Maryann Xue >Assignee: Maryann Xue > Attachments: HBASE-5821.patch > > > Both in AggregateImplementation and AggregationClient, the evaluation of the > current minimum value is like: > min = (min == null || ci.compare(result, min) < 0) ? result : min; > The LongColumnInterpreter takes null value is treated as the least value, > while the above expression takes min as the greater value when it is null. > Thus, the real minimum value gets discarded if a null value comes later. > max() could also be wrong if a different ColumnInterpreter other than > LongColumnInterpreter treats null value differently (as the greatest). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine
[ https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5732: - Assignee: Devaraj Das > Remove the SecureRPCEngine and merge the security-related logic in the core > engine > -- > > Key: HBASE-5732 > URL: https://issues.apache.org/jira/browse/HBASE-5732 > Project: HBase > Issue Type: Improvement >Reporter: Devaraj Das >Assignee: Devaraj Das > Attachments: rpcengine-merge.patch > > > Remove the SecureRPCEngine and merge the security-related logic in the core > engine. Follow up to HBASE-5727. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira