[jira] [Resolved] (HBASE-1960) Master should wait for DFS to come up when creating hbase.version
[
https://issues.apache.org/jira/browse/HBASE-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-1960.
---
Resolution: Fixed
Release Note: This has been addressed as part of HBASE-5003 and others.
Master should wait for DFS to come up when creating hbase.version
-
Key: HBASE-1960
URL: https://issues.apache.org/jira/browse/HBASE-1960
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Minor
Attachments: HBASE-1960-redux.patch, HBASE-1960.patch
The master does not wait for DFS to come up in the circumstance where the DFS
master is started for the first time after format and no datanodes have been
started yet.
{noformat}
2009-11-07 11:47:28,115 INFO org.apache.hadoop.hbase.master.HMaster:
vmName=Java HotSpot(TM) 64-Bit Server VM, vmVendor=Sun Microsystems Inc.,
vmVersion=14.2-b01
2009-11-07 11:47:28,116 INFO org.apache.hadoop.hbase.master.HMaster:
vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError,
-XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode,
-Dhbase.log.dir=/mnt/hbase/logs,
-Dhbase.log.file=hbase-root-master-ip-10-242-15-159.log,
-Dhbase.home.dir=/usr/local/hbase-0.20.1/bin/.., -Dhbase.id.str=root,
-Dhbase.root.logger=INFO,DRFA,
-Djava.library.path=/usr/local/hbase-0.20.1/bin/../lib/native/Linux-amd64-64]
2009-11-07 11:47:28,247 INFO org.apache.hadoop.hbase.master.HMaster: My
address is ip-10-242-15-159.ec2.internal:6
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer
Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/hbase/hbase.version could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
[...]
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery
for block null bad datanode[0] nodes == null
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Could not get
block locations. Source file /hbase/hbase.version - Aborting...
2009-11-07 11:47:28,729 FATAL org.apache.hadoop.hbase.master.HMaster: Not
starting HMaster because:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/hbase/hbase.version could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
{noformat}
Should probably sleep and retry the write a few times.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5480) Fixups to MultithreadedTableMapper for Hadoop 0.23.2+
[ https://issues.apache.org/jira/browse/HBASE-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-5480. --- Resolution: Fixed Fix Version/s: 0.94.0 Hadoop Flags: Reviewed Passes TestMulitthreadedTableMapper with 0.23 and default Hadoop profiles. @Stack: Into 0.92 branch too? (You open it yet?) Fixups to MultithreadedTableMapper for Hadoop 0.23.2+ - Key: HBASE-5480 URL: https://issues.apache.org/jira/browse/HBASE-5480 Project: HBase Issue Type: Bug Components: mapreduce Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Critical Fix For: 0.94.0 Attachments: HBASE-5480.patch There are two issues: - StatusReporter has a new method getProgress() - Mapper and reducer context objects can no longer be directly instantiated. See attached patch. I'm not thrilled with the added reflection but it was the minimally intrusive change. Raised the priority to critical because compilation fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5228) [REST] Rip out transform feature
[ https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-5228. --- Resolution: Fixed Hadoop Flags: Reviewed [REST] Rip out transform feature -- Key: HBASE-5228 URL: https://issues.apache.org/jira/browse/HBASE-5228 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.94.0, 0.92.1, 0.90.6 Attachments: HBASE-5228-0.92.patch, HBASE-5228-trunk.patch The 'transform' feature, where REST can be instructed, via a table attribute, to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of column values before serving them up to a client or storing them into HBase, was added some time ago at the request of Jack Levin. I have since come to regret it, it was not a well thought out feature: - This is really an application concern. - It adds significant overhead to request processing: Periodically a HBaseAdmin is used to retrieve the table descriptor, in order to scan through table attributes for transformation directives. I think it is best to rip it out, its a real problem area, and REST should be no more concerned about data formats than the Java API. I doubt anyone uses this, not even Jack. Will need to follow up with him to confirm. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-1788) [stargate] multipart binary encoding option
[ https://issues.apache.org/jira/browse/HBASE-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-1788. --- Resolution: Later Assignee: (was: Andrew Purtell) There doesn't seem to be any need for this, and nothing I require. [stargate] multipart binary encoding option --- Key: HBASE-1788 URL: https://issues.apache.org/jira/browse/HBASE-1788 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Priority: Trivial Add support for multipart/related encoded entity bodies. There is support already for binary encoding also but not multipart -- operations with application/octet-stream encoding are limited to single KVs. Multipart/related is not necessarily efficient. Timestamps and column names would be sent as X-headers mixed between the binary blobs, and the part headers and border adds more overhead. The only reason to support it is if someone cannot use protobufs as an alternative to XML. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2058) Coprocessors: CPU and memory limit policy enforcment via runtime weaving
[ https://issues.apache.org/jira/browse/HBASE-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-2058. --- Resolution: Later I suspect that given the available option of external coprocessor hosts (HBASE-4047), which enables things like resource reservation and limits via cgroups or similar OS-level facilities, there will never be sufficient impetus to undertake the difficult and research-y work described in this issue. But, marking as later as opposed to closing. Coprocessors: CPU and memory limit policy enforcment via runtime weaving Key: HBASE-2058 URL: https://issues.apache.org/jira/browse/HBASE-2058 Project: HBase Issue Type: Improvement Components: coprocessors Reporter: Andrew Purtell Use the ASM bytecode analysis and rewriting engine to impose some constraints on CPU and memory use. This is middle ground between arbitrary function and a locked down language. We will be given arbitrary bytecode input. It is acceptable to reject a class and abort coprocessor loading if it defies analysis at load time such that we have insufficient confidence about the result. Wrap allocations to simply disallow large allocations. Hook or add finalizers to keep a running tally of aggregate heap charge. Disallow allocation beyond policy limit. Weave CPU usage tracking into loop headers. Throw an uncatchable exception if time limits prescribed by policy are exceeded. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-904) Make client capable of riding over a cluster restart
[ https://issues.apache.org/jira/browse/HBASE-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-904. -- Resolution: Not A Problem Assignee: (was: Andrew Purtell) No longer a problem. Make client capable of riding over a cluster restart Key: HBASE-904 URL: https://issues.apache.org/jira/browse/HBASE-904 Project: HBase Issue Type: Sub-task Reporter: stack Currently, clients buried in REST server at least, are unable to recalibrate if the cluster is restarted on them. Fix it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-1911) Convert ad hoc Writable versioning to uses of VersionedWritable
[ https://issues.apache.org/jira/browse/HBASE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-1911. --- Resolution: Invalid This issue is out of date. Convert ad hoc Writable versioning to uses of VersionedWritable --- Key: HBASE-1911 URL: https://issues.apache.org/jira/browse/HBASE-1911 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor There are a number of instances of ad hoc versioning of Writables: HTD, HCD, etc. We should convert our classes to inherit VersionedWritable. See http://hadoop.apache.org/common/docs/r0.19.1/api/org/apache/hadoop/io/VersionedWritable.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4831) LRU stats thread should be a daemon thread
[ https://issues.apache.org/jira/browse/HBASE-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-4831. --- Resolution: Duplicate Duplicate of HBASE-4745, already resolved LRU stats thread should be a daemon thread -- Key: HBASE-4831 URL: https://issues.apache.org/jira/browse/HBASE-4831 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani I have seen the hung processes where the following was the only non-daemon thread LRU Statistics #0 prio=10 tid=0x2ab0bc04f800 nid=0x11ac waiting on condition [0x42f57000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x2aaab9a1c000 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4827) TestAdmin should clean up resources after tests
[ https://issues.apache.org/jira/browse/HBASE-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-4827. --- Resolution: Fixed Fix Version/s: 0.90.5 0.94.0 0.92.0 Committed to trunk, 0.92, and 0.90 branches. TestAdmin passes locally in all three. TestAdmin should clean up resources after tests --- Key: HBASE-4827 URL: https://issues.apache.org/jira/browse/HBASE-4827 Project: HBase Issue Type: Test Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.92.0, 0.94.0, 0.90.5 Attachments: HBASE-4827.patch TestAdmin creates a new HBaseAdmin for each test case but does not clean up resources afterward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2395) Coprocessors: implement propagation constraints
[
https://issues.apache.org/jira/browse/HBASE-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-2395.
---
Resolution: Duplicate
Dup of HBASE-4605
Coprocessors: implement propagation constraints
---
Key: HBASE-2395
URL: https://issues.apache.org/jira/browse/HBASE-2395
Project: HBase
Issue Type: New Feature
Components: coprocessors
Reporter: Andrew Purtell
Priority: Minor
Some fevered stuff I posted to hbase-dev@:
A coprocessor or coprocessors (which implement RegionObserver) could
implement propagation constraints in a pluggable manner, is one possibility.
Coprocessors also get an ioctl-like interface which could be the channel for
attaching constraints to KVs.
{quote}
First, what about attaching metadata (itself KVs) to KVs in the store, in a
way that it is efficient to look up the metadata for a given KV or set of KVs?
Second, what about the notion of references? For the case above specifically,
metadata on an inode KV that consists of a list of pointers to other KVs.
When deleting the inode KV -- one that fell off the tail of a stack of
versions -- at compaction time, then the store could follow the pointers and
delete the referenced values also. Or better, decrement a specified Long
encoded KV and then take the delete action on another specified KV (or set of
KVs) if the result = 0.
So just to be clear I'm not advocating building my use case into HBase -- it
is a motivating example -- but rather there is perhaps some interesting
generic primitives to consider here. They could support mechanisms for
referential integrity that people coming from RDBMS are quite familiar with.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
