[jira] [Commented] (HBASE-4818) HBase Shell - Add support for formatting row keys before output
[
https://issues.apache.org/jira/browse/HBASE-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247246#comment-13247246
]
Ben West commented on HBASE-4818:
-
I can work on adding this to the web UI if someone can suggest a place to store
the formatter preference.
Should it just be in hbase-site.xml?
HBase Shell - Add support for formatting row keys before output
---
Key: HBASE-4818
URL: https://issues.apache.org/jira/browse/HBASE-4818
Project: HBase
Issue Type: Improvement
Components: shell
Reporter: Eran Kampf
Priority: Trivial
Attachments: format3.patch, hbase-4818.patch
Original Estimate: 24h
Remaining Estimate: 24h
As many HBase users use binary row keys rather than strings to optimize
memory consumption displaying an escaped string in the HBase shell isn't
useful (and takes a lot of screen space)
Allowing user to provide a row key formatter as part of the scan\get commands
would allow developers to display the row key in a way thats makes sense for
them.
Example:
scan 'stats', { ROWFORMATTER = MyRowFormatter.new }
The row formatter simply gets the bytes array key and formats it to a string.
Its an easy change tomake with simple monkey-patching of the shell commands
but I would be happy to see it as part of the shell itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4818) HBase Shell - Add support for formatting row keys before output
[
https://issues.apache.org/jira/browse/HBASE-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240451#comment-13240451
]
Ben West commented on HBASE-4818:
-
The ReverseIDFormatter in that patch overrides the default formatter to display
row keys in reverse order.
Something which we will have to think about is how we can maintain usability
with these new formatters. Scans, for example, might not go in the order the
user predicts because the stored format is different from the displayed one.
Similarly with where regions split and so forth. Maybe we should require sort
order to be constant across formatted and unformatted row keys (which would
make the ReverseIDFormatter and probably most formatters impossible).
I'm not super familiar with the web UI, but it looks like the only spots we
display row keys is when we specify the start and end rows of each region, and
when we issue splits/compactions. So that shouldn't be too bad to change.
HBase Shell - Add support for formatting row keys before output
---
Key: HBASE-4818
URL: https://issues.apache.org/jira/browse/HBASE-4818
Project: HBase
Issue Type: Improvement
Components: shell
Reporter: Eran Kampf
Priority: Trivial
Attachments: format3.patch, hbase-4818.patch
Original Estimate: 24h
Remaining Estimate: 24h
As many HBase users use binary row keys rather than strings to optimize
memory consumption displaying an escaped string in the HBase shell isn't
useful (and takes a lot of screen space)
Allowing user to provide a row key formatter as part of the scan\get commands
would allow developers to display the row key in a way thats makes sense for
them.
Example:
scan 'stats', { ROWFORMATTER = MyRowFormatter.new }
The row formatter simply gets the bytes array key and formats it to a string.
Its an easy change tomake with simple monkey-patching of the shell commands
but I would be happy to see it as part of the shell itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4653) Master can't easily get rid of LZO compressed tables when the codec isn't available
[
https://issues.apache.org/jira/browse/HBASE-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239509#comment-13239509
]
Ben West commented on HBASE-4653:
-
Running .92.0rc3 we ran into this and the problem seems to be worse than
reported here. I couldn't alter the table because HBase told me it was enabled,
and I couldn't disable the table because HBase told me it was already disabled.
I ended up having to delete the node from zookeeper and HDFS, which was fine
because it happened in a dev environment, but would've been really bad if it
was in prod.
+1 to getting this fixed.
Master can't easily get rid of LZO compressed tables when the codec isn't
available
---
Key: HBASE-4653
URL: https://issues.apache.org/jira/browse/HBASE-4653
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.4
Reporter: Benoit Sigoure
Priority: Minor
I mistakenly created a table with {{COMPRESSION = LZO}}, and I don't have
LZO installed. I'm running a vanilla 0.90.4 release. The master is unable
to deploy the region of that table because the codec is missing. I can't get
rid of it. I can't drop the table from the shell, although it seems I could
disable it. Thankfully I found a workaround for this bug (see further below).
{code}
hbase(main):003:0 disable 'mytable'
0 row(s) in 1.1010 seconds
hbase(main):004:0 drop 'mytable'
[hung forever]
{code}
in the logs:
{code}
2011-10-22 03:05:42,153 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Instantiated mytable,,1319278131519.6eb6891a8b072402b5064f4cc68c210d.
2011-10-22 03:05:42,154 ERROR
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
of region=mytable,,1319278131519.6eb6891a8b072402b5064f4cc68c210d.
java.io.IOException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2573)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2562)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2550)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:272)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:156)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
com.hadoop.compression.lzo.LzoCodec
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm$1.getCodec(Compression.java:92)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:197)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.lang.ClassNotFoundException:
com.hadoop.compression.lzo.LzoCodec
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm$1.getCodec(Compression.java:87)
... 11 more
[...]
2011-10-22 03:15:30,620 DEBUG
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Waiting on region
to clear regions in transition;
mytable,,1319278131519.6eb6891a8b072402b5064f4cc68c210d. state=OPENING,
ts=1319278483001
2011-10-22 03:15:31,621 DEBUG
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Waiting on region
to clear regions in transition;
mytable,,1319278131519.6eb6891a8b072402b5064f4cc68c210d. state=OPENING,
ts=1319278483001
[repeat message above indefinitely every 1s]
{code}
I tried restarting HBase, no luck. How do I get rid of this table so I can
recreate it without {{COMPRESSION = LZO}}?
h2. Workaround
Change the schema for each family, restart HBase, drop the table.
{code}
hbase(main):004:0 alter 'mytable', {NAME = 'fam1', COMPRESSION = 'NONE'}
0 row(s) in 0.1160 seconds
hbase(main):005:0 alter
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232600#comment-13232600
]
Ben West commented on HBASE-5592:
-
I would prefer HBASE-5548 as well. Fine with closing this.
Make it easier to get a table from shell
Key: HBASE-5592
URL: https://issues.apache.org/jira/browse/HBASE-5592
Project: HBase
Issue Type: Improvement
Components: shell
Affects Versions: 0.94.0
Reporter: Ben West
Assignee: Ben West
Priority: Trivial
Labels: shell
Fix For: 0.92.2, 0.94.0
Attachments: publicTable.patch
The one argument constructor to HTable was removed at some point, which means
that you now have to pass in a Configuration to instantiate an HTable. This
is annoying for me when I create quick scripts.
This JIRA is a tiny patch which lets you get an HTable instance in the shell
by doing
{code}foo_table = @shell.hbase_table('foo').table{code}
Basically, it is changing table to be a public member rather than a private
one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5600) Make Endpoint Coprocessors Available from Thrift
[
https://issues.apache.org/jira/browse/HBASE-5600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232813#comment-13232813
]
Ben West commented on HBASE-5600:
-
I'm not entirely sure how to accomplish this in full generality. The most
obvious methods would involve sending java code through thrift, which is
probably a no-go. However, here is one simple coprocessor which I have that it
seems could be handled:
{code}
Batch.Call call = Batch.forMethod(IMyEndpoint.class, getRow,
rowKey.getBytes());
Mapbyte[], byte[] results = table.coprocessorExec(IMyEndpoint.class, null,
null, call);
{code}
We could create a thrift method to take the name of the class, method, and an
array of params and then call coprocessorExec. If this sounds reasonable, I can
try to allocate some time for a patch.
Alternatively, we could just require that you write a client for your
endpoints, and then thrift calls the client.
Make Endpoint Coprocessors Available from Thrift
Key: HBASE-5600
URL: https://issues.apache.org/jira/browse/HBASE-5600
Project: HBase
Issue Type: Improvement
Components: thrift
Reporter: Ben West
Priority: Minor
Labels: thrift
Currently, the only way to access an endpoint coprocessor via thrift is to
modify the schema and Thrift server for every coprocessor function. This is
annoying. It should be possible to use your coprocessors without having to
mangle HBase core code (since that's the point of coprocessors).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4818) HBase Shell - Add support for formatting row keys before output
[
https://issues.apache.org/jira/browse/HBASE-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226347#comment-13226347
]
Ben West commented on HBASE-4818:
-
Todd: I think since we're using JRuby the formatter can be a java class, right?
You'd just have --format=org.apache
But I guess we could store it as a table property.
(Btw: if the formatters are to be useful outside of shell, we'll need a revamp
of how they work. Right now, it just formats text without much knowledge of
what the text is - we'd probably want to have FormatKey() FormatColumn() etc.
methods. Which is a good idea anyway.)
HBase Shell - Add support for formatting row keys before output
---
Key: HBASE-4818
URL: https://issues.apache.org/jira/browse/HBASE-4818
Project: HBase
Issue Type: Improvement
Components: shell
Reporter: Eran Kampf
Priority: Trivial
Attachments: hbase-4818.patch
Original Estimate: 24h
Remaining Estimate: 24h
As many HBase users use binary row keys rather than strings to optimize
memory consumption displaying an escaped string in the HBase shell isn't
useful (and takes a lot of screen space)
Allowing user to provide a row key formatter as part of the scan\get commands
would allow developers to display the row key in a way thats makes sense for
them.
Example:
scan 'stats', { ROWFORMATTER = MyRowFormatter.new }
The row formatter simply gets the bytes array key and formats it to a string.
Its an easy change tomake with simple monkey-patching of the shell commands
but I would be happy to see it as part of the shell itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[
https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13196420#comment-13196420
]
Ben West commented on HBASE-5290:
-
@Zhihong: fine with me. I was thinking it would be good to change to a
primitive, but it's a public field so didn't want to break the API.
Maybe change to get/set for the public piece while we're breaking stuff? Not
sure what the HBase standard is.
[FindBugs] Synchronization on boxed primitive
-
Key: HBASE-5290
URL: https://issues.apache.org/jira/browse/HBASE-5290
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Liyin Tang
Assignee: Liyin Tang
Priority: Minor
Attachments: 5290-v3.txt, HBASE-5290.patch, HBASE-5290.patch,
HBASE-5290v2.patch
This bug is reported by the findBugs tool, which is a static analysis tool.
Bug: Synchronization on Integer in
org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
The code synchronizes on a boxed primitive constant, such as an Integer.
{code}
private static Integer count = 0;
...
synchronized(count) {
count++;
}
...
{code}
Since Integer objects can be cached and shared, this code could be
synchronizing on the same object as other, unrelated code, leading to
unresponsiveness and possible deadlock
See CERT CON08-J. Do not synchronize on objects that may be reused for more
information.
Confidence: Normal, Rank: Troubling (14)
Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[
https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13196461#comment-13196461
]
Ben West commented on HBASE-5290:
-
TestCompactSelection is a different package, so making it package private won't
help, right? (Unless JUnit has some magic here?)
[FindBugs] Synchronization on boxed primitive
-
Key: HBASE-5290
URL: https://issues.apache.org/jira/browse/HBASE-5290
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Liyin Tang
Assignee: Liyin Tang
Priority: Minor
Attachments: 5290-v3.txt, 5290-v4.txt, HBASE-5290.patch,
HBASE-5290.patch, HBASE-5290v2.patch
This bug is reported by the findBugs tool, which is a static analysis tool.
Bug: Synchronization on Integer in
org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
The code synchronizes on a boxed primitive constant, such as an Integer.
{code}
private static Integer count = 0;
...
synchronized(count) {
count++;
}
...
{code}
Since Integer objects can be cached and shared, this code could be
synchronizing on the same object as other, unrelated code, leading to
unresponsiveness and possible deadlock
See CERT CON08-J. Do not synchronize on objects that may be reused for more
information.
Confidence: Normal, Rank: Troubling (14)
Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[
https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13196483#comment-13196483
]
Ben West commented on HBASE-5290:
-
v4 looks good to me. +1.
[FindBugs] Synchronization on boxed primitive
-
Key: HBASE-5290
URL: https://issues.apache.org/jira/browse/HBASE-5290
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Liyin Tang
Assignee: Ben West
Priority: Minor
Fix For: 0.94.0
Attachments: 5290-v3.txt, 5290-v4.txt, HBASE-5290.patch,
HBASE-5290.patch, HBASE-5290v2.patch
This bug is reported by the findBugs tool, which is a static analysis tool.
Bug: Synchronization on Integer in
org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
The code synchronizes on a boxed primitive constant, such as an Integer.
{code}
private static Integer count = 0;
...
synchronized(count) {
count++;
}
...
{code}
Since Integer objects can be cached and shared, this code could be
synchronizing on the same object as other, unrelated code, leading to
unresponsiveness and possible deadlock
See CERT CON08-J. Do not synchronize on objects that may be reused for more
information.
Confidence: Normal, Rank: Troubling (14)
Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5228) [REST] Rip out transform feature
[
https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189220#comment-13189220
]
Ben West commented on HBASE-5228:
-
Thanks Andrew for this patch.
For others who might run into this: running a jstack on our REST server
revealed that virtually every thread was hung waiting on a table descriptor:
{code}
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:885)
- locked 0x00070218e7d0 (a
org.apache.hadoop.hbase.ipc.HBaseClient$Call)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
at $Proxy61.getHTableDescriptors(Unknown Source)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHTableDescriptor(HConnectionManager.java:1871)
at
org.apache.hadoop.hbase.client.HTable.getTableDescriptor(HTable.java:404)
at
org.apache.hadoop.hbase.client.HTablePool$PooledHTable.getTableDescriptor(HTablePool.java:344)
at
org.apache.hadoop.hbase.rest.RowResultGenerator.init(RowResultGenerator.java:64)
at
org.apache.hadoop.hbase.rest.ResultGenerator.fromRowSpec(ResultGenerator.java:35)
at org.apache.hadoop.hbase.rest.RowResource.get(RowResource.java:86)
{code}
So if your REST server is running slowly on an affected version, I guess try
running a jstack and see if you see this behavior.
[REST] Rip out transform feature
--
Key: HBASE-5228
URL: https://issues.apache.org/jira/browse/HBASE-5228
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0, 0.90.5
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.94.0, 0.92.1, 0.90.6
Attachments: HBASE-5228-0.92.patch, HBASE-5228-trunk.patch
The 'transform' feature, where REST can be instructed, via a table attribute,
to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of
column values before serving them up to a client or storing them into HBase,
was added some time ago at the request of Jack Levin. I have since come to
regret it, it was not a well thought out feature:
- This is really an application concern.
- It adds significant overhead to request processing: Periodically a
HBaseAdmin is used to retrieve the table descriptor, in order to scan through
table attributes for transformation directives.
I think it is best to rip it out, its a real problem area, and REST should be
no more concerned about data formats than the Java API.
I doubt anyone uses this, not even Jack. Will need to follow up with him to
confirm.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5228) [REST] Rip out transform feature
[ https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189267#comment-13189267 ] Ben West commented on HBASE-5228: - @Zhihong, Andrew: yes, that stack trace was prior to Andrew's patch. Installing the patch now, will update with results. [REST] Rip out transform feature -- Key: HBASE-5228 URL: https://issues.apache.org/jira/browse/HBASE-5228 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.94.0, 0.92.1, 0.90.6 Attachments: HBASE-5228-0.92.patch, HBASE-5228-trunk.patch The 'transform' feature, where REST can be instructed, via a table attribute, to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of column values before serving them up to a client or storing them into HBase, was added some time ago at the request of Jack Levin. I have since come to regret it, it was not a well thought out feature: - This is really an application concern. - It adds significant overhead to request processing: Periodically a HBaseAdmin is used to retrieve the table descriptor, in order to scan through table attributes for transformation directives. I think it is best to rip it out, its a real problem area, and REST should be no more concerned about data formats than the Java API. I doubt anyone uses this, not even Jack. Will need to follow up with him to confirm. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5228) [REST] Rip out transform feature
[ https://issues.apache.org/jira/browse/HBASE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189461#comment-13189461 ] Ben West commented on HBASE-5228: - @Andrew, Zhihong: Confirmed that I'm no longer seeing all those threads blocked on the HTableDescriptor call. I'm not seeing a huge performance improvement, but I switched boxes in between tests (and am having trouble reverting to 0.90.4), so if it's faster for you guys I'll believe that's the cause. [REST] Rip out transform feature -- Key: HBASE-5228 URL: https://issues.apache.org/jira/browse/HBASE-5228 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.94.0, 0.92.1, 0.90.6 Attachments: HBASE-5228-0.92.patch, HBASE-5228-trunk.patch The 'transform' feature, where REST can be instructed, via a table attribute, to apply a transformation (e.g. base64 encoding or decoding) to a (sub)set of column values before serving them up to a client or storing them into HBase, was added some time ago at the request of Jack Levin. I have since come to regret it, it was not a well thought out feature: - This is really an application concern. - It adds significant overhead to request processing: Periodically a HBaseAdmin is used to retrieve the table descriptor, in order to scan through table attributes for transformation directives. I think it is best to rip it out, its a real problem area, and REST should be no more concerned about data formats than the Java API. I doubt anyone uses this, not even Jack. Will need to follow up with him to confirm. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3373) Allow regions to be load-balanced by table
[ https://issues.apache.org/jira/browse/HBASE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13180633#comment-13180633 ] Ben West commented on HBASE-3373: - @Zhihong: Thank you for looking into this issue. Just to verify (since I'm not too familiar with the code): with this change, the balancing will be done at the per-table level, right? So if I have one big table, this may not address the issue (since a randomly chosen region from the table is unlikely to be the newly split one) but with lots of small tables the issue will be addressed. Correct? Allow regions to be load-balanced by table -- Key: HBASE-3373 URL: https://issues.apache.org/jira/browse/HBASE-3373 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.20.6 Reporter: Ted Yu Fix For: 0.94.0 Attachments: 3373.txt, HbaseBalancerTest2.java From our experience, cluster can be well balanced and yet, one table's regions may be badly concentrated on few region servers. For example, one table has 839 regions (380 regions at time of table creation) out of which 202 are on one server. It would be desirable for load balancer to distribute regions for specified tables evenly across the cluster. Each of such tables has number of regions many times the cluster size. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3373) Allow regions to be load-balanced by table
[ https://issues.apache.org/jira/browse/HBASE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13180675#comment-13180675 ] Ben West commented on HBASE-3373: - OK, thanks Zhihong, I was confusing which JIRA this was. +1 from me. Allow regions to be load-balanced by table -- Key: HBASE-3373 URL: https://issues.apache.org/jira/browse/HBASE-3373 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.20.6 Reporter: Ted Yu Fix For: 0.94.0 Attachments: 3373.txt, HbaseBalancerTest2.java From our experience, cluster can be well balanced and yet, one table's regions may be badly concentrated on few region servers. For example, one table has 839 regions (380 regions at time of table creation) out of which 202 are on one server. It would be desirable for load balancer to distribute regions for specified tables evenly across the cluster. Each of such tables has number of regions many times the cluster size. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3437) Allow Explicit Splits from the Shell
[ https://issues.apache.org/jira/browse/HBASE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13160952#comment-13160952 ] Ben West commented on HBASE-3437: - I'm not seeing this in 0.90.2-0.90.4 but I do see it in trunk. Is this correct? If so, could we update the fix versions on this ticket? I'm looking at split.rb which should have the additional parameter. You can see it's not in: * [0.90.2|http://svn.apache.org/viewvc/hbase/tags/0.90.2/src/main/ruby/shell/commands/split.rb?view=markup] * [0.90.3|http://svn.apache.org/viewvc/hbase/tags/0.90.3/src/main/ruby/shell/commands/split.rb?view=markup] * [0.90.4|http://svn.apache.org/viewvc/hbase/tags/0.90.4/src/main/ruby/shell/commands/split.rb?view=markup] But it is in: * [0.92RC0|http://svn.apache.org/viewvc/hbase/tags/0.92.0rc0/src/main/ruby/shell/commands/split.rb?view=markup] * [trunk|http://svn.apache.org/viewvc/hbase/trunk/src/main/ruby/shell/commands/split.rb?view=markup] Allow Explicit Splits from the Shell Key: HBASE-3437 URL: https://issues.apache.org/jira/browse/HBASE-3437 Project: HBase Issue Type: Improvement Components: shell Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.90.2, 0.92.0 Attachments: HBASE-3437.patch There is no way currently to issue explicit splits from the shell. You can only accomplish it through HBaseAdmin currently. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3373) Allow regions of specific table to be load-balanced
[
https://issues.apache.org/jira/browse/HBASE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13160177#comment-13160177
]
Ben West commented on HBASE-3373:
-
We're running 0.94 and ran into this. With 4 region servers, we had one table
with ~1800 regions, evenly balanced. We then used importtsv to import ~300
regions of a new table. We ended up with virtually all regions on one server;
when I look at the master's log it looks like there were 159 rebalances (which
makes sense); 123 were moving regions from the old table, and 26 moved new
table regions. The result is that about 90% of the regions of the new table are
on one server.
When I look at DefaultLoadBalancer.balanceCluster, it has:
{CODE}
// fetch in alternate order if there is new region server
if (emptyRegionServerPresent) {
fetchFromTail = !fetchFromTail;
}
{CODE}
so we're only doing the randomization stuff in HBASE-3609 if there's a new
region server? Is there a reason we don't do this all the time?
Allow regions of specific table to be load-balanced
---
Key: HBASE-3373
URL: https://issues.apache.org/jira/browse/HBASE-3373
Project: HBase
Issue Type: Improvement
Components: master
Affects Versions: 0.20.6
Reporter: Ted Yu
Attachments: HbaseBalancerTest2.java
From our experience, cluster can be well balanced and yet, one table's
regions may be badly concentrated on few region servers.
For example, one table has 839 regions (380 regions at time of table
creation) out of which 202 are on one server.
It would be desirable for load balancer to distribute regions for specified
tables evenly across the cluster. Each of such tables has number of regions
many times the cluster size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3373) Allow regions of specific table to be load-balanced
[
https://issues.apache.org/jira/browse/HBASE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13160277#comment-13160277
]
Ben West commented on HBASE-3373:
-
@Ted: Thanks! I think I was looking at trunk instead of .94, I see that in .94
it should be random:
{code}
ListHRegionInfo regions = randomize(server.getValue());
{code}
Your per-table balance will be very useful for this case. Look forward to
seeing it!
Allow regions of specific table to be load-balanced
---
Key: HBASE-3373
URL: https://issues.apache.org/jira/browse/HBASE-3373
Project: HBase
Issue Type: Improvement
Components: master
Affects Versions: 0.20.6
Reporter: Ted Yu
Attachments: HbaseBalancerTest2.java
From our experience, cluster can be well balanced and yet, one table's
regions may be badly concentrated on few region servers.
For example, one table has 839 regions (380 regions at time of table
creation) out of which 202 are on one server.
It would be desirable for load balancer to distribute regions for specified
tables evenly across the cluster. Each of such tables has number of regions
many times the cluster size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
