[jira] [Updated] (HBASE-3863) Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
[
https://issues.apache.org/jira/browse/HBASE-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-3863:
-
Attachment: 3863.txt
Fix.
Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
-
Key: HBASE-3863
URL: https://issues.apache.org/jira/browse/HBASE-3863
Project: HBase
Issue Type: Bug
Reporter: stack
Fix For: 0.92.0
Attachments: 3863.txt
Failing with:
{code}
Error Message
org.apache.hadoop.hbase.HServerAddress cannot be cast to
org.apache.hadoop.hbase.ServerName
Stacktrace
java.lang.ClassCastException: org.apache.hadoop.hbase.HServerAddress cannot
be cast to org.apache.hadoop.hbase.ServerName
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixDupeAssignment(HBaseFsckRepair.java:59)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsckMeta(TestHBaseFsck.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-3863) Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
-
Key: HBASE-3863
URL: https://issues.apache.org/jira/browse/HBASE-3863
Project: HBase
Issue Type: Bug
Reporter: stack
Attachments: 3863.txt
Failing with:
{code}
Error Message
org.apache.hadoop.hbase.HServerAddress cannot be cast to
org.apache.hadoop.hbase.ServerName
Stacktrace
java.lang.ClassCastException: org.apache.hadoop.hbase.HServerAddress cannot be
cast to org.apache.hadoop.hbase.ServerName
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixDupeAssignment(HBaseFsckRepair.java:59)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsckMeta(TestHBaseFsck.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3863) Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
[
https://issues.apache.org/jira/browse/HBASE-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-3863.
--
Resolution: Fixed
Fix Version/s: 0.92.0
Assignee: stack
Committing small fix.
Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
-
Key: HBASE-3863
URL: https://issues.apache.org/jira/browse/HBASE-3863
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.92.0
Attachments: 3863.txt
Failing with:
{code}
Error Message
org.apache.hadoop.hbase.HServerAddress cannot be cast to
org.apache.hadoop.hbase.ServerName
Stacktrace
java.lang.ClassCastException: org.apache.hadoop.hbase.HServerAddress cannot
be cast to org.apache.hadoop.hbase.ServerName
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixDupeAssignment(HBaseFsckRepair.java:59)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsckMeta(TestHBaseFsck.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3855) Performance degradation of memstore because reseek is linear
[ https://issues.apache.org/jira/browse/HBASE-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13029787#comment-13029787 ] dhruba borthakur commented on HBASE-3855: - I have a cf with three columns. I insert 100K puts on a single rowkey with random column values. all these kvs are cached in the memstore. A single scan to retrieve the values for two specified columnname takes about 4 millisecons of CPU without this patch and about 0.025 millieseconds of cpu with this patch. Unfortunately, I cannot share the test program because it is intertwined with quite a few non-opensource-able code. However, I think the patch will benefit all cases because it is better to do a O(logn) lookup rather then even a small number of sequential scans. Performance degradation of memstore because reseek is linear Key: HBASE-3855 URL: https://issues.apache.org/jira/browse/HBASE-3855 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Priority: Blocker Fix For: 0.94.0 Attachments: memstoreReseek.txt The scanner use reseek to find the next row (or next column) as part of a scan. The reseek code iterates over a Set to position itself at the right place. If there are many thousands of kvs that need to be skipped over, then the time-cost is very high. In this case, a seek would be far lesser in cost than a reseek. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3855) Performance degradation of memstore because reseek is linear
[ https://issues.apache.org/jira/browse/HBASE-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HBASE-3855: Attachment: memstoreReseek2.txt A minor improvement from the previous patch 1. invoke seek only if the getNext()call returned null and numIterReseek is zero. Performance degradation of memstore because reseek is linear Key: HBASE-3855 URL: https://issues.apache.org/jira/browse/HBASE-3855 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Priority: Blocker Fix For: 0.94.0 Attachments: memstoreReseek.txt, memstoreReseek2.txt The scanner use reseek to find the next row (or next column) as part of a scan. The reseek code iterates over a Set to position itself at the right place. If there are many thousands of kvs that need to be skipped over, then the time-cost is very high. In this case, a seek would be far lesser in cost than a reseek. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nichole Treadway updated HBASE-3691: Attachment: hbase-snappy-3691-trunk-003.patch Thanks for the patch...I made a few additional changes in HColumnDescriptor, and I updated the test files to include snappy. I noticed there are places in the hbase.avro classes where snappy support would need to be added in. Is it ok to add these changes in the patch, or do the avro classes need to be auto-generated somehow? Add compressor support for 'snappy', google's compressor Key: HBASE-3691 URL: https://issues.apache.org/jira/browse/HBASE-3691 Project: HBase Issue Type: Task Reporter: stack Priority: Critical Fix For: 0.92.0 Attachments: hbase-snappy-3691-trunk-002.patch, hbase-snappy-3691-trunk.patch http://code.google.com/p/snappy/ is apache licensed. bq. Snappy is a compression/decompression library. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, Snappy is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses at about 500 MB/sec or more. bq. Snappy is widely used inside Google, in everything from BigTable and MapReduce to our internal RPC systems. (Snappy has previously been referred to as Zippy in some presentations and the likes.) Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nichole Treadway updated HBASE-3691: Attachment: (was: hbase-snappy-3691-trunk-003.patch) Add compressor support for 'snappy', google's compressor Key: HBASE-3691 URL: https://issues.apache.org/jira/browse/HBASE-3691 Project: HBase Issue Type: Task Reporter: stack Priority: Critical Fix For: 0.92.0 Attachments: hbase-snappy-3691-trunk-002.patch, hbase-snappy-3691-trunk.patch http://code.google.com/p/snappy/ is apache licensed. bq. Snappy is a compression/decompression library. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, Snappy is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses at about 500 MB/sec or more. bq. Snappy is widely used inside Google, in everything from BigTable and MapReduce to our internal RPC systems. (Snappy has previously been referred to as Zippy in some presentations and the likes.) Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nichole Treadway updated HBASE-3691: Attachment: hbase-snappy-3691-trunk-003.patch Accidentally selected wrong license option. Add compressor support for 'snappy', google's compressor Key: HBASE-3691 URL: https://issues.apache.org/jira/browse/HBASE-3691 Project: HBase Issue Type: Task Reporter: stack Priority: Critical Fix For: 0.92.0 Attachments: hbase-snappy-3691-trunk-002.patch, hbase-snappy-3691-trunk-003.patch, hbase-snappy-3691-trunk.patch http://code.google.com/p/snappy/ is apache licensed. bq. Snappy is a compression/decompression library. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, Snappy is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses at about 500 MB/sec or more. bq. Snappy is widely used inside Google, in everything from BigTable and MapReduce to our internal RPC systems. (Snappy has previously been referred to as Zippy in some presentations and the likes.) Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3855) Performance degradation of memstore because reseek is linear
[ https://issues.apache.org/jira/browse/HBASE-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3855: - Resolution: Fixed Fix Version/s: (was: 0.94.0) 0.92.0 Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Committed to TRUNK. Thanks for the patch Dhruba. Performance degradation of memstore because reseek is linear Key: HBASE-3855 URL: https://issues.apache.org/jira/browse/HBASE-3855 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Priority: Blocker Fix For: 0.92.0 Attachments: memstoreReseek.txt, memstoreReseek2.txt The scanner use reseek to find the next row (or next column) as part of a scan. The reseek code iterates over a Set to position itself at the right place. If there are many thousands of kvs that need to be skipped over, then the time-cost is very high. In this case, a seek would be far lesser in cost than a reseek. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3850) Log more details when a scanner lease expires
[
https://issues.apache.org/jira/browse/HBASE-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-3850:
-
Priority: Critical (was: Minor)
Tags: noob
Log more details when a scanner lease expires
-
Key: HBASE-3850
URL: https://issues.apache.org/jira/browse/HBASE-3850
Project: HBase
Issue Type: Improvement
Components: regionserver
Reporter: Benoit Sigoure
Priority: Critical
The message logged by the RegionServer when a Scanner lease expires isn't as
useful as it could be. {{Scanner 4765412385779771089 lease expired}} - most
clients don't log their scanner ID, so it's really hard to figure out what
was going on. I think it would be useful to at least log the name of the
region on which the Scanner was open, and it would be great to have the
ip:port of the client that had that lease too.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3417) CacheOnWrite is using the temporary output path for block names, need to use a more consistent block naming scheme
[ https://issues.apache.org/jira/browse/HBASE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3417: - Status: Patch Available (was: Open) This patch did not go in. Want me to massage it in? CacheOnWrite is using the temporary output path for block names, need to use a more consistent block naming scheme -- Key: HBASE-3417 URL: https://issues.apache.org/jira/browse/HBASE-3417 Project: HBase Issue Type: Bug Components: io, regionserver Affects Versions: 0.92.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Critical Fix For: 0.92.0 Attachments: HBASE-3417-v1.patch, HBASE-3417-v2.patch, HBASE-3417-v5.patch Currently the block names used in the block cache are built using the filesystem path. However, for cache on write, the path is a temporary output file. The original COW patch actually made some modifications to block naming stuff to make it more consistent but did not do enough. Should add a separate method somewhere for generating block names using some more easily mocked scheme (rather than just raw path as we generate a random unique file name twice, once for tmp and then again when moved into place). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-2765) Master should display currently unassigned regions in UI
[ https://issues.apache.org/jira/browse/HBASE-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-2765: Assignee: Todd Lipcon You have fixed this, haven't you Todd? I see RIT in master UI now. And we have the hbck tool. Can we close this issue out? Master should display currently unassigned regions in UI Key: HBASE-2765 URL: https://issues.apache.org/jira/browse/HBASE-2765 Project: HBase Issue Type: New Feature Components: master Affects Versions: 0.90.0 Reporter: Todd Lipcon Assignee: Todd Lipcon The master often knows that a region is unassigned, but thinks it's getting assigned. It would be very useful to display the current set of unassigned regions in the UI, as well as their current state (eg where it's supposed to be getting assigned, how long ago it last heard from it, etc). This would be useful in debugging/diagnosing cluster issues (eg I just noticed that a client had hung, took me a while to figure out that I had a region that was repeatedly failing to open with an NPE due to HBASE-2729) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2675) Quick smoke tests testsuite
[
https://issues.apache.org/jira/browse/HBASE-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030021#comment-13030021
]
stack commented on HBASE-2675:
--
Is the following enough?
{code}
mvn clean test -Dtest=TestFromClientSide
{code}
It excercises the UI from the client side -- it starts up a minicluster --
doing about 40 odd little tests.
Quick smoke tests testsuite
-
Key: HBASE-2675
URL: https://issues.apache.org/jira/browse/HBASE-2675
Project: HBase
Issue Type: Test
Reporter: Benoit Sigoure
Priority: Minor
It would be nice if there was a known subset of the tests that run fast (e.g.
not more than a few seconds) and quickly help us check whether the code isn't
horribly broken. This way one could run those tests at a frequent interval
when iterating and only run the entire testsuite at the end, when they think
they're done, since doing so is very time consuming.
Someone would need to identify which tests really focus on the core
functionality and add a target in the build system to just run those tests.
As a bonus, it would be awesome++ if the core tests ran, say, 10x faster than
they currently do. There's a lot of sleep-based synchronization in the
tests and it would be nice to remove some of that where possible to make the
tests run as fast as the machine can handle them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2631) Decide between InMB and MB as suffix for field names in ClusterStatus objects
[ https://issues.apache.org/jira/browse/HBASE-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-2631: - Labels: noob (was: ) Decide between InMB and MB as suffix for field names in ClusterStatus objects - Key: HBASE-2631 URL: https://issues.apache.org/jira/browse/HBASE-2631 Project: HBase Issue Type: Improvement Reporter: Jeff Hammerbacher Labels: noob We vacillate between InMB (e.g. HServerLoad.getMemStoreSizeInMB(), HServerLoad.getStorefileIndexSizeInMB()) and MB (e.g. HServerLoad.getUsedHeapMB()) in the various ClusterStatus objects (HServerInfo, HServerLoad, HServerLoad.RegionLoad). We should probably pick one and stick to it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2646) Compaction requests should be prioritized to prevent blocking
[
https://issues.apache.org/jira/browse/HBASE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-2646.
--
Resolution: Fixed
Fix Version/s: 0.90.0
Assignee: Jeff Whiting
Hadoop Flags: [Reviewed]
This was applied a while back. Resolving. Thanks for the patch Jeff (Assigned
it to you).
Compaction requests should be prioritized to prevent blocking
-
Key: HBASE-2646
URL: https://issues.apache.org/jira/browse/HBASE-2646
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.20.4
Environment: ubuntu server 10; hbase 0.20.4; 4 machine cluster (each
machine is an 8 core xeon with 16 GB of ram and 6TB of storage); ~250 Million
rows;
Reporter: Jeff Whiting
Assignee: Jeff Whiting
Priority: Critical
Labels: compaction, split
Fix For: 0.90.0
Attachments: 2646-fix-race-condition-r1004349.txt, 2646-v2.txt,
2646-v3.txt, PriorityQueue-r996664.patch, prioritycompactionqueue-0.20.4.patch
While testing the write capacity of a 4 machine hbase cluster we were getting
long and frequent client pauses as we attempted to load the data. Looking
into the problem we'd get a relatively large compaction queue and when a
region hit the hbase.hstore.blockingStoreFiles limit it would get block the
client and the compaction request would get put on the back of the queue
waiting for many other less important compactions. The client is basically
stuck at that point until a compaction is done. Prioritizing the compaction
requests and allowing the request that is blocking other actions go first
would help solve the problem.
You can see the problem by looking at our log files:
You'll first see an event such as a too many HLog which will put a lot of
requests on the compaction queue.
{noformat}
2010-05-25 10:53:26,570 INFO org.apache.hadoop.hbase.regionserver.HLog: Too
many hlogs: logs=33, maxlogs=32; forcing flush of 22 regions(s):
responseCounts,RS_6eZzLtdwhGiTwHy,1274232223324,
responses,RS_0qhkL5rUmPCbx3K-1274213057242,1274513189592,
responses,RS_1ANYnTegjzVIsHW-12742177419
21,1274511001873, responses,RS_1HQ4UG5BdOlAyuE-1274216757425,1274726323747,
responses,RS_1Y7SbqSTsZrYe7a-1274328697838,1274478031930,
responses,RS_1ZH5TB5OdW4BVLm-1274216239894,1274538267659,
responses,RS_3BHc4KyoM3q72Yc-1274290546987,1274502062319,
responses,RS_3ra9BaBMAXFAvbK-127421457
9958,1274381552543, responses,RS_6SDrGNuyyLd3oR6-1274219941155,1274385453586,
responses,RS_8AGCEMWbI6mZuoQ-1274306857429,1274319602718,
responses,RS_8C8T9DN47uwTG1S-1274215381765,1274289112817,
responses,RS_8J5wmdmKmJXzK6g-1274299593861,1274494738952,
responses,RS_8e5Sz0HeFPAdb6c-1274288
641459,1274495868557,
responses,RS_8rjcnmBXPKzI896-1274306981684,1274403047940,
responses,RS_9FS3VedcyrF0KX2-1274245971331,1274754745013,
responses,RS_9oZgPtxO31npv3C-1274214027769,1274396489756,
responses,RS_a3FdO2jhqWuy37C-1274209228660,1274399508186,
responses,RS_a3LJVxwTj29MHVa-12742
{noformat}
Then you see the too many log files:
{noformat}
2010-05-25 10:53:31,364 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for region
responses-index,--1274799047787--R_cBKrGxx0FdWjPso,1274804575862/783020138
because: regionserver/192.168.0.81:60020.cacheFlusher
2010-05-25 10:53:32,364 WARN
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Region
responses-index,--1274799047787--R_cBKrGxx0FdWjPso,1274804575862 has too many
store files, putting it back at the end of the flush queue.
{noformat}
Which leads to this:
{noformat}
2010-05-25 10:53:27,061 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Blocking updates for 'IPC Server handler 60 on 60020' on region
responses-index,--1274799047787--R_cBKrGxx0FdWjPso,1274804575862: memstore
size 128.0m is = than blocking 128.0m size
2010-05-25 10:53:27,061 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Blocking updates for 'IPC Server handler 84 on 60020' on region
responses-index,--1274799047787--R_cBKrGxx0FdWjPso,1274804575862: memstore
size 128.0m is = than blocking 128.0m size
2010-05-25 10:53:27,065 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Blocking updates for 'IPC Server handler 1 on 60020' on region
responses-index,--1274799047787--R_cBKrGxx0FdWjPso,1274804575862: memstore
size 128.0m is = than blocking 128.0m size
{noformat}
Once the compaction / split is done a flush is able to happen which unblocks
the IPC allowing writes to continue. Unfortunately this process can take
upwards of 15+ minutes (the specific case shown here from our logs took about
4 minutes).
--
This message is automatically generated by JIRA.
For more information on JIRA,
[jira] [Resolved] (HBASE-2465) HMaster should not contact each RS on startup
[ https://issues.apache.org/jira/browse/HBASE-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-2465. -- Resolution: Won't Fix We don't have a verifyClusterState any more and I don't think we contact each regionserver on startup any more (Reopen if I have this wrong Todd). HMaster should not contact each RS on startup - Key: HBASE-2465 URL: https://issues.apache.org/jira/browse/HBASE-2465 Project: HBase Issue Type: Improvement Components: master Reporter: Todd Lipcon On startup, in verifyClusterState, the master contacts each region server serially. If a region server is down it will retry for several minutes (if the client retry setting is high). During this period, the master cannot be shut down, and also isn't processing real work. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3861) MiniZooKeeperCluster.startup() should refer to hbase.zookeeper.property.maxClientCnxns
[
https://issues.apache.org/jira/browse/HBASE-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030029#comment-13030029
]
Andrew Purtell commented on HBASE-3861:
---
+1, I'll commit with Stack's comment addressed also.
MiniZooKeeperCluster.startup() should refer to
hbase.zookeeper.property.maxClientCnxns
--
Key: HBASE-3861
URL: https://issues.apache.org/jira/browse/HBASE-3861
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.3
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Fix For: 0.90.3
Attachments: HBASE-3861.patch, HBASE-3861.patch
Original Estimate: 1h
Remaining Estimate: 1h
Currently the number of the client connections is hard-wired to 1000:
{{{
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(new
InetSocketAddress(clientPort),1000);
} catch (BindException e) {
}}}
This should be set according to the test environment's hbase configuration.
The property in
question is : hbase.zookeeper.property.maxClientCnxns.
Currently some tests such as org.apache.hadoop.hbase.client.TestHCM fail
because the number of connections used by the HBase client exceeds 1000.
Recently MAX_CACHED_HBASE_INSTANCES increased from 31 to 2000 on 0.90 branch:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javap1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javar1=1096818r2=1096817view=diffpathrev=1096818
and correspondingly the hbase config on the Zookeeper server-side also
increased in hbase-default.xml:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/resources/hbase-default.xml?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlp1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlr1=1091594r2=1091593view=diffpathrev=1091594
So if MiniZKCluster looks at this setting, the test won't have this failure.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3861) MiniZooKeeperCluster.startup() should refer to hbase.zookeeper.property.maxClientCnxns
[
https://issues.apache.org/jira/browse/HBASE-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030036#comment-13030036
]
Hbase Build Acct commented on HBASE-3861:
-
Sounds good
MiniZooKeeperCluster.startup() should refer to
hbase.zookeeper.property.maxClientCnxns
--
Key: HBASE-3861
URL: https://issues.apache.org/jira/browse/HBASE-3861
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.3
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Fix For: 0.90.3
Attachments: HBASE-3861.patch, HBASE-3861.patch
Original Estimate: 1h
Remaining Estimate: 1h
Currently the number of the client connections is hard-wired to 1000:
{{{
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(new
InetSocketAddress(clientPort),1000);
} catch (BindException e) {
}}}
This should be set according to the test environment's hbase configuration.
The property in
question is : hbase.zookeeper.property.maxClientCnxns.
Currently some tests such as org.apache.hadoop.hbase.client.TestHCM fail
because the number of connections used by the HBase client exceeds 1000.
Recently MAX_CACHED_HBASE_INSTANCES increased from 31 to 2000 on 0.90 branch:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javap1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javar1=1096818r2=1096817view=diffpathrev=1096818
and correspondingly the hbase config on the Zookeeper server-side also
increased in hbase-default.xml:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/resources/hbase-default.xml?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlp1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlr1=1091594r2=1091593view=diffpathrev=1091594
So if MiniZKCluster looks at this setting, the test won't have this failure.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2297) Make it so can send email and have it show as comments in JIRA
[ https://issues.apache.org/jira/browse/HBASE-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-2297. -- Resolution: Fixed Assignee: stack Resolving. This works now (Though it shows as being from Hbase Build Acct --- can open new issue for that). Make it so can send email and have it show as comments in JIRA -- Key: HBASE-2297 URL: https://issues.apache.org/jira/browse/HBASE-2297 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Some discussion up in mailing list suggests it may be just a matter of removing the 'Reply-To' which has the list in it. I filed INFRA-2533. I also tested replying to an issue emission and changing the to to be j...@apache.org instead and my comment went into the issue as a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2240) Balancer should not reassign (because of overloading) a region that was just opened
[ https://issues.apache.org/jira/browse/HBASE-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-2240. -- Resolution: Invalid Resolving as invalid (Jon argues this in body of this issue and stuff suggested in the issue -- upping balancer period -- has been implemented) Balancer should not reassign (because of overloading) a region that was just opened --- Key: HBASE-2240 URL: https://issues.apache.org/jira/browse/HBASE-2240 Project: HBase Issue Type: Bug Components: documentation Reporter: stack Priority: Minor I'm running a mapreduce job. I see a region split and its daughters come on line and then 8 seconds later, master judges the regionserver overloaded and so closes the just-opened region. This messes up clients. They may have just picked up the new location for their row query and now its moved again and client has to go hunting anew. We need to assign the vintage regions first. I'm going to change balancer slop. Its not sloppy enough and balancing cuts in too early. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2963) based on hadoop0.21.0
[ https://issues.apache.org/jira/browse/HBASE-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-2963. -- Resolution: Duplicate Duplicate of hbase-2233 based on hadoop0.21.0 - Key: HBASE-2963 URL: https://issues.apache.org/jira/browse/HBASE-2963 Project: HBase Issue Type: Improvement Affects Versions: 0.20.6 Reporter: chenjiajun Priority: Blocker Fix For: 0.92.0 I upgrade my hadoop from 0.20.2 to 0.21.0，but hbase is does't work! -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2765) Master should display currently unassigned regions in UI
[ https://issues.apache.org/jira/browse/HBASE-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HBASE-2765. Resolution: Duplicate oh hey, yea, I forgot I had filed this! Yes, fixed for 0.92 Master should display currently unassigned regions in UI Key: HBASE-2765 URL: https://issues.apache.org/jira/browse/HBASE-2765 Project: HBase Issue Type: New Feature Components: master Affects Versions: 0.90.0 Reporter: Todd Lipcon Assignee: Todd Lipcon The master often knows that a region is unassigned, but thinks it's getting assigned. It would be very useful to display the current set of unassigned regions in the UI, as well as their current state (eg where it's supposed to be getting assigned, how long ago it last heard from it, etc). This would be useful in debugging/diagnosing cluster issues (eg I just noticed that a client had hung, took me a while to figure out that I had a region that was repeatedly failing to open with an NPE due to HBASE-2729) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2165) Improve fragmentation display and implementation
[ https://issues.apache.org/jira/browse/HBASE-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-2165. -- Resolution: Won't Fix We don't have this attribute any more. Resolving issue till it comes back. Improve fragmentation display and implementation Key: HBASE-2165 URL: https://issues.apache.org/jira/browse/HBASE-2165 Project: HBase Issue Type: Improvement Affects Versions: 0.20.4, 0.90.0 Reporter: Lars George Priority: Minor Fix For: 0.92.0 Improve by - moving the blocking FS scan into a thread so that the UI loads fast and initially displays n/a but once it has completed the scan it displays the proper numbers - explaining what fragmentation means to the user (better hints or help text in UI) - Switch -ROOT- (and maybe even .META.?) to simply say Yes or a tick that it is fragmented as it only has 0% or 100% available (since it has only a single region) - also computing the space occupied by each table and the total and - if easily done - add a graph to display it (Google Pie Chart would be nice but is an external link) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2010) Ant task to create the test jar is inconsistent with hadoop
[ https://issues.apache.org/jira/browse/HBASE-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-2010. -- Resolution: Invalid Marking as invalid since move to maven (We are no longer like hadoop -- it uses ivy) Ant task to create the test jar is inconsistent with hadoop --- Key: HBASE-2010 URL: https://issues.apache.org/jira/browse/HBASE-2010 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.90.0 Reporter: Cosmin Lehene Priority: Trivial Fix For: 0.92.0 Attachments: HBASE-2010.patch Original Estimate: 1h Remaining Estimate: 1h In hadoop projects the jar-test task is used to compile the test jars. In hbase this task is called compile-core-test. Shouldn't we have a jar-test task that depends on the compile-core-test? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3861) MiniZooKeeperCluster.startup() should refer to hbase.zookeeper.property.maxClientCnxns
[
https://issues.apache.org/jira/browse/HBASE-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-3861:
--
Resolution: Fixed
Fix Version/s: 0.92.0
Release Note: (was: added new patch with numClientCnxns set to 5000
(enough for TestHCM to pass).)
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to trunk and 0.90 branch. Tests pass locally.
MiniZooKeeperCluster.startup() should refer to
hbase.zookeeper.property.maxClientCnxns
--
Key: HBASE-3861
URL: https://issues.apache.org/jira/browse/HBASE-3861
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.3
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Fix For: 0.90.3, 0.92.0
Attachments: HBASE-3861.patch, HBASE-3861.patch
Original Estimate: 1h
Remaining Estimate: 1h
Currently the number of the client connections is hard-wired to 1000:
{{{
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(new
InetSocketAddress(clientPort),1000);
} catch (BindException e) {
}}}
This should be set according to the test environment's hbase configuration.
The property in
question is : hbase.zookeeper.property.maxClientCnxns.
Currently some tests such as org.apache.hadoop.hbase.client.TestHCM fail
because the number of connections used by the HBase client exceeds 1000.
Recently MAX_CACHED_HBASE_INSTANCES increased from 31 to 2000 on 0.90 branch:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javap1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javar1=1096818r2=1096817view=diffpathrev=1096818
and correspondingly the hbase config on the Zookeeper server-side also
increased in hbase-default.xml:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/resources/hbase-default.xml?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlp1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlr1=1091594r2=1091593view=diffpathrev=1091594
So if MiniZKCluster looks at this setting, the test won't have this failure.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-3864) Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
---
Key: HBASE-3864
URL: https://issues.apache.org/jira/browse/HBASE-3864
Project: HBase
Issue Type: Bug
Components: mapreduce
Reporter: Aaron T. Myers
HBASE-2899 renamed {{hfile.min.blocksize.size}} to
{{hbase.mapreduce.hfileoutputformat.blocksize}}. However, HBASE-1861 (committed
after HBASE-2899) reverted this change.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3864) Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
[
https://issues.apache.org/jira/browse/HBASE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HBASE-3864:
--
Attachment: hbase-3864.0.patch
Patch which changes the config back from {{hfile.min.blocksize.size}} to
{{hbase.mapreduce.hfileoutputformat.blocksize}}.
Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
---
Key: HBASE-3864
URL: https://issues.apache.org/jira/browse/HBASE-3864
Project: HBase
Issue Type: Bug
Components: mapreduce
Reporter: Aaron T. Myers
Attachments: hbase-3864.0.patch
HBASE-2899 renamed {{hfile.min.blocksize.size}} to
{{hbase.mapreduce.hfileoutputformat.blocksize}}. However, HBASE-1861
(committed after HBASE-2899) reverted this change.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3864) Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
[
https://issues.apache.org/jira/browse/HBASE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HBASE-3864:
--
Status: Patch Available (was: Open)
Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
---
Key: HBASE-3864
URL: https://issues.apache.org/jira/browse/HBASE-3864
Project: HBase
Issue Type: Bug
Components: mapreduce
Reporter: Aaron T. Myers
Attachments: hbase-3864.0.patch
HBASE-2899 renamed {{hfile.min.blocksize.size}} to
{{hbase.mapreduce.hfileoutputformat.blocksize}}. However, HBASE-1861
(committed after HBASE-2899) reverted this change.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030114#comment-13030114 ] stack commented on HBASE-3744: -- I applied the addendum w/ some massaging, no refactoring. Thanks for the patch Ted. Lets see if it cures the failing TestAdmin in 0.90. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.90.3 Attachments: 3744-addendum-for-TestAdmin.patch, 3744-addendum.txt, 3744-v2.txt, 3744-v3.txt, 3744.txt, create_big_tables.rb, create_big_tables.rb, create_big_tables.rb In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3865) Failing TestWALReplay
[ https://issues.apache.org/jira/browse/HBASE-3865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3865: - Attachment: 3865.txt Failing TestWALReplay - Key: HBASE-3865 URL: https://issues.apache.org/jira/browse/HBASE-3865 Project: HBase Issue Type: Bug Reporter: stack Attachments: 3865.txt Failed last few times up on jenkins. The change to the signature of the internalFlushCache to add passing of a MonitorVisitor meant the override here was no longer called. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-3865) Failing TestWALReplay
Failing TestWALReplay - Key: HBASE-3865 URL: https://issues.apache.org/jira/browse/HBASE-3865 Project: HBase Issue Type: Bug Reporter: stack Attachments: 3865.txt Failed last few times up on jenkins. The change to the signature of the internalFlushCache to add passing of a MonitorVisitor meant the override here was no longer called. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3865) Failing TestWALReplay
[ https://issues.apache.org/jira/browse/HBASE-3865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-3865. -- Resolution: Fixed Fix Version/s: 0.92.0 Assignee: stack Applied to TRUNK Failing TestWALReplay - Key: HBASE-3865 URL: https://issues.apache.org/jira/browse/HBASE-3865 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.0 Attachments: 3865.txt Failed last few times up on jenkins. The change to the signature of the internalFlushCache to add passing of a MonitorVisitor meant the override here was no longer called. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3861) MiniZooKeeperCluster.startup() should refer to hbase.zookeeper.property.maxClientCnxns
[
https://issues.apache.org/jira/browse/HBASE-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030154#comment-13030154
]
stack commented on HBASE-3861:
--
I just backed this out of TRUNK because its OOME'ing TestHCM up on hudson and
locally on two different platforms (I can't see why it would do it though...
must be some interaction w/ hbase-3777). It runs fine for me on branch.
MiniZooKeeperCluster.startup() should refer to
hbase.zookeeper.property.maxClientCnxns
--
Key: HBASE-3861
URL: https://issues.apache.org/jira/browse/HBASE-3861
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.3
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Fix For: 0.90.3, 0.92.0
Attachments: HBASE-3861.patch, HBASE-3861.patch
Original Estimate: 1h
Remaining Estimate: 1h
Currently the number of the client connections is hard-wired to 1000:
{{{
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(new
InetSocketAddress(clientPort),1000);
} catch (BindException e) {
}}}
This should be set according to the test environment's hbase configuration.
The property in
question is : hbase.zookeeper.property.maxClientCnxns.
Currently some tests such as org.apache.hadoop.hbase.client.TestHCM fail
because the number of connections used by the HBase client exceeds 1000.
Recently MAX_CACHED_HBASE_INSTANCES increased from 31 to 2000 on 0.90 branch:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javap1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javar1=1096818r2=1096817view=diffpathrev=1096818
and correspondingly the hbase config on the Zookeeper server-side also
increased in hbase-default.xml:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/resources/hbase-default.xml?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlp1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlr1=1091594r2=1091593view=diffpathrev=1091594
So if MiniZKCluster looks at this setting, the test won't have this failure.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-3861) MiniZooKeeperCluster.startup() should refer to hbase.zookeeper.property.maxClientCnxns
[
https://issues.apache.org/jira/browse/HBASE-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-3861:
---
Caused TestHCM to OOME on trunk up on Jenkins. Backed out for now. Reopening to
investigate / fix.
MiniZooKeeperCluster.startup() should refer to
hbase.zookeeper.property.maxClientCnxns
--
Key: HBASE-3861
URL: https://issues.apache.org/jira/browse/HBASE-3861
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.3
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Fix For: 0.90.3, 0.92.0
Attachments: HBASE-3861.patch, HBASE-3861.patch
Original Estimate: 1h
Remaining Estimate: 1h
Currently the number of the client connections is hard-wired to 1000:
{{{
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(new
InetSocketAddress(clientPort),1000);
} catch (BindException e) {
}}}
This should be set according to the test environment's hbase configuration.
The property in
question is : hbase.zookeeper.property.maxClientCnxns.
Currently some tests such as org.apache.hadoop.hbase.client.TestHCM fail
because the number of connections used by the HBase client exceeds 1000.
Recently MAX_CACHED_HBASE_INSTANCES increased from 31 to 2000 on 0.90 branch:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javap1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javar1=1096818r2=1096817view=diffpathrev=1096818
and correspondingly the hbase config on the Zookeeper server-side also
increased in hbase-default.xml:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/resources/hbase-default.xml?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlp1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlr1=1091594r2=1091593view=diffpathrev=1091594
So if MiniZKCluster looks at this setting, the test won't have this failure.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-1242) HBase Design Overview
[ https://issues.apache.org/jira/browse/HBASE-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-1242. -- Resolution: Later Marking as later. No work done on this issue in a while. HBase Design Overview - Key: HBASE-1242 URL: https://issues.apache.org/jira/browse/HBASE-1242 Project: HBase Issue Type: Task Components: documentation Reporter: Evgeny Ryabitskiy Assignee: Evgeny Ryabitskiy Priority: Minor Attachments: HBase_Arc6.jpeg I'm going to make wiki page contain HBase Design overview. That will cover - architecture of HBase, - used solutions - unresolved problems It should be something like http://labs.google.com/papers/bigtable.html focused on Implementation part. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3732) New configuration option for client-side compression
[
https://issues.apache.org/jira/browse/HBASE-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030167#comment-13030167
]
Karthick Sankarachary commented on HBASE-3732:
--
Oh, I see. I like the idea of keeping the value in a compressed form until the
client tries to get it. Perhaps we can compress the value depending on
whether it's fatter than a certain threshold? Also, given that the value
typically accounts for most of the {{KeyValue}}'s size, do we need to call
{{HFile#getCompressingStream}} if the value is already compressed up front?
New configuration option for client-side compression
Key: HBASE-3732
URL: https://issues.apache.org/jira/browse/HBASE-3732
Project: HBase
Issue Type: New Feature
Reporter: Jean-Daniel Cryans
Fix For: 0.92.0
Attachments: compressed_streams.jar
We have a case here where we have to store very fat cells (arrays of
integers) which can amount into the hundreds of KBs that we need to read
often, concurrently, and possibly keep in cache. Compressing the values on
the client using java.util.zip's Deflater before sending them to HBase proved
to be in our case almost an order of magnitude faster.
There reasons are evident: less data sent to hbase, memstore contains
compressed data, block cache contains compressed data too, etc.
I was thinking that it might be something useful to add to a family schema,
so that Put/Result do the conversion for you. The actual compression algo
should also be configurable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3864) Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
[
https://issues.apache.org/jira/browse/HBASE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reassigned HBASE-3864:
Assignee: Aaron T. Myers
Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
---
Key: HBASE-3864
URL: https://issues.apache.org/jira/browse/HBASE-3864
Project: HBase
Issue Type: Bug
Components: mapreduce
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Attachments: hbase-3864.0.patch
HBASE-2899 renamed {{hfile.min.blocksize.size}} to
{{hbase.mapreduce.hfileoutputformat.blocksize}}. However, HBASE-1861
(committed after HBASE-2899) reverted this change.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3864) Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
[
https://issues.apache.org/jira/browse/HBASE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-3864:
-
Resolution: Fixed
Fix Version/s: 0.92.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to TRUNK. Thanks for the patch Aaron.
Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
---
Key: HBASE-3864
URL: https://issues.apache.org/jira/browse/HBASE-3864
Project: HBase
Issue Type: Bug
Components: mapreduce
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Fix For: 0.92.0
Attachments: hbase-3864.0.patch
HBASE-2899 renamed {{hfile.min.blocksize.size}} to
{{hbase.mapreduce.hfileoutputformat.blocksize}}. However, HBASE-1861
(committed after HBASE-2899) reverted this change.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-3866) Script to add regions gradually to a new regionserver.
Script to add regions gradually to a new regionserver. -- Key: HBASE-3866 URL: https://issues.apache.org/jira/browse/HBASE-3866 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.90.2 Reporter: Aravind Gottipati Priority: Minor When a new region server is brought online, the current balancer kicks off a whole bunch of region moves and causes a lot of regions to be un-available right away. A slower balancer that gradually balances the cluster is probably a good script to have. I have an initial version that mooches off the region_mover script to do this. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3866) Script to add regions gradually to a new regionserver.
[ https://issues.apache.org/jira/browse/HBASE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravind Gottipati updated HBASE-3866: - Attachment: slow_balancer.rb This script uses a lot of the code from region_mover.rb. The script should be invoked like this. HBASE_NOEXEC=true $HBASE_HOME/bin/hbase org.jruby.Main $HBASE_HOME/bin/slow_balancer.rb --debug -l 2 The -l option is the target difference between the server with the maximum regions and the server with the minimum regions. Once the delta reaches this point, the script exits. If -l is not passed, it defaults to the number of region servers in your environment. Script to add regions gradually to a new regionserver. -- Key: HBASE-3866 URL: https://issues.apache.org/jira/browse/HBASE-3866 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.90.2 Reporter: Aravind Gottipati Priority: Minor Attachments: slow_balancer.rb When a new region server is brought online, the current balancer kicks off a whole bunch of region moves and causes a lot of regions to be un-available right away. A slower balancer that gradually balances the cluster is probably a good script to have. I have an initial version that mooches off the region_mover script to do this. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3836) Add facility to track currently progressing actions/workflows
[ https://issues.apache.org/jira/browse/HBASE-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030199#comment-13030199 ] Hudson commented on HBASE-3836: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Add facility to track currently progressing actions/workflows - Key: HBASE-3836 URL: https://issues.apache.org/jira/browse/HBASE-3836 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.92.0 Attachments: hbase-3836.txt A lot of troubleshooting involves answering the question well, what is your server doing right now? Today, that involves some combination of interpreting jstack output and/or trudging through logs. Problems with these methods are: (a) users may not have direct ssh access to regionserver machines in production environments, (b) logs are very verbose, so hard to separate what's still going on vs stuff that might have completed, and (c) interpreting jstack requires a pretty good knowledge of the codebase plus diving into source code. I'd like to add a singleton (for now) which takes care of tracking any major actions going on in the region server and master. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3838) RegionCoprocesorHost.preWALRestore throws npe in case there is no RegionObserver registered.
[ https://issues.apache.org/jira/browse/HBASE-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030202#comment-13030202 ] Hudson commented on HBASE-3838: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) RegionCoprocesorHost.preWALRestore throws npe in case there is no RegionObserver registered. Key: HBASE-3838 URL: https://issues.apache.org/jira/browse/HBASE-3838 Project: HBase Issue Type: Bug Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Fix For: 0.92.0 Attachments: patch.txt It seems the check to bypass the Observers chain is at wrong place in case of pre/post WALRestore. It should be inside the if statement that checks whether the CP is instance of RegionObserver or not. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3866) Script to add regions gradually to a new regionserver.
[ https://issues.apache.org/jira/browse/HBASE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravind Gottipati updated HBASE-3866: - Attachment: slow_balancer.rb Script to add regions gradually to a new regionserver. -- Key: HBASE-3866 URL: https://issues.apache.org/jira/browse/HBASE-3866 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.90.2 Reporter: Aravind Gottipati Priority: Minor Attachments: slow_balancer.rb, slow_balancer.rb When a new region server is brought online, the current balancer kicks off a whole bunch of region moves and causes a lot of regions to be un-available right away. A slower balancer that gradually balances the cluster is probably a good script to have. I have an initial version that mooches off the region_mover script to do this. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3837) Expose regionsInTransition on master UI
[ https://issues.apache.org/jira/browse/HBASE-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030201#comment-13030201 ] Hudson commented on HBASE-3837: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Expose regionsInTransition on master UI --- Key: HBASE-3837 URL: https://issues.apache.org/jira/browse/HBASE-3837 Project: HBase Issue Type: New Feature Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.92.0 Attachments: hbase-3837.txt There have been some bugs in the past that can cause a region to get stuck in transition. It's currently hard to see this without tailing the logs and noticing periodic timeout messages, etc. I'd like to expose the regionsInTransition map on the master UI, so ops can quickly identify what might be causing a region to get stuck. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3855) Performance degradation of memstore because reseek is linear
[ https://issues.apache.org/jira/browse/HBASE-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030200#comment-13030200 ] Hudson commented on HBASE-3855: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Performance degradation of memstore because reseek is linear Key: HBASE-3855 URL: https://issues.apache.org/jira/browse/HBASE-3855 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Priority: Blocker Fix For: 0.92.0 Attachments: memstoreReseek.txt, memstoreReseek2.txt The scanner use reseek to find the next row (or next column) as part of a scan. The reseek code iterates over a Set to position itself at the right place. If there are many thousands of kvs that need to be skipped over, then the time-cost is very high. In this case, a seek would be far lesser in cost than a reseek. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3858) performance.xml / troubleshooting.xml - porting rest of PerformanceTuning wiki page
[ https://issues.apache.org/jira/browse/HBASE-3858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030204#comment-13030204 ] Hudson commented on HBASE-3858: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) performance.xml / troubleshooting.xml - porting rest of PerformanceTuning wiki page --- Key: HBASE-3858 URL: https://issues.apache.org/jira/browse/HBASE-3858 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: performance_HBASE_3858.xml.patch, troubleshooting_HBASE_3858.xml.patch The rest of the HBase PerformanceTuning wiki page has been ported to the HBase book. The top-part of the page was ported into performance.xml One thing I want to call out specifically is that the PerformanceTuning wiki suggested to turn off WAL on puts for performance. I added a section in the Client sub-section on this as an option, however, I added several cautions about the potential for data loss and that for high-volume loading bulk-imports should be considered instead. The bottom part (GC logging) was ported into troubleshooting.xml After this has been applied, the PerformanceTuning wiki can be de-commissioned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3813) Change RPC callQueue size from handlerCount * MAX_QUEUE_SIZE_PER_HANDLER;
[
https://issues.apache.org/jira/browse/HBASE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030206#comment-13030206
]
Hudson commented on HBASE-3813:
---
Integrated in HBase-TRUNK #1909 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
Change RPC callQueue size from handlerCount * MAX_QUEUE_SIZE_PER_HANDLER;
---
Key: HBASE-3813
URL: https://issues.apache.org/jira/browse/HBASE-3813
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: stack
Priority: Critical
Attachments: 3813.txt
Yesterday debugging w/ Jack we noticed that with few handlers on a big box,
he was seeing stats like this:
{code}
2011-04-21 11:54:49,451 DEBUG org.apache.hadoop.ipc.HBaseServer: Server
connection from X.X.X.X:60931; # active connections: 11; # queued calls: 2500
{code}
We had 2500 items in the rpc queue waiting to be processed.
Turns out he had too few handlers for number of clients (but also, it seems
like he figured hw issues in that his RAM bus was running at 1/4 the rate
that it should have been running at).
Chatting w/ J-D this morning, he asked if the queues hold 'data'. The queues
hold 'Calls'. Calls are the client request. They contain data.
Jack had 2500 items queued. If each item to insert was 1MB, thats 25k * 1MB
of memory that is outside of our generally accounting.
Currently the queue size is handlers * MAX_QUEUE_SIZE_PER_HANDLER where
MAX_QUEUE_SIZE_PER_HANDLER is hardcoded to be 100.
If the queue is full we block (LinkedBlockingQueue).
Going to change the queue size from 100 to 10 by default -- but also will
make it configurable and will doc. this as possible cause of OOME. Will try
it on production here before committing patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3835) Switch web pages to Jamon template engine instead of JSP
[ https://issues.apache.org/jira/browse/HBASE-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030205#comment-13030205 ] Hudson commented on HBASE-3835: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Switch web pages to Jamon template engine instead of JSP Key: HBASE-3835 URL: https://issues.apache.org/jira/browse/HBASE-3835 Project: HBase Issue Type: Improvement Components: master, regionserver Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.92.0 Attachments: hbase-3835.txt, hbase-3835.txt, hbase-3835.txt Jamon (http://jamon.org) is a template engine that I think is preferable to JSP. You can read an interview with some comparisons vs JSP here: http://www.artima.com/lejava/articles/jamon.html In particular, I think it will give us the following advantages: - Since we'll have a servlet in front of each template, it will encourage us to write less inline Java code and do more code in the servlets. - Makes proper unit testing easier since you can trivially render a template and pass in mock arguments without having to start a whole HTTP stack - Static typing of template arguments makes it easier to know at compile-time if you've made a mistake. Thoughts? I converted the Master UI yesterday and only took a couple hours. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3695) Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
[ https://issues.apache.org/jira/browse/HBASE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030208#comment-13030208 ] Hudson commented on HBASE-3695: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting Key: HBASE-3695 URL: https://issues.apache.org/jira/browse/HBASE-3695 Project: HBase Issue Type: Improvement Components: util Affects Versions: 0.90.1 Reporter: Marc Limotte Assignee: Marc Limotte Fix For: 0.90.3 The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback. * Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change. * More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false. * In the case of overlaps and holes, output both ends of the broken chain. * Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3862) Race conditions in aggregate calculation
[ https://issues.apache.org/jira/browse/HBASE-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030211#comment-13030211 ] Hudson commented on HBASE-3862: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Race conditions in aggregate calculation Key: HBASE-3862 URL: https://issues.apache.org/jira/browse/HBASE-3862 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.92.0 Reporter: John Heitmann Attachments: coprocessor_race.patch The AggregationClient requests aggregations from multiple region servers in parallel. The calculations in the reducer callbacks of the AggregationClient are not thread safe, and therefore could return an incorrect result due to simultaneous/interleaved execution. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3839) Expose in-progress tasks on web UIs
[ https://issues.apache.org/jira/browse/HBASE-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030203#comment-13030203 ] Hudson commented on HBASE-3839: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Expose in-progress tasks on web UIs --- Key: HBASE-3839 URL: https://issues.apache.org/jira/browse/HBASE-3839 Project: HBase Issue Type: New Feature Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.92.0 Attachments: hbase-3839.txt, tasks.png HBASE-3836 adds a TaskMonitor class which collects info about what's going on inside processes. This ticket is to expose the task monitor info on the web UIs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3853) TestInfoServers failing after HBASE-3835
[ https://issues.apache.org/jira/browse/HBASE-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030209#comment-13030209 ] Hudson commented on HBASE-3853: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) TestInfoServers failing after HBASE-3835 Key: HBASE-3853 URL: https://issues.apache.org/jira/browse/HBASE-3853 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.92.0 Attachments: hbase-3853.txt Forgot to update this test - the test was checking the redirections from index.html to the JSP pages, but since the JSP pages no longer exist, the test case needs to be fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3670) Fix error handling in get(ListGet gets)
[ https://issues.apache.org/jira/browse/HBASE-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030207#comment-13030207 ] Hudson commented on HBASE-3670: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Fix error handling in get(ListGet gets) - Key: HBASE-3670 URL: https://issues.apache.org/jira/browse/HBASE-3670 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.92.0 Reporter: Lars George Assignee: Harsh J Chouraria Fix For: 0.92.0 Attachments: HBASE-3670.r1.diff See HBASE-3634 for details. The get(ListGet gets) call needs to catch (or rather use a try/finally) the exception thrown by batch() and copy the Result instances over and return it. If that is not intended then we need to fix the JavaDoc in HTableInterface to reflect the new behavior. In general it seems to make sense to check the various methods (list based put, get, delete compared to batch) and agree on the correct behavior. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3861) MiniZooKeeperCluster.startup() should refer to hbase.zookeeper.property.maxClientCnxns
[
https://issues.apache.org/jira/browse/HBASE-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030212#comment-13030212
]
Hudson commented on HBASE-3861:
---
Integrated in HBase-TRUNK #1909 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
HBASE-3861 MiniZooKeeperCluster.startup() should refer to
hbase.zookeeper.property.maxClientCnxns; backing it out till figure the TestHCM
OOME issue
MiniZooKeeperCluster.startup() should refer to
hbase.zookeeper.property.maxClientCnxns
--
Key: HBASE-3861
URL: https://issues.apache.org/jira/browse/HBASE-3861
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.3
Reporter: Eugene Koontz
Assignee: Eugene Koontz
Fix For: 0.90.3, 0.92.0
Attachments: HBASE-3861.patch, HBASE-3861.patch
Original Estimate: 1h
Remaining Estimate: 1h
Currently the number of the client connections is hard-wired to 1000:
{{{
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(new
InetSocketAddress(clientPort),1000);
} catch (BindException e) {
}}}
This should be set according to the test environment's hbase configuration.
The property in
question is : hbase.zookeeper.property.maxClientCnxns.
Currently some tests such as org.apache.hadoop.hbase.client.TestHCM fail
because the number of connections used by the HBase client exceeds 1000.
Recently MAX_CACHED_HBASE_INSTANCES increased from 31 to 2000 on 0.90 branch:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javap1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhbase%2Fclient%2FHConnectionManager.javar1=1096818r2=1096817view=diffpathrev=1096818
and correspondingly the hbase config on the Zookeeper server-side also
increased in hbase-default.xml:
http://svn.apache.org/viewvc/hbase/branches/0.90/src/main/resources/hbase-default.xml?p2=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlp1=%2Fhbase%2Fbranches%2F0.90%2Fsrc%2Fmain%2Fresources%2Fhbase-default.xmlr1=1091594r2=1091593view=diffpathrev=1091594
So if MiniZKCluster looks at this setting, the test won't have this failure.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3860) HLog shouldn't create a new HBC when rolling
[
https://issues.apache.org/jira/browse/HBASE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030210#comment-13030210
]
Hudson commented on HBASE-3860:
---
Integrated in HBase-TRUNK #1909 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
HLog shouldn't create a new HBC when rolling
Key: HBASE-3860
URL: https://issues.apache.org/jira/browse/HBASE-3860
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.2
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.90.3
HBASE-2059 added this change in HLog.rollWriter:
{code}
this.writer = createWriter(fs, newPath, new HBaseConfiguration(conf));
{code}
Which has since become:
{code}
HLog.Writer nextWriter = this.createWriterInstance(fs, newPath,
HBaseConfiguration.create(conf));
{code}
It's unclear to me why it needs to do that, but it bite us today because we
swapped jars under a running hbase with:
{quote}
2011-05-05 12:06:12,876 FATAL org.apache.hadoop.conf.Configuration: error
parsing conf file: java.util.zip.ZipException: invalid stored block lengths
2011-05-05 12:06:12,877 ERROR org.apache.hadoop.hbase.regionserver.LogRoller:
Log rolling failed
java.lang.RuntimeException: java.util.zip.ZipException: invalid stored block
lengths
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1352)
at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1227)
at
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1156)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:427)
at
org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfiguration.java:63)
at
org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfiguration.java:89)
at
org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:100)
at
org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:110)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:485)
at
org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:94)
Caused by: java.util.zip.ZipException: invalid stored block lengths
at
java.util.zip.InflaterInputStream.read(InflaterInputStream.java:147)
at
java.util.zip.InflaterInputStream.read(InflaterInputStream.java:105)
at java.io.FilterInputStream.read(FilterInputStream.java:66)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2932)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:704)
at
com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
at
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1266)
... 9 more
{quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3863) Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
[
https://issues.apache.org/jira/browse/HBASE-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030214#comment-13030214
]
Hudson commented on HBASE-3863:
---
Integrated in HBase-TRUNK #1909 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
-
Key: HBASE-3863
URL: https://issues.apache.org/jira/browse/HBASE-3863
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.92.0
Attachments: 3863.txt
Failing with:
{code}
Error Message
org.apache.hadoop.hbase.HServerAddress cannot be cast to
org.apache.hadoop.hbase.ServerName
Stacktrace
java.lang.ClassCastException: org.apache.hadoop.hbase.HServerAddress cannot
be cast to org.apache.hadoop.hbase.ServerName
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixDupeAssignment(HBaseFsckRepair.java:59)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsckMeta(TestHBaseFsck.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3864) Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
[
https://issues.apache.org/jira/browse/HBASE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030213#comment-13030213
]
Hudson commented on HBASE-3864:
---
Integrated in HBase-TRUNK #1909 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
HBASE-3864 Rename of hfile.min.blocksize.size in HBASE-2899 reverted in
HBASE-1861
Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861
---
Key: HBASE-3864
URL: https://issues.apache.org/jira/browse/HBASE-3864
Project: HBase
Issue Type: Bug
Components: mapreduce
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Fix For: 0.92.0
Attachments: hbase-3864.0.patch
HBASE-2899 renamed {{hfile.min.blocksize.size}} to
{{hbase.mapreduce.hfileoutputformat.blocksize}}. However, HBASE-1861
(committed after HBASE-2899) reverted this change.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3865) Failing TestWALReplay
[ https://issues.apache.org/jira/browse/HBASE-3865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030216#comment-13030216 ] Hudson commented on HBASE-3865: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) HBASE-3865 Failing TestWALReplay Failing TestWALReplay - Key: HBASE-3865 URL: https://issues.apache.org/jira/browse/HBASE-3865 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.0 Attachments: 3865.txt Failed last few times up on jenkins. The change to the signature of the internalFlushCache to add passing of a MonitorVisitor meant the override here was no longer called. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3597) ageOfLastAppliedOp should update after cluster replication failures
[ https://issues.apache.org/jira/browse/HBASE-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030217#comment-13030217 ] Hudson commented on HBASE-3597: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) ageOfLastAppliedOp should update after cluster replication failures --- Key: HBASE-3597 URL: https://issues.apache.org/jira/browse/HBASE-3597 Project: HBase Issue Type: Bug Components: replication Affects Versions: 0.90.1 Reporter: Otis Gospodnetic Assignee: Jean-Daniel Cryans Fix For: 0.90.3 Attachments: HBASE-3597.patch The value of ageOfLastAppliedOp in JMX doesn't update after replication starts failing, and it should. See: http://search-hadoop.com/m/jFPgF1HfnLc -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3846) Set RIT timeout higher
[ https://issues.apache.org/jira/browse/HBASE-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030218#comment-13030218 ] Hudson commented on HBASE-3846: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Set RIT timeout higher -- Key: HBASE-3846 URL: https://issues.apache.org/jira/browse/HBASE-3846 Project: HBase Issue Type: Task Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Priority: Critical Fix For: 0.90.3 Attachments: HBASE-3846.patch As I was talking in HBASE-3669, it is really easy with the current RIT timeout to end up in situations where regions are doubly assigned, not assigned at all or assigned but the master doesn't know about it. As a bandaid, we should set hbase.master.assignment.timeoutmonitor.timeout to what the ZK session timeout is. We had to do that to one of our clusters to be able to start it, else the master kept racing with itself. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3825) performance.xml - adding a few common configuration changes in the 'config' sub-section
[ https://issues.apache.org/jira/browse/HBASE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030220#comment-13030220 ] Hudson commented on HBASE-3825: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) performance.xml - adding a few common configuration changes in the 'config' sub-section --- Key: HBASE-3825 URL: https://issues.apache.org/jira/browse/HBASE-3825 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Fix For: 0.92.0 Attachments: performance_HBASE_3825.xml.patch There are a few common configurations that people make to adjust the RegionServer memory behavior (% of block cache, upperLimit, etc.) Adding a few entries under the 'configurations' sub-section in performance to reference the configuration section for each item. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3844) Book.xml (removing link to defunct wiki) and Performance.xml (adding client tip)
[ https://issues.apache.org/jira/browse/HBASE-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030221#comment-13030221 ] Hudson commented on HBASE-3844: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Book.xml (removing link to defunct wiki) and Performance.xml (adding client tip) Key: HBASE-3844 URL: https://issues.apache.org/jira/browse/HBASE-3844 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Fix For: 0.92.0 Attachments: book_HBASE_3844.xml.patch, performance_HBASE_3844.xml.patch Book.xml - in the FAQ it had a link to a Frequently Seen Errors wiki page. This page is labeled as defunct and doesn't even have anything useful on it anyway. Removed the link to that page. Performance.xml - added tip in Performance under client about attribute selection. This is one of those obvious but not so obvious topics, if you only need 3 attributes don't select the entire column family. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3843) splitLogWorker starts too early
[ https://issues.apache.org/jira/browse/HBASE-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030219#comment-13030219 ] Hudson commented on HBASE-3843: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) splitLogWorker starts too early --- Key: HBASE-3843 URL: https://issues.apache.org/jira/browse/HBASE-3843 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani Fix For: 0.92.0 Attachments: 0001-HBASE-3843-start-splitLogWorker-later-at-region-serv.patch splitlogworker should be started in startServiceThreads() instead of in initializeZookeeper(). This will ensure that the region server accepts a split-logging tasks only after it has successfully done reportForDuty() to the master. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3721) Speedup LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030224#comment-13030224 ] Hudson commented on HBASE-3721: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Speedup LoadIncrementalHFiles - Key: HBASE-3721 URL: https://issues.apache.org/jira/browse/HBASE-3721 Project: HBase Issue Type: Improvement Components: util Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.92.0 Attachments: 3721-v2.txt, 3721-v3.txt, 3721-v4.txt, 3721-v6.patch, 3721.txt, LoadIncrementalHFiles.java From Adam Phelps: from the logs it looks like 1% of the hfiles we're loading have to be split. Looking at the code for LoadIncrementHFiles (hbase v0.90.1), I'm actually thinking our problem is that this code loads the hfiles sequentially. Our largest table has over 2500 regions and the data being loaded is fairly well distributed across them, so there end up being around 2500 HFiles for each load period. At 1-2 seconds per HFile that means the loading process is very time consuming. Currently server.bulkLoadHFile() is a blocking call. We can utilize ExecutorService to achieve better parallelism on multi-core computer. New configuration parameter hbase.loadincremental.threads.max is introduced which sets the maximum number of threads for parallel bulk load. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3431) Regionserver is not using the name given it by the master; double entry in master listing of servers
[
https://issues.apache.org/jira/browse/HBASE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030215#comment-13030215
]
Hudson commented on HBASE-3431:
---
Integrated in HBase-TRUNK #1909 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
Regionserver is not using the name given it by the master; double entry in
master listing of servers
Key: HBASE-3431
URL: https://issues.apache.org/jira/browse/HBASE-3431
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.0
Reporter: stack
Assignee: stack
Priority: Blocker
Fix For: 0.92.0
Attachments: 3431-v2.txt, 3431-v3.txt, 3431-v3.txt, 3431-v4.txt,
3431.txt
Our man Ted Dunning found the following where RS checks in with one name, the
master tells it use another name but we seem to go ahead and continue with
our original name.
In RS logs I see:
{code}
2011-01-07 15:45:50,757 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer [regionserver60020]:
Master passed us address to use. Was=perfnode11:60020, Now=10.10.30.11:60020
{code}
On master I see
{code}
2011-01-07 15:45:38,613 INFO org.apache.hadoop.hbase.master.ServerManager
[IPC Server handler 0 on 6]: Registering
server=10.10.30.11,60020,1294443935414, regionCount=0, userLoad=false
{code}
then later
{code}
2011-01-07 15:45:44,247 INFO org.apache.hadoop.hbase.master.ServerManager
[IPC Server handler 2 on 6]: Registering
server=perfnode11,60020,1294443935414, regionCount=0, userLoad=true
{code}
This might be since we started letting servers register in other than with
the reportStartup.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3849) Fix master ui; hbase-1502 broke requests/second
[ https://issues.apache.org/jira/browse/HBASE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030222#comment-13030222 ] Hudson commented on HBASE-3849: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Fix master ui; hbase-1502 broke requests/second --- Key: HBASE-3849 URL: https://issues.apache.org/jira/browse/HBASE-3849 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1861) Multi-Family support for bulk upload tools (HFileOutputFormat / loadtable.rb)
[ https://issues.apache.org/jira/browse/HBASE-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030223#comment-13030223 ] Hudson commented on HBASE-1861: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) HBASE-3864 Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861 Multi-Family support for bulk upload tools (HFileOutputFormat / loadtable.rb) - Key: HBASE-1861 URL: https://issues.apache.org/jira/browse/HBASE-1861 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.20.0 Reporter: Jonathan Gray Assignee: Nicolas Spiegelberg Fix For: 0.92.0 Attachments: HBASE1861-incomplete.patch Add multi-family support to bulk upload tools from HBASE-48. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1502) Remove need for heartbeats in HBase
[ https://issues.apache.org/jira/browse/HBASE-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030227#comment-13030227 ] Hudson commented on HBASE-1502: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Remove need for heartbeats in HBase --- Key: HBASE-1502 URL: https://issues.apache.org/jira/browse/HBASE-1502 Project: HBase Issue Type: Task Reporter: Nitay Joffe Assignee: stack Priority: Blocker Fix For: 0.92.0 Attachments: 1502-4.txt, 1502-v2.txt, 1502-v5.txt, 1502-v6.txt, 1502-v7.txt, 1502.txt HBase currently uses heartbeats between region servers and the master, piggybacking information on them when it can. This issue is to investigate if we can get rid of the need for those using ZooKeeper events. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3847) Turn off DEBUG logging of RPCs in WriteableRPCEngine on TRUNK
[ https://issues.apache.org/jira/browse/HBASE-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030225#comment-13030225 ] Hudson commented on HBASE-3847: --- Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/]) Turn off DEBUG logging of RPCs in WriteableRPCEngine on TRUNK - Key: HBASE-3847 URL: https://issues.apache.org/jira/browse/HBASE-3847 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3777) Redefine Identity Of HBase Configuration
[
https://issues.apache.org/jira/browse/HBASE-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030228#comment-13030228
]
Hudson commented on HBASE-3777:
---
Integrated in HBase-TRUNK #1909 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
Redefine Identity Of HBase Configuration
Key: HBASE-3777
URL: https://issues.apache.org/jira/browse/HBASE-3777
Project: HBase
Issue Type: Improvement
Components: client, ipc
Affects Versions: 0.90.2
Reporter: Karthick Sankarachary
Assignee: Karthick Sankarachary
Priority: Minor
Fix For: 0.92.0
Attachments: 3777-TOF.patch, HBASE-3777-V2.patch,
HBASE-3777-V3.patch, HBASE-3777-V4.patch, HBASE-3777-V6.patch,
HBASE-3777.patch
Judging from the javadoc in {{HConnectionManager}}, sharing connections
across multiple clients going to the same cluster is supposedly a good thing.
However, the fact that there is a one-to-one mapping between a configuration
and connection instance, kind of works against that goal. Specifically, when
you create {{HTable}} instances using a given {{Configuration}} instance and
a copy thereof, we end up with two distinct {{HConnection}} instances under
the covers. Is this really expected behavior, especially given that the
configuration instance gets cloned a lot?
Here, I'd like to play devil's advocate and propose that we deep-compare
{{HBaseConfiguration}} instances, so that multiple {{HBaseConfiguration}}
instances that have the same properties map to the same {{HConnection}}
instance. In case one is concerned that a single {{HConnection}} is
insufficient for sharing amongst clients, to quote the javadoc, then one
should be able to mark a given {{HBaseConfiguration}} instance as being
uniquely identifiable.
Note that sharing connections makes clean up of {{HConnection}} instances a
little awkward, unless of course, you apply the change described in
HBASE-3766.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2899) hfile.min.blocksize.size ignored/documentation wrong
[
https://issues.apache.org/jira/browse/HBASE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030229#comment-13030229
]
Hudson commented on HBASE-2899:
---
Integrated in HBase-TRUNK #1909 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
HBASE-3864 Rename of hfile.min.blocksize.size in HBASE-2899 reverted in
HBASE-1861
hfile.min.blocksize.size ignored/documentation wrong
Key: HBASE-2899
URL: https://issues.apache.org/jira/browse/HBASE-2899
Project: HBase
Issue Type: Bug
Reporter: Lars Francke
Assignee: stack
Priority: Trivial
Attachments: 2899.txt
There is a property in hbase-default.xml called {{hfile.min.blocksize.size}}
set to {{65536}}.
The description says: Minimum store file block size. The smaller you make
this, the bigger your index and the less you fetch on a random-access. Set
size down if you have small cells and want faster random-access of
individual cells.
This property is only used in the HFileOutputFormat and nowhere else. So we
should at least change the description to something more meaningful.
The other option I see would be: HFile now has a DEFAULT_BLOCKSIZE field
which could be moved to HConstants and HFile could somehow read the
{{hfile.min.blocksize.size}} from the Configuration or use
HConstansts.DEFAULT_BLOCKSIZE if it's not defined. I believe this is what's
happening to the other config variables?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3763) Add Bloom Block Index Support
[ https://issues.apache.org/jira/browse/HBASE-3763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030242#comment-13030242 ] Joydeep Sen Sarma commented on HBASE-3763: -- Dhruba pointed me to some of these jiras. one quick comment is that _if_ the intention is to keep the filters pinned in memory - then we can convert the load at read time to: - load at startup time as quickly as possible - keep the filter pinned in memory when writing out new hfile (never have to read it in) this would also take out filter reads from client read path. Add Bloom Block Index Support - Key: HBASE-3763 URL: https://issues.apache.org/jira/browse/HBASE-3763 Project: HBase Issue Type: Improvement Components: io, regionserver Affects Versions: 0.89.20100924, 0.90.0, 0.90.1, 0.90.2 Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Labels: hbase, performance Fix For: 0.89.20100924 Original Estimate: 0h Remaining Estimate: 0h Add a way to save HBase Bloom filters into an array of Meta blocks instead of one big Meta block, and load only the blocks required to answer a query. This will allow us faster bloom load times for large StoreFiles pave the path for adding Bloom Filter support to HFileOutputFormat bulk load. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3863) Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
[
https://issues.apache.org/jira/browse/HBASE-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030273#comment-13030273
]
Hudson commented on HBASE-3863:
---
Integrated in HBase-TRUNK #1910 (See
[https://builds.apache.org/hudson/job/HBase-TRUNK/1910/])
HBASE-3863 Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
Fix failing TestHBaseFsck.testHBaseFsckMeta unit test
-
Key: HBASE-3863
URL: https://issues.apache.org/jira/browse/HBASE-3863
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.92.0
Attachments: 3863.txt
Failing with:
{code}
Error Message
org.apache.hadoop.hbase.HServerAddress cannot be cast to
org.apache.hadoop.hbase.ServerName
Stacktrace
java.lang.ClassCastException: org.apache.hadoop.hbase.HServerAddress cannot
be cast to org.apache.hadoop.hbase.ServerName
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixDupeAssignment(HBaseFsckRepair.java:59)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsckMeta(TestHBaseFsck.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
