[jira] [Reopened] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5809: --- The test failure was visible on Hadoop QA as well: https://builds.apache.org/job/PreCommit-HBASE-Build/1594//testReport/org.apache.hadoop.hbase.coprocessor/TestMasterObserver/testRegionTransitionOperations/ > Avoid move api to take the destination server same as the source server. > > > Key: HBASE-5809 > URL: https://issues.apache.org/jira/browse/HBASE-5809 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: ramkrishna.s.vasudevan >Assignee: rajeshbabu >Priority: Minor > Labels: client > Fix For: 0.96.0 > > Attachments: HBASE-5809.patch, HBASE-5809.patch > > > In Move currently we take any destination specified and if the destination is > same as the source we still do unassign and assign. Here we can have > problems due to RegionAlreadyInTransitionException and thus hanging the > region in RIT for long time. We can avoid this scenario by not allowing the > move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[
https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu reopened HBASE-5824:
---
I can reproduce one of the test failures reported by Hadoop QA:
{code}
testConstraintFails(org.apache.hadoop.hbase.constraint.TestConstraint) Time
elapsed: 3.174 sec <<< ERROR!
org.apache.hadoop.hbase.constraint.ConstraintException:
org.apache.hadoop.hbase.constraint.ConstraintException: AllFailConstraint fails
for all puts
...
Caused by: org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hbase.constraint.ConstraintException: AllFailConstraint fails
for all puts
at
org.apache.hadoop.hbase.constraint.AllFailConstraint.check(AllFailConstraint.java:29)
at
org.apache.hadoop.hbase.constraint.ConstraintProcessor.prePut(ConstraintProcessor.java:87)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:656)
at org.apache.hadoop.hbase.regionserver.HRegion.internalPut(HRegion.java:2434)
at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1891)
at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1857)
at
org.apache.hadoop.hbase.regionserver.RegionServer.mutate(RegionServer.java:523)
{code}
> HRegion.incrementColumnValue is not used in trunk
> -
>
> Key: HBASE-5824
> URL: https://issues.apache.org/jira/browse/HBASE-5824
> Project: HBase
> Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5824.patch, hbase-5824_v2.patch
>
>
> on 0.94 a call to client.HTable#incrementColumnValue will cause
> HRegion#incrementColumnValue. On trunk all calls to
> HTable.incrementColumnValue got to HRegion#increment.
> My guess is that HTable#incrementColumnValue and HTable#increment serialize
> to the same thing over the wire so that the remote HRegionServer no longer
> knows which htable method was called.
> To repro I checked out trunk and put a break point in
> HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint
> wasn't hit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5620) Convert the client protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5620: --- As you can see in https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-security/171/, there were 12 more test failures. To narrow the checkin that caused additional test failures, I locally backed out HBASE-5684 and HBASE-5747. TestAccessControlFilter#testQualifierAccess still failed with access permission error. After backing out this JIRA, the test passed. > Convert the client protocol of HRegionInterface to PB > - > > Key: HBASE-5620 > URL: https://issues.apache.org/jira/browse/HBASE-5620 > Project: HBase > Issue Type: Sub-task > Components: ipc, master, migration, regionserver >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5620_v3.patch, hbase-5620_v4.patch, > hbase-5620_v4.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5747) Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test"
[ https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5747: --- Trunk builds #2757 to #2760 all failed due to some hanging test. > Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a > subdirectory of target/test" > > > Key: HBASE-5747 > URL: https://issues.apache.org/jira/browse/HBASE-5747 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 5474.txt, 5474v2.txt, 5474v3 (1).txt, 5474v3.txt, > 5708v4.txt, 5708v4.txt > > > Forward port as much as we can of Mikhail's hard-won test cleanups over on > 0.89 branch Will improve our being able to run unit tests in //. He also > found a few bugs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5727) secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs'
[ https://issues.apache.org/jira/browse/HBASE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5727: --- The compilation issue is real. > secure hbase build broke because of 'HBASE-5451 Switch RPC call > envelope/headers to PBs' > > > Key: HBASE-5727 > URL: https://issues.apache.org/jira/browse/HBASE-5727 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Devaraj Das >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 5727.1.patch, 5727.2.patch, 5727.patch > > > If you build with the security profile -- i.e. add '-P security' on the > command line -- you'll see that the secure build is broke since we messed in > rpc. > Assigning Deveraj to take a look. If you can't work on this now DD, just > give it back to me and I'll have a go at it. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5727) secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs'
[
https://issues.apache.org/jira/browse/HBASE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu reopened HBASE-5727:
---
Compilation failed because unresolved conflict in
security/src/main/java/org/apache/hadoop/hbase/ipc/SecureClient.java:
{code}
<<< .mine
final DataOutputBuffer d = new DataOutputBuffer();
{code}
> secure hbase build broke because of 'HBASE-5451 Switch RPC call
> envelope/headers to PBs'
>
>
> Key: HBASE-5727
> URL: https://issues.apache.org/jira/browse/HBASE-5727
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Devaraj Das
>Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: 5727.1.patch, 5727.2.patch, 5727.patch
>
>
> If you build with the security profile -- i.e. add '-P security' on the
> command line -- you'll see that the secure build is broke since we messed in
> rpc.
> Assigning Deveraj to take a look. If you can't work on this now DD, just
> give it back to me and I'll have a go at it. Thanks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5213) "hbase master stop" does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5213: --- TestLogRolling hangs in 0.90 > "hbase master stop" does not bring down backup masters > -- > > Key: HBASE-5213 > URL: https://issues.apache.org/jira/browse/HBASE-5213 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: 5213.jstack, HBASE-5213-v0-trunk.patch, > HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, > HBASE-5213-v2-trunk.patch > > > Typing "hbase master stop" produces the following message: > "stop Start cluster shutdown; Master signals RegionServer shutdown" > It seems like backup masters should be considered part of the cluster, but > they are not brought down by "hbase master stop". > "stop-hbase.sh" does correctly bring down the backup masters. > The same behavior is observed when a client app makes use of the client API > HBaseAdmin.shutdown() > http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() > -- this isn't too surprising since I think "hbase master stop" just calls > this API. > It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5711) Tests are failing with incorrect data directory permissions.
[
https://issues.apache.org/jira/browse/HBASE-5711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu reopened HBASE-5711:
---
@ 04/Apr/12 18:30, TestRowProcessorEndpoint was reported by Hadoop QA to fail.
After I reverted the patch, TestRowProcessorEndpoint passed on trunk.
> Tests are failing with incorrect data directory permissions.
>
>
> Key: HBASE-5711
> URL: https://issues.apache.org/jira/browse/HBASE-5711
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-5711.patch
>
>
> When we run some tests in Hbase (TestAdmin), it is failing with following
> error.
> {quote}
> Starting DataNode 0 with dfs.data.dir:
> E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb\dfs\data\data1,E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb\dfs\data\data2
> 2012-04-04 18:04:51,036 WARN [main] impl.MetricsSystemImpl(137): Metrics
> system not started: Cannot locate configuration: tried
> hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
> 2012-04-04 18:04:51,255 WARN [main] datanode.DataNode(1548): Invalid
> directory in dfs.data.dir: Incorrect permission for
> E:/Repositories/Hbase/target/test-data/5ff23198-892e-4f1c-8022-b3d9969fcf0b/dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb/dfs/data/data1,
> expected: rwxr-xr-x, while actual: rwx--
> 2012-04-04 18:04:51,411 WARN [main] datanode.DataNode(1548): Invalid
> directory in dfs.data.dir: Incorrect permission for
> E:/Repositories/Hbase/target/test-data/5ff23198-892e-4f1c-8022-b3d9969fcf0b/dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb/dfs/data/data2,
> expected: rwxr-xr-x, while actual: rwx--
> 2012-04-04 18:04:51,411 ERROR [main] datanode.DataNode(1554): All directories
> in dfs.data.dir are invalid.
> 2012-04-04 18:04:51,411 INFO [main] hbase.HBaseTestingUtility(684): Shutting
> down minicluster
> 2012-04-04 18:04:51,646 WARN [main] hbase.HBaseTestingUtility(696): Failed
> delete of
> E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb
> 2012-04-04 18:04:51,646 INFO [main] hbase.HBaseTestingUtility(700):
> Minicluster is down
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[
https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu reopened HBASE-5724:
---
0.90 branch doesn't compile now:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
/x1/jenkins/jenkins-slave/workspace/hbase-0.90/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java:[404,64]
cannot find symbol
[ERROR] symbol : variable WritableUtils
[ERROR] location: class org.apache.hadoop.hbase.TestKeyValue
{code}
> Row cache of KeyValue should be cleared in readFields().
>
>
> Key: HBASE-5724
> URL: https://issues.apache.org/jira/browse/HBASE-5724
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: Teruyoshi Zenmyo
>Assignee: Teruyoshi Zenmyo
> Fix For: 0.90.7, 0.92.2, 0.94.0
>
> Attachments: 5724.092.txt, HBASE-5724.txt, HBASE-5724v2.txt
>
>
> KeyValue does not clear its row cache in reading new values (readFields()).
> Therefore, If a KeyValue (kv) which caches its row bytes reads another
> KeyValue instance, kv.getRow() returns a wrong value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5564) Bulkload is discarding duplicate records
[ https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5564: --- By reverting the patch applied to trunk, TestImportTsv#testMROnTableWithCustomMapper passes. > Bulkload is discarding duplicate records > > > Key: HBASE-5564 > URL: https://issues.apache.org/jira/browse/HBASE-5564 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 0.96.0 > Environment: HBase 0.92 >Reporter: Laxman >Assignee: Laxman > Labels: bulkloader > Fix For: 0.96.0 > > Attachments: 5564.lint, 5564v5.txt, HBASE-5564_trunk.1.patch, > HBASE-5564_trunk.1.patch, HBASE-5564_trunk.2.patch, HBASE-5564_trunk.3.patch, > HBASE-5564_trunk.4_final.patch, HBASE-5564_trunk.patch > > > Duplicate records are getting discarded when duplicate records exists in same > input file and more specifically if they exists in same split. > Duplicate records are considered if the records are from diffrent different > splits. > Version under test: HBase 0.92 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5673) The OOM problem of IPC client call cause all handle block
[ https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5673: --- The patch applies cleanly to TRUNK. It was imprudent to integrate the patch without going through QA cycle. Now all branches are broken. > The OOM problem of IPC client call cause all handle block > -- > > Key: HBASE-5673 > URL: https://issues.apache.org/jira/browse/HBASE-5673 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.6 > Environment: 0.90.6 >Reporter: xufeng >Assignee: xufeng > Fix For: 0.90.7, 0.92.2, 0.94.1 > > Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch > > > if HBaseClient meet "unable to create new native thread" exception, the call > will never complete because it be lost in calls queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu reopened HBASE-5592:
---
The reversion didn't happen, right ?
This would be in conflict with Jesse's current work.
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.96.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5568) Multi concurrent flushcache() for one region could cause data loss
[ https://issues.apache.org/jira/browse/HBASE-5568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5568: --- Patch for 0.92 hasn't been integrated. > Multi concurrent flushcache() for one region could cause data loss > -- > > Key: HBASE-5568 > URL: https://issues.apache.org/jira/browse/HBASE-5568 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5568-90.patch, HBASE-5568-92v2.patch, > HBASE-5568.patch, HBASE-5568.patch, HBASE-5568v2.patch > > > We could call HRegion#flushcache() concurrently now through > HRegionServer#splitRegion or HRegionServer#flushRegion by HBaseAdmin. > However, we find if HRegion#internalFlushcache() is called concurrently by > multi thread, HRegion.memstoreSize will be calculated wrong. > At the end of HRegion#internalFlushcache(), we will do > this.addAndGetGlobalMemstoreSize(-flushsize), but the flushsize may not the > actual memsize which flushed to hdfs. It cause HRegion.memstoreSize is > negative and prevent next flush if we close this region. > Logs in RS for region e9d827913a056e696c39bc569ea3 > 2012-03-11 16:31:36,690 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 128.0m > 2012-03-11 16:31:37,999 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/8162481165586107427, entries=153106, sequenceid=619316544, > memsize=59.6m, filesize=31.2m > 2012-03-11 16:31:38,830 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 134.8m > 2012-03-11 16:31:39,458 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/3425971951499794221, entries=230183, sequenceid=619316544, > memsize=68.5m, filesize=26.6m > 2012-03-11 16:31:39,459 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~128.1m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 2769ms, sequenceid=619316544, compaction > requested=false > 2012-03-11 16:31:39,459 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 6.8m > 2012-03-11 16:31:39,529 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/1811012969998104626, entries=8002, sequenceid=619332759, > memsize=3.1m, filesize=1.6m > 2012-03-11 16:31:39,640 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/770333473623552048, entries=12231, sequenceid=619332759, > memsize=3.6m, filesize=1.4m > 2012-03-11 16:31:39,641 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~134.8m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 811ms, sequenceid=619332759, compaction > requested=true > 2012-03-11 16:31:39,707 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/5656568849587368557, entries=119, sequenceid=619332979, > memsize=47.4k, filesize=25.6k > 2012-03-11 16:31:39,775 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/794343845650987521, entries=157, sequenceid=619332979, > memsize=47.8k, filesize=19.3k > 2012-03-11 16:31:39,777 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~6.8m for region > writetest1,,1331454657410.e9d827913a05 > 6e696c39bc569ea3f99f. in 318ms, sequenceid=619332979, compaction > requested=true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5399: --- TestZooKeeper.testClientSessionExpired fails on Hadoop QA after this patch went in. > Cut the link between the client and the zookeeper ensemble > -- > > Key: HBASE-5399 > URL: https://issues.apache.org/jira/browse/HBASE-5399 > Project: HBase > Issue Type: Improvement > Components: client >Affects Versions: 0.94.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, > 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399.v42.patch, > 5399.v42.patch, 5399.v42.patch, 5399_inprogress.patch, > 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, > 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, > 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, > 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, > 5399_inprogress.v9.patch, nochange.patch > > > The link is often considered as an issue, for various reasons. One of them > being that there is a limit on the number of connection that ZK can manage. > Stack was suggesting as well to remove the link to master from HConnection. > There are choices to be made considering the existing API (that we don't want > to break). > The first patches I will submit on hadoop-qa should not be committed: they > are here to show the progress on the direction taken. > ZooKeeper is used for: > - public getter, to let the client do whatever he wants, and close ZooKeeper > when closing the connection => we have to deprecate this but keep it. > - read get master address to create a master => now done with a temporary > zookeeper connection > - read root location => now done with a temporary zookeeper connection, but > questionable. Used in public function "locateRegion". To be reworked. > - read cluster id => now done once with a temporary zookeeper connection. > - check if base done is available => now done once with a zookeeper > connection given as a parameter > - isTableDisabled/isTableAvailable => public functions, now done with a > temporary zookeeper connection. > - Called internally from HBaseAdmin and HTable > - getCurrentNrHRS(): public function to get the number of region servers and > create a pool of thread => now done with a temporary zookeeper connection > - > Master is used for: > - getMaster public getter, as for ZooKeeper => we have to deprecate this but > keep it. > - isMasterRunning(): public function, used internally by HMerge & HBaseAdmin > - getHTableDescriptor*: public functions offering access to the master. => > we could make them using a temporary master connection as well. > Main points are: > - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a > strongly coupled architecture ;-). This can be changed, but requires a lot of > modifications in these classes (likely adding a class in the middle of the > hierarchy, something like that). Anyway, non connected client will always be > really slower, because it's a tcp connection, and establishing a tcp > connection is slow. > - having a link between ZK and all the client seems to make sense for some > Use Cases. However, it won't scale if a TCP connection is required for every > client > - if we move the table descriptor part away from the client, we need to find > a new place for it. > - we will have the same issue if HBaseAdmin (for both ZK & Master), may be we > can put a timeout on the connection. That would make the whole system less > deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5473) Metrics does not push pread time
[
https://issues.apache.org/jira/browse/HBASE-5473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu reopened HBASE-5473:
---
0.92 build #305 failed due to compilation error:
{code}
[ERROR]
:[313,10]
cannot find symbol
[ERROR] symbol : variable fsPreadLatency
[ERROR] location: class
org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
{code}
> Metrics does not push pread time
>
>
> Key: HBASE-5473
> URL: https://issues.apache.org/jira/browse/HBASE-5473
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: dhruba borthakur
>Assignee: dhruba borthakur
>Priority: Minor
> Fix For: 0.92.1, 0.94.0
>
> Attachments: D1947.1.patch, D1947.1.patch, D1947.1.patch, D1947.patch
>
>
> The RegionServerMetrics is not pushing the pread times to the MetricsRecord
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5318) Support Eclipse Indigo
[ https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5318: --- After reverting this patch, TestInfoServers passed. > Support Eclipse Indigo > --- > > Key: HBASE-5318 > URL: https://issues.apache.org/jira/browse/HBASE-5318 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.94.0 > Environment: Eclipse Indigo (1.4.1) which includes m2eclipse (1.0 > SR1). >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Minor > Labels: maven > Attachments: mvn_HBASE-5318_r0.patch > > > The current 'standard' release of Eclipse (indigo) comes with m2eclipse > installed. However, as of m2e v1.0, "interesting lifecycle phases" are now > handled via a 'connector'. However, several of the plugins we use don't > support connectors. This means that eclipse bails out and won't build the > project or view it as 'working' even though it builds just fine from the the > command line. > Since Eclipse is one of the major java IDEs and that Indigo has been around > for a while, we should make it easy to for new devs to pick up the code and > for older devs to upgrade painlessly. The original build should not be > modified in any significant way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5235) HLogSplitter writer thread's streams not getting closed when any of the writer threads has exceptions.
[
https://issues.apache.org/jira/browse/HBASE-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu reopened HBASE-5235:
---
Patch should be integrated to 0.92 branch as well.
> HLogSplitter writer thread's streams not getting closed when any of the
> writer threads has exceptions.
> --
>
> Key: HBASE-5235
> URL: https://issues.apache.org/jira/browse/HBASE-5235
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: HBASE-5235_0.90.patch, HBASE-5235_0.90_1.patch,
> HBASE-5235_0.90_2.patch, HBASE-5235_trunk.patch
>
>
> Pls find the analysis. Correct me if am wrong
> {code}
> 2012-01-15 05:14:02,374 FATAL
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: WriterThread-9 Got
> while writing log entry to log
> java.io.IOException: All datanodes 10.18.40.200:50010 are bad. Aborting...
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:3373)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2811)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3026)
> {code}
> Here we have an exception in one of the writer threads. If any exception we
> try to hold it in an Atomic variable
> {code}
> private void writerThreadError(Throwable t) {
> thrown.compareAndSet(null, t);
> }
> {code}
> In the finally block of splitLog we try to close the streams.
> {code}
> for (WriterThread t: writerThreads) {
> try {
> t.join();
> } catch (InterruptedException ie) {
> throw new IOException(ie);
> }
> checkForErrors();
> }
> LOG.info("Split writers finished");
>
> return closeStreams();
> {code}
> Inside checkForErrors
> {code}
> private void checkForErrors() throws IOException {
> Throwable thrown = this.thrown.get();
> if (thrown == null) return;
> if (thrown instanceof IOException) {
> throw (IOException)thrown;
> } else {
> throw new RuntimeException(thrown);
> }
> }
> So once we throw the exception the DFSStreamer threads are not getting closed.
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5099) ZK event thread waiting for root region assignment may block server shutdown handler for the region sever the root region was on
[ https://issues.apache.org/jira/browse/HBASE-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5099: --- 0.92 Jenkins builds have failed 4 times in a roll. TestReplication#queueFailover failed in builds 217 and 218. It failed consistently on MacBook as well. Rolling back the patches. > ZK event thread waiting for root region assignment may block server shutdown > handler for the region sever the root region was on > > > Key: HBASE-5099 > URL: https://issues.apache.org/jira/browse/HBASE-5099 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0, 0.94.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.92.0, 0.94.0 > > Attachments: 5099.92, ZK-event-thread-waiting-for-root.png, > distributed-log-splitting-hangs.png, hbase-5099-v2.patch, > hbase-5099-v3.patch, hbase-5099-v4.patch, hbase-5099-v5.patch, > hbase-5099-v6.patch, hbase-5099.patch > > > A RS died. The ServerShutdownHandler kicked in and started the logspliting. > SpliLogManager > installed the tasks asynchronously, then started to wait for them to complete. > The task znodes were not created actually. The requests were just queued. > At this time, the zookeeper connection expired. HMaster tried to recover the > expired ZK session. > During the recovery, a new zookeeper connection was created. However, this > master became the > new master again. It tried to assign root and meta. > Because the dead RS got the old root region, the master needs to wait for the > log splitting to complete. > This waiting holds the zookeeper event thread. So the async create split > task is never retried since > there is only one event thread, which is waiting for the root region assigned. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5021) Enforce upper bound on timestamp
[ https://issues.apache.org/jira/browse/HBASE-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5021: --- TestHeapSize fails on Jenkins > Enforce upper bound on timestamp > > > Key: HBASE-5021 > URL: https://issues.apache.org/jira/browse/HBASE-5021 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Critical > Fix For: 0.94.0 > > Attachments: D849.1.patch, D849.2.patch, D849.3.patch, > HBASE-5021-trunk.patch > > > We have been getting hit with performance problems on our time-series > database due to invalid timestamps being inserted by the timestamp. We are > working on adding proper checks to app server, but production performance > could be severely impacted with significant recovery time if something slips > past. Since timestamps are considered a fundamental part of the HBase schema > & multiple optimizations use timestamp information, we should allow the > option to sanity check the upper bound on the server-side in HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5029) TestDistributedLogSplitting fails on occasion
[ https://issues.apache.org/jira/browse/HBASE-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5029: --- The test failed again in build #193 > TestDistributedLogSplitting fails on occasion > - > > Key: HBASE-5029 > URL: https://issues.apache.org/jira/browse/HBASE-5029 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Prakash Khemani > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-5029-jira-TestDistributedLogSplitting-fails-on.patch, > HBASE-5029.D891.1.patch, HBASE-5029.D891.2.patch > > > This is how it usually fails: > https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/lastCompletedBuild/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testWorkerAbort/ > Assigning mighty Prakash since he offered to take a looksee. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
