[jira] [Updated] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[
https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-4398:
-
Fix Version/s: 0.94.0
If HRegionPartitioner is used in MapReduce, client side configurations are
overwritten by hbase-site.xml.
-
Key: HBASE-4398
URL: https://issues.apache.org/jira/browse/HBASE-4398
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.90.4
Reporter: Takuya Ueshin
Assignee: Takuya Ueshin
Fix For: 0.94.0, 0.95.1
Attachments: HBASE-4398.patch
If HRegionPartitioner is used in MapReduce, client side configurations are
overwritten by hbase-site.xml.
We can reproduce the problem by the following instructions:
{noformat}
- Add HRegionPartitioner.class to the 4th argument of
TableMapReduceUtil#initTableReducerJob()
at line around 133
in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
- Change or remove hbase.zookeeper.property.clientPort property
in hbase-site.xml ( for example, changed to 12345 ).
- run testMultiRegionTable()
{noformat}
Then I got error messages as following:
{noformat}
2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection
opening connection to ZooKeeper with ensemble (localhost:12345)
2011-09-12 22:28:51,022 INFO [Thread-832]
zookeeper.RecoverableZooKeeper(89): The identifier of this process is
43200@imac.local
2011-09-12 22:28:51,123 WARN [Thread-832]
zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception:
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
= ConnectionLoss for /hbase/master
2011-09-12 22:28:51,123 INFO [Thread-832]
zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after
sleeping 1000 ms
=
2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125):
java.io.IOException:
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d
closed
2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256):
job_local_0001
java.lang.NullPointerException
at
org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
{noformat}
I think HTable should connect to ZooKeeper at port 21818 configured at client
side instead of 12345 in hbase-site.xml
and It might be caused by HBaseConfiguration.addHbaseResources(conf); in
HRegionPartitioner#setConf(Configuration).
And this might mean that all of client side configurations, also configured
in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[
https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-4398:
-
Fix Version/s: (was: 0.94.1)
0.94.0
If HRegionPartitioner is used in MapReduce, client side configurations are
overwritten by hbase-site.xml.
-
Key: HBASE-4398
URL: https://issues.apache.org/jira/browse/HBASE-4398
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.90.4
Reporter: Takuya Ueshin
Assignee: Takuya Ueshin
Fix For: 0.92.2, 0.94.0
Attachments: HBASE-4398.patch
If HRegionPartitioner is used in MapReduce, client side configurations are
overwritten by hbase-site.xml.
We can reproduce the problem by the following instructions:
{noformat}
- Add HRegionPartitioner.class to the 4th argument of
TableMapReduceUtil#initTableReducerJob()
at line around 133
in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
- Change or remove hbase.zookeeper.property.clientPort property
in hbase-site.xml ( for example, changed to 12345 ).
- run testMultiRegionTable()
{noformat}
Then I got error messages as following:
{noformat}
2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection
opening connection to ZooKeeper with ensemble (localhost:12345)
2011-09-12 22:28:51,022 INFO [Thread-832]
zookeeper.RecoverableZooKeeper(89): The identifier of this process is
43200@imac.local
2011-09-12 22:28:51,123 WARN [Thread-832]
zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception:
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
= ConnectionLoss for /hbase/master
2011-09-12 22:28:51,123 INFO [Thread-832]
zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after
sleeping 1000 ms
=
2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125):
java.io.IOException:
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d
closed
2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256):
job_local_0001
java.lang.NullPointerException
at
org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
{noformat}
I think HTable should connect to ZooKeeper at port 21818 configured at client
side instead of 12345 in hbase-site.xml
and It might be caused by HBaseConfiguration.addHbaseResources(conf); in
HRegionPartitioner#setConf(Configuration).
And this might mean that all of client side configurations, also configured
in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[
https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takuya Ueshin updated HBASE-4398:
-
Attachment: HBASE-4398.patch
I attatched a patch file.
Tnis includes a patch for HRegionPartitioner and a test case.
If HRegionPartitioner is used in MapReduce, client side configurations are
overwritten by hbase-site.xml.
-
Key: HBASE-4398
URL: https://issues.apache.org/jira/browse/HBASE-4398
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.90.4
Reporter: Takuya Ueshin
Attachments: HBASE-4398.patch
If HRegionPartitioner is used in MapReduce, client side configurations are
overwritten by hbase-site.xml.
We can reproduce the problem by the following instructions:
{noformat}
- Add HRegionPartitioner.class to the 4th argument of
TableMapReduceUtil#initTableReducerJob()
at line around 133
in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
- Change or remove hbase.zookeeper.property.clientPort property
in hbase-site.xml ( for example, changed to 12345 ).
- run testMultiRegionTable()
{noformat}
Then I got error messages as following:
{noformat}
2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection
opening connection to ZooKeeper with ensemble (localhost:12345)
2011-09-12 22:28:51,022 INFO [Thread-832]
zookeeper.RecoverableZooKeeper(89): The identifier of this process is
43200@imac.local
2011-09-12 22:28:51,123 WARN [Thread-832]
zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception:
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
= ConnectionLoss for /hbase/master
2011-09-12 22:28:51,123 INFO [Thread-832]
zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after
sleeping 1000 ms
=
2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125):
java.io.IOException:
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d
closed
2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256):
job_local_0001
java.lang.NullPointerException
at
org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
{noformat}
I think HTable should connect to ZooKeeper at port 21818 configured at client
side instead of 12345 in hbase-site.xml
and It might be caused by HBaseConfiguration.addHbaseResources(conf); in
HRegionPartitioner#setConf(Configuration).
And this might mean that all of client side configurations, also configured
in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
