[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13246954#comment-13246954 ] Hudson commented on HBASE-4398: --- Integrated in HBase-0.92-security #104 (See [https://builds.apache.org/job/HBase-0.92-security/104/]) HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307118) Result = FAILURE stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.0 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247030#comment-13247030 ] Hudson commented on HBASE-4398: --- Integrated in HBase-TRUNK-security #157 (See [https://builds.apache.org/job/HBase-TRUNK-security/157/]) HBASE-5704 HBASE-4398 mistakenly rolled back on trunk (Revision 1308964) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.0 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13245363#comment-13245363 ] Takuya Ueshin commented on HBASE-4398: -- Rolled back in trunk? If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.0 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13245381#comment-13245381 ] stack commented on HBASE-4398: -- @Takuya Sorry about that. A temporary disable while trying to figure why trunk build was broken got committed. I fixed trunk over in HBASE-5704. Thanks for noticing this. If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.0 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13245750#comment-13245750 ] Hudson commented on HBASE-4398: --- Integrated in HBase-TRUNK #2706 (See [https://builds.apache.org/job/HBase-TRUNK/2706/]) HBASE-5704 HBASE-4398 mistakenly rolled back on trunk (Revision 1308964) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.0 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242447#comment-13242447 ] Hudson commented on HBASE-4398: --- Integrated in HBase-0.94-security #6 (See [https://builds.apache.org/job/HBase-0.94-security/6/]) HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307117) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.1 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13241787#comment-13241787 ] stack commented on HBASE-4398: -- Good one Takuya. Thats broke. I would suggest that your test case while comprehensive, it is way overkill for your fix. Its also mostly a copy of another heavy-weight test we already run. I can't believe I'm saying this but I'm going to commit this fix w/o including the test. If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13241792#comment-13241792 ] stack commented on HBASE-4398: -- Thanks for the patch Takuya. I applied it to 0.92, 0.94, and trunk. If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.1 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13241891#comment-13241891 ] Hudson commented on HBASE-4398: --- Integrated in HBase-0.94 #67 (See [https://builds.apache.org/job/HBase-0.94/67/]) HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307117) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.1 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13241994#comment-13241994 ] Hudson commented on HBASE-4398: --- Integrated in HBase-0.92 #344 (See [https://builds.apache.org/job/HBase-0.92/344/]) HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307118) Result = SUCCESS stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.1 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242062#comment-13242062 ] Hudson commented on HBASE-4398: --- Integrated in HBase-TRUNK-security #154 (See [https://builds.apache.org/job/HBase-TRUNK-security/154/]) HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307116) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. - Key: HBASE-4398 URL: https://issues.apache.org/jira/browse/HBASE-4398 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Takuya Ueshin Assignee: Takuya Ueshin Fix For: 0.92.2, 0.94.1 Attachments: HBASE-4398.patch If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml. We can reproduce the problem by the following instructions: {noformat} - Add HRegionPartitioner.class to the 4th argument of TableMapReduceUtil#initTableReducerJob() at line around 133 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java - Change or remove hbase.zookeeper.property.clientPort property in hbase-site.xml ( for example, changed to 12345 ). - run testMultiRegionTable() {noformat} Then I got error messages as following: {noformat} 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345) 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms = 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001 java.lang.NullPointerException at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) {noformat} I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml and It might be caused by HBaseConfiguration.addHbaseResources(conf); in HRegionPartitioner#setConf(Configuration). And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira