Hi St.Ack, thinking about it again, I think that HBase cannot really do much about it.
I checked in hadoop/trunk and Configuration still has the WeakHashmap REGISTRY that is used upon calling addDefaultResource (which is called from mapred classes at least) to reset all reachable Configuration objects plus it has addResource(InputStream). I sent a note to the hadoop mailing list on this. Thanks, Henning On Wed, 2010-10-13 at 18:06 -0400, Stack wrote: > On Wed, Oct 13, 2010 at 10:09 AM, Henning Blohm > <[email protected]> wrote: > > Hi, > > > > I used > > > > Configuration cfg = new Configuration(); > > cfg.addResource(InputStream) > > > > on org.apache.hadoop.conf.Configuration to create hadoop configuration. > > That configuration is > > used to construct HbaseConfiguration objects several times later on: > > > > HbaseConfiguration hbcfg = new HbaseConfiguration(cfg) > > > > In TRUNK HBaseConfiguration is deprecated. Do you see same phenomena > when you do HBaseConfiguration.create(cfg)? > > See below... > > > At the second this leads to an error as the passed on configuraiton > > object > > is asked to load its properties again, trying to read from the > > inputstream once more > > (as the stream is memorized!!). > > > > The fact that the properties of the passed on configuration object have > > been reset somewhat > > surprising: When using the hbase configuration object constructed above > > in HbaseAdmin, this > > will eventually load JobConf which has a static initializer that calls > > > > Configuration.addDefaultResource("mapred-default.xml"); > > > > which in turn goes through the Configuration.REGISTRY which holds on to > > any previously > > not yet collected Configuration objects to (hold your breath!) force > > them to reload their configuration > > by calling reloadConfiguration that (now we are there) sets > > properties=null. > > > > Did anybody follow that.... > > > > It seems there is somewhat surprising side effects in hadoop/hbase > > configuration handling. > > > > Wouldn't it be better to have the default resource (pragmatically) > > defined once in Configuration > > and not (even think about) touch already instantiated config objects > > later on? > > > > Yes. That sounds completely reasonable. > > See if HBaseConfiguration.create(cfg) gives you the same issue. If > so, then its the way Hadoop Configuration works currently. I haven't > spent time on it in a while but I remember getting into interesting > scenarios loading properties. If you can confine the issue some, lets > file an issue up in hadoop common? > > Thanks Henning, > St.Ack
