I think, many things should be fixed.
1. Basically the value of "bsp.master.address" is setted to
"localhost" In hama-default.xml:
<name>bsp.master.address</name>
<value>localhost</value>
<description>The address of the bsp master server. Either the
literal string "local" or a host[:port] (where host is a name or
If we add a LocalBSPJobRunner to Hama 0.3, I think, we should use
"local" as a default value. ("local" ≠ "localhost"). Description
strings should be fixed, too.
2. In eclipse, conf.get("bsp.master.address") returns null.
String mode = conf.get("bsp.master.address");
LOG.info(">>>>>> " + mode);
It means that the config files are not loaded. If we run with advanced
configurations in eclipse, it'll be work but, it's really annoying.
I think, we can set the basic configurations for JUnit tests by
hard-coding in MiniClusterTest constructor.
On Thu, Apr 14, 2011 at 8:02 PM, chl501 <[email protected]> wrote:
> My question should be `Can't we simply use the default value, which sets
> bsp.master.address to localhost, in hama-default.xml'?
>
> Currently when using the default value (localhost), the bspMasterAddr will
> has the value referenced to master address/port.
>
> Resetting bsp.master.address to local will cause bspMasterAddr to null.
>
> Why do we need to reset bsp.master.address to local?
>
> Sorry for my poor English.
>
> -----Original message-----
> From:chl501 <[email protected]>
> To:hama-dev <[email protected]>
> Date:Thu, 14 Apr 2011 17:45:38 +0800 (CST)
> Subject:Re: Eclipse test failure for HAMA-376
>
> Looks like the problem is because BSPMaster.getAddress(conf) in GroomServer
> is trying to create socket again after BSPMaster successfully occupies the
> ip/port (GroomServers need to wait until BSPMaster is up).
>
> in the end of NetUtil.createSocketAddr() it shows
>
> public static InetSocketAddress createSocketAddr(String target,
> int defaultPort) {
> ...
> return new InetSocketAddress(hostname, port);
> }
>
> Should we change not to use bspMasterAddr because GroomServer can directly
> obtain bsp.master.address (ip/port) from conf/hama-site.xml?
>
> -----Original message-----
> From:Edward J. Yoon <[email protected]>
> To:[email protected],[email protected]
> Date:Thu, 14 Apr 2011 17:32:01 +0900
> Subject:Re: Eclipse test failure for HAMA-376
>
> Nope, "Class not found error" and "setJar()" problems are unconnected
> things from my report.
>
> I think, the point of miniBSPCluster problem with eclipse and maven
> build, is that the value of "bsp.master.address" is always null:
>
>>11/04/12 20:49:50 INFO bsp.GroomServer: groom start
>>>11/04/12 20:49:51 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:52 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:53 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:54 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:55 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>
> Try to set the value of "mode" to "local" as below:
>
> public GroomServer(Configuration conf) throws IOException {
> ....
> mode = "local";
> ....
>
> 2011/4/14 chl501 <[email protected]>:
>> It is the same issue that the jar file is missing. If it is required to ask
>> use explicitly setJar, is there any place that we can reference to the build
>> version? For instance, the BSPJob.setJar() requires to set jar path e.g.
>> /path/to/eclipse/hama-1090935/build/hama-0.3.0-test.jar where build version
>> 0.3.0 is changed according to ant's property ${version}. Otherwise, we may
>> need to parse ant file for explicitly adding this info.
>>
>> 11/04/14 14:50:13 WARN bsp.BSPJobClient: No job jar file set. User classes
>> may not be found. See BSPJob#setJar(String) or check Your jar file.
>> 11/04/14 14:50:13 INFO bsp.GroomServer: jobConf.getJar():null
>> 11/04/14 14:50:13 WARN bsp.GroomServer: Error initializing
>> attempt_201104141450_0001_000001_0:
>> java.lang.IllegalArgumentException: Can not create a Path from a null string
>> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:78)
>> at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>> at org.apache.hama.bsp.GroomServer.localizeJob(GroomServer.java:519)
>> at org.apache.hama.bsp.GroomServer.startNewTask(GroomServer.java:451)
>> at org.apache.hama.bsp.GroomServer.access$100(GroomServer.java:70)
>> at
>> org.apache.hama.bsp.GroomServer$DispatchTasksHandler.handle(GroomServer.java:144)
>> at
>> org.apache.hama.bsp.GroomServer$Instructor.run(GroomServer.java:192)
>>
>>
>>>Basically, your miniBSPCluster doesn't work with eclipse JUnit test and
>>>maven build as below:
>>
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: Cleaning up the system directory
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: file:/tmp/hadoop/bsp/system
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: java.lang.NullPointerException
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server Responder: starting
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server listener on 40000: starting
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server handler 0 on 40000: starting
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: Starting RUNNING
>>>11/04/12 20:49:50 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:50 INFO bsp.GroomServer: groom start
>>>11/04/12 20:49:51 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:52 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:53 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:54 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:55 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:56 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>^CDestroying 1 processes
>>>Destroying process..
>>>Destroyed 1 processes
>>>Can you find out the reason why?
>>
>>
>>
>>
>>
>> ChiaHung Lin
>> Department of Information Management
>> National University of Kaohsiung
>> Taiwan
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> http://blog.udanax.org
> http://twitter.com/eddieyoon
>
>
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
>
>
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
>
--
Best Regards, Edward J. Yoon
http://blog.udanax.org
http://twitter.com/eddieyoon
