[
https://issues.apache.org/jira/browse/HAMA-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Jungblut updated HAMA-601:
---------------------------------
Attachment: HAMA-601.patch
Yay it works!
{noformat}
12/09/12 18:01:36 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
12/09/12 18:01:36 WARN bsp.BSPJobClient: No job jar file set. User classes may
not be found. See BSPJob#setJar(String) or check Your jar file.
12/09/12 18:01:36 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
12/09/12 18:01:39 INFO bsp.LocalBSPRunner: Setting up a new barrier for 1 tasks!
12/09/12 18:01:39 INFO pipes.Application: executable:
C:/Users/thomas.jungblut/workspace/hama-trunk/core/BSPRunner.py
12/09/12 18:01:39 INFO pipes.Application: STDOUT:
C:\Users\thomas.jungblut\workspace\hama-trunk\core\tasklogs\job_localrunner_0001\attempt_localrunner_0001_000000_0.log
12/09/12 18:01:39 INFO pipes.Application: STDERR:
C:\Users\thomas.jungblut\workspace\hama-trunk\core\tasklogs\job_localrunner_0001\attempt_localrunner_0001_000000_0.log
12/09/12 18:01:39 INFO pipes.Application: DEBUG: cmd: [bash, -c, 'python'
'C:/Users/thomas.jungblut/workspace/hama-trunk/core/BSPRunner.py'
'HelloWorldBSP' 2>&1 | tee
/cygdrive/c/Users/thomas.jungblut/workspace/hama-trunk/core/tasklogs/job_localrunner_0001/attempt_localrunner_0001_000000_0.log]
12/09/12 18:01:39 INFO pipes.Application: DEBUG: waiting for Client at
0.0.0.0/0.0.0.0:2367
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Starting setup!
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Starting BSP!
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 0
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 1
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 2
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 3
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 4
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 5
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 6
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 7
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 8
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 9
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 10
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 11
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 12
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 13
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Hello from local:0 in superstep 14
12/09/12 18:01:39 INFO pipes.BinaryProtocol: Starting cleanup!
12/09/12 18:01:39 INFO bsp.BSPJobClient: Current supersteps number: 14
12/09/12 18:01:39 INFO bsp.BSPJobClient: The total number of supersteps: 14
12/09/12 18:01:39 INFO bsp.BSPJobClient: Counters: 6
12/09/12 18:01:39 INFO bsp.BSPJobClient:
org.apache.hama.bsp.JobInProgress$JobCounter
12/09/12 18:01:39 INFO bsp.BSPJobClient: LAUNCHED_TASKS=1
12/09/12 18:01:39 INFO bsp.BSPJobClient:
org.apache.hama.bsp.BSPPeerImpl$PeerCounter
12/09/12 18:01:39 INFO bsp.BSPJobClient: SUPERSTEPS=14
12/09/12 18:01:39 INFO bsp.BSPJobClient: SUPERSTEP_SUM=15
12/09/12 18:01:39 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=2
12/09/12 18:01:39 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=30
12/09/12 18:01:39 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=15
{noformat}
Note that this was generated in windows and needs a review on a distributed
mode.
> Hama Streaming
> --------------
>
> Key: HAMA-601
> URL: https://issues.apache.org/jira/browse/HAMA-601
> Project: Hama
> Issue Type: New Feature
> Components: bsp core, messaging
> Affects Versions: 0.6.0
> Reporter: Thomas Jungblut
> Attachments: HAMA-601.patch, streaming_1.patch
>
>
> We can also do a Streaming job to allow other languages to use Hama's BSP API.
> Basically you fork a new process in the BSP method, then set a inputstream
> for the process which it can read very simple.
> Then an outputstream from the childprocess can be read to give it following
> abilities:
> - get a received message
> - send a new message
> - sync
> - read a line from input
> - write to output
> - reset the input to reread
> Those actions must have a constant prefix, for example send a message could
> look like this:
> %SEND_MESSAGE%=this is the message
> or sync:
> $SYNC$=
> The logic behind it is that we can simply split in Java code by "=" and the
> lefthand side is the action and the righthandside is the value of this action.
> Between the peers the messages are Text, which has some overhead but is
> easier to implement and the communication between the BSP task and the forked
> process is based on text/strings anyway.
> This time I do not advise to copy the whole streaming from Hadoop itself.
> However the parts that repacks the jar with needed execution scripts and the
> option handling seems good to reuse.
> The input- and outputstream handling must be written from scratch because we
> want to take actions into account.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
