[
https://issues.apache.org/jira/browse/HAMA-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453162#comment-13453162
]
Thomas Jungblut commented on HAMA-601:
--------------------------------------
The main operations are:
{noformat}
START(0), SET_BSPJOB_CONF(1), SET_INPUT_TYPES(2), RUN_SETUP(3), RUN_BSP(4),
RUN_CLEANUP(5), READ_KEYVALUE(6), WRITE_KEYVALUE(7), GET_MSG(8),
GET_MSG_COUNT(9), SEND_MSG(10), SYNC(11), GET_ALL_PEERNAME(12),
GET_PEERNAME(13), GET_PEER_INDEX(14), GET_PEER_COUNT(15),
GET_SUPERSTEP_COUNT(16), REOPEN_INPUT(17), CLEAR(18), CLOSE(19), ABORT(20),
DONE(21), TASK_DONE(22), REGISTER_COUNTER(23), INCREMENT_COUNTER(24)
{noformat}
These op-codes will be used in streaming like in the description:
%OP_CODE_ID%=_here comes a value if needed, e.G. the message text_
Text will be transfered as byte to the java task and send via byteswritable to
the other peers.
My first version will check a simple python implementation.
> Hama Streaming
> --------------
>
> Key: HAMA-601
> URL: https://issues.apache.org/jira/browse/HAMA-601
> Project: Hama
> Issue Type: New Feature
> Components: bsp core, messaging
> Affects Versions: 0.6.0
> Reporter: Thomas Jungblut
>
> We can also do a Streaming job to allow other languages to use Hama's BSP API.
> Basically you fork a new process in the BSP method, then set a inputstream
> for the process which it can read very simple.
> Then an outputstream from the childprocess can be read to give it following
> abilities:
> - get a received message
> - send a new message
> - sync
> - read a line from input
> - write to output
> - reset the input to reread
> Those actions must have a constant prefix, for example send a message could
> look like this:
> %SEND_MESSAGE%=this is the message
> or sync:
> $SYNC$=
> The logic behind it is that we can simply split in Java code by "=" and the
> lefthand side is the action and the righthandside is the value of this action.
> Between the peers the messages are Text, which has some overhead but is
> easier to implement and the communication between the BSP task and the forked
> process is based on text/strings anyway.
> This time I do not advise to copy the whole streaming from Hadoop itself.
> However the parts that repacks the jar with needed execution scripts and the
> option handling seems good to reuse.
> The input- and outputstream handling must be written from scratch because we
> want to take actions into account.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
