Oh, .... I just noticed that why you asked that question today. On second thoughts, converter is not good idea. Partitioner is just a partitioner, Vertex should be parsed at GraphJobRunner.
Leave it to me. On Sun, Dec 16, 2012 at 9:57 AM, Edward J. Yoon <[email protected]> wrote: >>> implementation. Now my question is, if the input to be read from HBase, do >>> we write every record from HBase to sequential file format? I am a little >>> confused here. > > In NoSQLs cases, they have own partitions based on ranges of key. If > the key is vertex and PartitioningRunner can be skipped, user can use > default partitions of table without re-writing splits. > > On Sun, Dec 16, 2012 at 8:50 AM, Edward J. Yoon <[email protected]> wrote: >> +1 Looks great! >> >> On Sun, Dec 16, 2012 at 7:20 AM, Suraj Menon <[email protected]> wrote: >>> +1 for hama-io, +0 for hama-commons. Can you look at the patch for HAMA-700 >>> and let me know if you have any issues going forward in that direction. >>> Basically it includes a raw format to intermediate binary format >>> conversion, that could be overridden as shown in VertexInputReader >>> implementation. Now my question is, if the input to be read from HBase, do >>> we write every record from HBase to sequential file format? I am a little >>> confused here. >>> >>> Thanks, >>> Suraj >>> >>> On Fri, Dec 14, 2012 at 2:19 AM, Tommaso Teofili >>> <[email protected]>wrote: >>> >>>> thinking about it a bit further, maybe a separate module for IO related >>>> stuff may be worth, so hama-io instead of hama-commons. >>>> Whatever it is I'd like to avoid duplicated code. >>>> Tommaso >>>> >>>> >>>> 2012/12/13 Edward J. Yoon <[email protected]> >>>> >>>> > Otherwise, we have to copy VertexInputReader class to core module or >>>> > add own partitioner to graph package. >>>> > >>>> > In fact, I don't want to put any graph-related code to core module. >>>> > But if we want to support VertexInputReader, I think this is best. >>>> > >>>> > On Fri, Dec 14, 2012 at 4:22 AM, Apurv Verma <[email protected]> wrote: >>>> > > +0 I am not against but I think that would be an overkill for now. >>>> > > >>>> > > -- >>>> > > Regards, >>>> > > Apurv Verma >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > On Thu, Dec 13, 2012 at 12:32 PM, Tommaso Teofili >>>> > > <[email protected]> wrote: >>>> > >> that sounds good to me, +1 for a hama-commons module. >>>> > >> >>>> > >> Tommaso >>>> > >> >>>> > >> >>>> > >> 2012/12/13 Edward J. Yoon <[email protected]> >>>> > >> >>>> > >>> Hi, >>>> > >>> >>>> > >>> I propose to create common module to share common classes among all >>>> > >>> modules. >>>> > >>> >>>> > >>> At the moment, the main reason is to support VertexInputReader. As I >>>> > >>> mentioned here[1], there's no way to integrate partitioner without >>>> > >>> merging graph to core or adding graph own partitioner. If we create >>>> > >>> common module, it'll all be clear. I'm not maven and java expert. If >>>> > >>> I'm wrong, please correct me. >>>> > >>> >>>> > >>> 1. >>>> > >>> >>>> > >>>> https://issues.apache.org/jira/browse/HAMA-531?focusedCommentId=13483009&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13483009 >>>> > >>> >>>> > >>> -- >>>> > >>> Best Regards, Edward J. Yoon >>>> > >>> @eddieyoon >>>> > >>> >>>> > >>>> > >>>> > >>>> > -- >>>> > Best Regards, Edward J. Yoon >>>> > @eddieyoon >>>> > >>>> >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon -- Best Regards, Edward J. Yoon @eddieyoon
