[jira] [Commented] (GIRAPH-17) Giraph doesn't give up properly after the maximum connect attempts to ZooKeeper
[ https://issues.apache.org/jira/browse/GIRAPH-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093786#comment-13093786 ] Owen O'Malley commented on GIRAPH-17: - Can you explain the fix? It looks like cleanup and making the limit configurable. > Giraph doesn't give up properly after the maximum connect attempts to > ZooKeeper > --- > > Key: GIRAPH-17 > URL: https://issues.apache.org/jira/browse/GIRAPH-17 > Project: Giraph > Issue Type: Bug >Reporter: Avery Ching >Assignee: Avery Ching >Priority: Minor > Attachments: ZooKeeperManager.java.diff > > > This produces incorrect and strange behavior. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-17) Giraph doesn't give up properly after the maximum connect attempts to ZooKeeper
[
https://issues.apache.org/jira/browse/GIRAPH-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093808#comment-13093808
]
Avery Ching commented on GIRAPH-17:
---
Sure. The main part of this fix is
-if (connectAttempts == 5) {
+if (connectAttempts == maxConnectAttempts) {
Basically this condition should be hit if the max connect attempts was tried,
but never was because they because maxConnectAttempts is now 10 and became out
of sync at some point (maxConnectAttempts probably used to be 5).
The limit is stil not configurable, we can address that in a later issue.
> Giraph doesn't give up properly after the maximum connect attempts to
> ZooKeeper
> ---
>
> Key: GIRAPH-17
> URL: https://issues.apache.org/jira/browse/GIRAPH-17
> Project: Giraph
> Issue Type: Bug
>Reporter: Avery Ching
>Assignee: Avery Ching
>Priority: Minor
> Attachments: ZooKeeperManager.java.diff
>
>
> This produces incorrect and strange behavior.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-17) Giraph doesn't give up properly after the maximum connect attempts to ZooKeeper
[ https://issues.apache.org/jira/browse/GIRAPH-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093820#comment-13093820 ] Owen O'Malley commented on GIRAPH-17: - Ok, thanks. In that case, +1. > Giraph doesn't give up properly after the maximum connect attempts to > ZooKeeper > --- > > Key: GIRAPH-17 > URL: https://issues.apache.org/jira/browse/GIRAPH-17 > Project: Giraph > Issue Type: Bug >Reporter: Avery Ching >Assignee: Avery Ching >Priority: Minor > Attachments: ZooKeeperManager.java.diff > > > This produces incorrect and strange behavior. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093828#comment-13093828 ] Joe Xie commented on GIRAPH-14: --- Great job Avery! I can compile on my side too and it works with examples on the cluster. I still got a few testing errors during mvn package & test, let me know if you have any updates. Joe Xie Tests in error: testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer) testBspCheckpoint(org.apache.giraph.TestManualCheckpoint) testSingleFault(org.apache.giraph.TestAutoCheckpoint) testInstantiateVertex(org.apache.giraph.TestBspBasic) testLocalJobRunnerConfig(org.apache.giraph.TestBspBasic) testBspFail(org.apache.giraph.TestBspBasic) testBspSuperStep(org.apache.giraph.TestBspBasic) testBspMsg(org.apache.giraph.TestBspBasic) testEmptyVertexInputFormat(org.apache.giraph.TestBspBasic) testBspCombiner(org.apache.giraph.TestBspBasic) testBspPageRank(org.apache.giraph.TestBspBasic) testBspShortestPaths(org.apache.giraph.TestBspBasic) testMutateGraph(org.apache.giraph.TestMutateGraphVertex) testContinue(org.apache.giraph.TestJsonBase64Format) testMatchingType(org.apache.giraph.TestVertexTypes) testDerivedMatchingType(org.apache.giraph.TestVertexTypes) testDerivedInputFormatType(org.apache.giraph.TestVertexTypes) testMismatchingVertex(org.apache.giraph.TestVertexTypes) testMismatchingCombiner(org.apache.giraph.TestVertexTypes) testJsonBase64FormatType(org.apache.giraph.TestVertexTypes) > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093831#comment-13093831 ] Joe Xie commented on GIRAPH-14: --- Thank you Avery! maybe my test file in the trunk is obsolete, let me know whether you can reproduce it. > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-17) Giraph doesn't give up properly after the maximum connect attempts to ZooKeeper
[ https://issues.apache.org/jira/browse/GIRAPH-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093861#comment-13093861 ] Avery Ching commented on GIRAPH-17: --- Thanks for taking a look. Committed. > Giraph doesn't give up properly after the maximum connect attempts to > ZooKeeper > --- > > Key: GIRAPH-17 > URL: https://issues.apache.org/jira/browse/GIRAPH-17 > Project: Giraph > Issue Type: Bug >Reporter: Avery Ching >Assignee: Avery Ching >Priority: Minor > Attachments: ZooKeeperManager.java.diff > > > This produces incorrect and strange behavior. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093879#comment-13093879 ] Avery Ching commented on GIRAPH-14: --- It's good to hear that you can run it on your cluster. As far as the unittests, that is strange. I was able to repeat the same issues and will look into a fix. > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Reviewboard for code reviews
Hi Avery, yes you should file INFRA ticket to add Giraph as Groups in reviews board. I filed tickets to create one for Kafka and Gora. - Henry On Mon, Aug 29, 2011 at 10:13 PM, Avery Ching wrote: > https://blogs.apache.org/infra/entry/reviewboard_instance_running_at_the > > I'll file an INFRA ticket. > > Thanks, > > Avery > > On Aug 29, 2011, at 10:07 PM, Hyunsik Choi wrote: > > Looks possible. Some incubator project (e.g., Kafka) already has a > reviewboard group. > > Best regards, > -- > Hyunsik Choi > > > > On Tue, Aug 30, 2011 at 1:48 PM, Avery Ching > mailto:ach...@yahoo-inc.com>> wrote: > Anyone know if we have reviewboard access? > > Thanks, > > Avery > > >
Re: Reviewboard for code reviews
Thanks Henry. I have filed issue https://issues.apache.org/jira/browse/INFRA-3892 to get reviewboard access. Avery On Aug 30, 2011, at 11:35 AM, Henry Saputra wrote: Hi Avery, yes you should file INFRA ticket to add Giraph as Groups in reviews board. I filed tickets to create one for Kafka and Gora. - Henry On Mon, Aug 29, 2011 at 10:13 PM, Avery Ching mailto:ach...@yahoo-inc.com>> wrote: https://blogs.apache.org/infra/entry/reviewboard_instance_running_at_the I'll file an INFRA ticket. Thanks, Avery On Aug 29, 2011, at 10:07 PM, Hyunsik Choi wrote: Looks possible. Some incubator project (e.g., Kafka) already has a reviewboard group. Best regards, -- Hyunsik Choi On Tue, Aug 30, 2011 at 1:48 PM, Avery Ching mailto:ach...@yahoo-inc.com>> wrote: Anyone know if we have reviewboard access? Thanks, Avery
Re: Reviewboard for code reviews
We've just gone around on this one for Kafka and, if reviewboard is provided, it would be good to keep it as an optional part of the process. I've had very negative experiences with it, both in Hadoop and Hive. If one would like to do a reviewboard review, that's great - but for those who don't, standard bullet points should suffice. -jakob On Tue, Aug 30, 2011 at 11:38 AM, Avery Ching wrote: > Thanks Henry. I have filed issue > > https://issues.apache.org/jira/browse/INFRA-3892 > > to get reviewboard access. > > Avery > > On Aug 30, 2011, at 11:35 AM, Henry Saputra wrote: > > Hi Avery, yes you should file INFRA ticket to add Giraph as Groups in > reviews board. > > I filed tickets to create one for Kafka and Gora. > > - Henry > > On Mon, Aug 29, 2011 at 10:13 PM, Avery Ching > mailto:ach...@yahoo-inc.com>> wrote: > https://blogs.apache.org/infra/entry/reviewboard_instance_running_at_the > > I'll file an INFRA ticket. > > Thanks, > > Avery > > On Aug 29, 2011, at 10:07 PM, Hyunsik Choi wrote: > > Looks possible. Some incubator project (e.g., Kafka) already has a > reviewboard group. > > Best regards, > -- > Hyunsik Choi > > > > On Tue, Aug 30, 2011 at 1:48 PM, Avery Ching > mailto:ach...@yahoo-inc.com>> wrote: > Anyone know if we have reviewboard access? > > Thanks, > > Avery > > > > >
Re: Reviewboard for code reviews
+1 It should just optional to help review not required. - Henry On Tue, Aug 30, 2011 at 11:48 AM, Jakob Homan wrote: > We've just gone around on this one for Kafka and, if reviewboard is > provided, it would be good to keep it as an optional part of the > process. I've had very negative experiences with it, both in Hadoop > and Hive. If one would like to do a reviewboard review, that's great > - but for those who don't, standard bullet points should suffice. > -jakob > > > > On Tue, Aug 30, 2011 at 11:38 AM, Avery Ching wrote: >> Thanks Henry. I have filed issue >> >> https://issues.apache.org/jira/browse/INFRA-3892 >> >> to get reviewboard access. >> >> Avery >> >> On Aug 30, 2011, at 11:35 AM, Henry Saputra wrote: >> >> Hi Avery, yes you should file INFRA ticket to add Giraph as Groups in >> reviews board. >> >> I filed tickets to create one for Kafka and Gora. >> >> - Henry >> >> On Mon, Aug 29, 2011 at 10:13 PM, Avery Ching >> mailto:ach...@yahoo-inc.com>> wrote: >> https://blogs.apache.org/infra/entry/reviewboard_instance_running_at_the >> >> I'll file an INFRA ticket. >> >> Thanks, >> >> Avery >> >> On Aug 29, 2011, at 10:07 PM, Hyunsik Choi wrote: >> >> Looks possible. Some incubator project (e.g., Kafka) already has a >> reviewboard group. >> >> Best regards, >> -- >> Hyunsik Choi >> >> >> >> On Tue, Aug 30, 2011 at 1:48 PM, Avery Ching >> mailto:ach...@yahoo-inc.com>> wrote: >> Anyone know if we have reviewboard access? >> >> Thanks, >> >> Avery >> >> >> >> >> >
Re: Reviewboard for code reviews
Argh I meant "It should just an option to help review and should not be required for patches." - Henry On Tue, Aug 30, 2011 at 11:51 AM, Henry Saputra wrote: > +1 > > It should just optional to help review not required. > > - Henry > > On Tue, Aug 30, 2011 at 11:48 AM, Jakob Homan wrote: >> We've just gone around on this one for Kafka and, if reviewboard is >> provided, it would be good to keep it as an optional part of the >> process. I've had very negative experiences with it, both in Hadoop >> and Hive. If one would like to do a reviewboard review, that's great >> - but for those who don't, standard bullet points should suffice. >> -jakob >> >> >> >> On Tue, Aug 30, 2011 at 11:38 AM, Avery Ching wrote: >>> Thanks Henry. I have filed issue >>> >>> https://issues.apache.org/jira/browse/INFRA-3892 >>> >>> to get reviewboard access. >>> >>> Avery >>> >>> On Aug 30, 2011, at 11:35 AM, Henry Saputra wrote: >>> >>> Hi Avery, yes you should file INFRA ticket to add Giraph as Groups in >>> reviews board. >>> >>> I filed tickets to create one for Kafka and Gora. >>> >>> - Henry >>> >>> On Mon, Aug 29, 2011 at 10:13 PM, Avery Ching >>> mailto:ach...@yahoo-inc.com>> wrote: >>> https://blogs.apache.org/infra/entry/reviewboard_instance_running_at_the >>> >>> I'll file an INFRA ticket. >>> >>> Thanks, >>> >>> Avery >>> >>> On Aug 29, 2011, at 10:07 PM, Hyunsik Choi wrote: >>> >>> Looks possible. Some incubator project (e.g., Kafka) already has a >>> reviewboard group. >>> >>> Best regards, >>> -- >>> Hyunsik Choi >>> >>> >>> >>> On Tue, Aug 30, 2011 at 1:48 PM, Avery Ching >>> mailto:ach...@yahoo-inc.com>> wrote: >>> Anyone know if we have reviewboard access? >>> >>> Thanks, >>> >>> Avery >>> >>> >>> >>> >>> >> >
Re: Reviewboard for code reviews
Okay, let's make it optional for now. For me, it definitely helps to visualize the changes better. Also, I think the feedback tool is pretty good. Avery On Aug 30, 2011, at 11:52 AM, Henry Saputra wrote: > Argh I meant "It should just an option to help review and should not > be required for patches." > > - Henry > > On Tue, Aug 30, 2011 at 11:51 AM, Henry Saputra > wrote: >> +1 >> >> It should just optional to help review not required. >> >> - Henry >> >> On Tue, Aug 30, 2011 at 11:48 AM, Jakob Homan wrote: >>> We've just gone around on this one for Kafka and, if reviewboard is >>> provided, it would be good to keep it as an optional part of the >>> process. I've had very negative experiences with it, both in Hadoop >>> and Hive. If one would like to do a reviewboard review, that's great >>> - but for those who don't, standard bullet points should suffice. >>> -jakob >>> >>> >>> >>> On Tue, Aug 30, 2011 at 11:38 AM, Avery Ching wrote: Thanks Henry. I have filed issue https://issues.apache.org/jira/browse/INFRA-3892 to get reviewboard access. Avery On Aug 30, 2011, at 11:35 AM, Henry Saputra wrote: Hi Avery, yes you should file INFRA ticket to add Giraph as Groups in reviews board. I filed tickets to create one for Kafka and Gora. - Henry On Mon, Aug 29, 2011 at 10:13 PM, Avery Ching mailto:ach...@yahoo-inc.com>> wrote: https://blogs.apache.org/infra/entry/reviewboard_instance_running_at_the I'll file an INFRA ticket. Thanks, Avery On Aug 29, 2011, at 10:07 PM, Hyunsik Choi wrote: Looks possible. Some incubator project (e.g., Kafka) already has a reviewboard group. Best regards, -- Hyunsik Choi On Tue, Aug 30, 2011 at 1:48 PM, Avery Ching mailto:ach...@yahoo-inc.com>> wrote: Anyone know if we have reviewboard access? Thanks, Avery >>> >>
[jira] [Commented] (GIRAPH-4) New project logo
[ https://issues.apache.org/jira/browse/GIRAPH-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094128#comment-13094128 ] Kenin Coloma commented on GIRAPH-4: --- What a great idea! > New project logo > > > Key: GIRAPH-4 > URL: https://issues.apache.org/jira/browse/GIRAPH-4 > Project: Giraph > Issue Type: New Feature >Reporter: Jakob Homan > > Now for the hard part: the project logo. We should create one and add it to > the website once done. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-4) New project logo
[ https://issues.apache.org/jira/browse/GIRAPH-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094134#comment-13094134 ] Avery Ching commented on GIRAPH-4: -- Yes, it would certainly be nice to have a real logo. Do you want to give it a shot? > New project logo > > > Key: GIRAPH-4 > URL: https://issues.apache.org/jira/browse/GIRAPH-4 > Project: Giraph > Issue Type: New Feature >Reporter: Jakob Homan > > Now for the hard part: the project logo. We should create one and add it to > the website once done. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-4) New project logo
[ https://issues.apache.org/jira/browse/GIRAPH-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094137#comment-13094137 ] Kenin Coloma commented on GIRAPH-4: --- Sure, I'll get on it. > New project logo > > > Key: GIRAPH-4 > URL: https://issues.apache.org/jira/browse/GIRAPH-4 > Project: Giraph > Issue Type: New Feature >Reporter: Jakob Homan > > Now for the hard part: the project logo. We should create one and add it to > the website once done. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (GIRAPH-4) New project logo
[ https://issues.apache.org/jira/browse/GIRAPH-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan reassigned GIRAPH-4: Assignee: Kenin Coloma > New project logo > > > Key: GIRAPH-4 > URL: https://issues.apache.org/jira/browse/GIRAPH-4 > Project: Giraph > Issue Type: New Feature >Reporter: Jakob Homan >Assignee: Kenin Coloma > > Now for the hard part: the project logo. We should create one and add it to > the website once done. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Ching updated GIRAPH-14: -- Attachment: facebook2.txt Looks like I needed to change the groupId so that the right dependencies are pulled in for hadoop. Please try this one out. The unittests all passed for me. (i.e. mvn -Dhadoop=facebook -Dhadoop.jar.path=/Users/aching/Desktop/hadoop-0.20.1-core.jar package) > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt, facebook2.txt > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094209#comment-13094209 ] Joe Xie commented on GIRAPH-14: --- thank you Avery! It passed the unittest for me too. > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt, facebook2.txt > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094216#comment-13094216 ] Avery Ching commented on GIRAPH-14: --- Great to hear it! When one of the committers gets a chance to review, I can commit. > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt, facebook2.txt > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (GIRAPH-18) Refactor BspServiceWorker::loadVertices()
Refactor BspServiceWorker::loadVertices() - Key: GIRAPH-18 URL: https://issues.apache.org/jira/browse/GIRAPH-18 Project: Giraph Issue Type: Improvement Reporter: Jakob Homan Assignee: Jakob Homan Currently BspServiceWorker::loadVertices() is more than 200 lines and convoluted. I found it difficult to grok while debugging today. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-18) Refactor BspServiceWorker::loadVertices()
[ https://issues.apache.org/jira/browse/GIRAPH-18?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated GIRAPH-18: -- Attachment: GIRAPH-18.patch Patch: * Refactors BspServiceWorker::loadVertices() into several smaller functions, each responsible for a subsection of the logic. This includes: ** gathering the input split from ZK ** actually reading the vertices from the input split ** generating the vertex ranges ** calculating the range size ** building the range stat instance that's returned. * Replaces the single arraylist that's created with one that's generated for each inputsplitpath. This is a performance win because, with spikey or dense graphs the backing array of the arraylist was resized but never shrunk, putting pressure on the heap. These new per-call arraylists will come and go quickly, making them eligible for gc. * Specifies the exact size of the list used in the maxIndexStatMap to 2, saving some space. This adds up! Post-refactor these methods will be much easier to unit test since they all have defined inputs and outputs. Patch follows current style and passes unit tests. > Refactor BspServiceWorker::loadVertices() > - > > Key: GIRAPH-18 > URL: https://issues.apache.org/jira/browse/GIRAPH-18 > Project: Giraph > Issue Type: Improvement >Reporter: Jakob Homan >Assignee: Jakob Homan > Attachments: GIRAPH-18.patch > > > Currently BspServiceWorker::loadVertices() is more than 200 lines and > convoluted. I found it difficult to grok while debugging today. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094271#comment-13094271 ] Jakob Homan commented on GIRAPH-14: --- I'm not up to date on FB's distribution. It's available to the public? If we're going to support this, should instructions be given in the README, as there are for the non-secure build option? What's the long term story for these API support #ifdefs? At the moment it's a clever solution to a vexing problem, but longer term it would be good to have a solution that doesn't leave code as comments. > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt, facebook2.txt > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094305#comment-13094305 ] Avery Ching commented on GIRAPH-14: --- In theory, I believe that Facebook's distro is online (https://github.com/facebook/hadoop-20-warehouse). The long term story is to factor out the parts into modules and then compile them based on the user profile. Then we don't have to "munge" anything anymore. At least that's what I've thought of for now. I'm open to better solutions. Pre-processing will get unmaintainable if we have to support every version of Hadoop. That being said, we should support the big customers of Giraph and that likely includes Facebook as well. I'll add instructions to the README and submit a new patch. > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt, facebook2.txt > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Ching updated GIRAPH-14: -- Attachment: facebook3.patch Updated with README instructions for building with the Facebook Hadoop release. > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt, facebook2.txt, facebook3.patch > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-14) Support for the Facebook Hadoop branch
[ https://issues.apache.org/jira/browse/GIRAPH-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094321#comment-13094321 ] Jakob Homan commented on GIRAPH-14: --- Applied the patch and verified everything still works as expected on the regular build. +1. > Support for the Facebook Hadoop branch > -- > > Key: GIRAPH-14 > URL: https://issues.apache.org/jira/browse/GIRAPH-14 > Project: Giraph > Issue Type: New Feature >Reporter: Avery Ching >Assignee: Avery Ching > Attachments: facebook.txt, facebook2.txt, facebook3.patch > > > I've been working with Joe Xie on support to get Giraph running on the > Facebook Hadoop branch. He verified today that the examples worked on their > cluster. I need to clean up my changes a little, but otherwise, will submit > a cleaned up diff. As a side note, does anyone know how we can get Hudson > support for Giraph? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-18) Refactor BspServiceWorker::loadVertices()
[ https://issues.apache.org/jira/browse/GIRAPH-18?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094330#comment-13094330 ] Avery Ching commented on GIRAPH-18: --- This isn't the only area that needs refactoring =). Let me take a deeper look tomorrow, but initially looks better. > Refactor BspServiceWorker::loadVertices() > - > > Key: GIRAPH-18 > URL: https://issues.apache.org/jira/browse/GIRAPH-18 > Project: Giraph > Issue Type: Improvement >Reporter: Jakob Homan >Assignee: Jakob Homan > Attachments: GIRAPH-18.patch > > > Currently BspServiceWorker::loadVertices() is more than 200 lines and > convoluted. I found it difficult to grok while debugging today. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
