[jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13141965#comment-13141965 ] Avery Ching commented on GIRAPH-36: --- +1. Jake, just re-ran with your new patch. Passed both local and MR unittests. I noticed some warnings and tried to address them (some aren't your fault, from an earlier patch). I'm uploading a diff of your diff. Take a look and if it's fine I'll commit on your behalf. I didn't address the warnings in src/test/ava/org/apache/giraph/lib with mockito since I don't understand it as well as I should, but we should knock out those warnings when we can. Ensure that subclassing BasicVertex is possible by user apps Key: GIRAPH-36 URL: https://issues.apache.org/jira/browse/GIRAPH-36 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.70.0 Reporter: Jake Mannix Assignee: Jake Mannix Priority: Blocker Fix For: 0.70.0 Attachments: GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff Original assumptions in Giraph were that all users would subclass Vertex (which extended MutableVertex extended BasicVertex). Classes which wish to have application specific data structures (ie. not a TreeMapI, EdgeI,E) may need to extend either MutableVertex or BasicVertex. Unfortunately VertexRange extends ArrayListVertex, and there are other places where the assumption is that vertex classes are either Vertex, or at least MutableVertex. Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Ching updated GIRAPH-36: -- Attachment: GIRAPH-36.diff.warnings Revised version of Jake's patch, addressing the warnings in src/main. Some minor cleanup. Ensure that subclassing BasicVertex is possible by user apps Key: GIRAPH-36 URL: https://issues.apache.org/jira/browse/GIRAPH-36 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.70.0 Reporter: Jake Mannix Assignee: Jake Mannix Priority: Blocker Fix For: 0.70.0 Attachments: GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff.warnings Original assumptions in Giraph were that all users would subclass Vertex (which extended MutableVertex extended BasicVertex). Classes which wish to have application specific data structures (ie. not a TreeMapI, EdgeI,E) may need to extend either MutableVertex or BasicVertex. Unfortunately VertexRange extends ArrayListVertex, and there are other places where the assumption is that vertex classes are either Vertex, or at least MutableVertex. Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13141994#comment-13141994 ] Jake Mannix commented on GIRAPH-36: --- Excellent, looks good to me. +1 for committing we can iterate from here is the main point. The sooner this gets in, the fewer other patches out there get broken. ;) Ensure that subclassing BasicVertex is possible by user apps Key: GIRAPH-36 URL: https://issues.apache.org/jira/browse/GIRAPH-36 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.70.0 Reporter: Jake Mannix Assignee: Jake Mannix Priority: Blocker Fix For: 0.70.0 Attachments: GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff.warnings Original assumptions in Giraph were that all users would subclass Vertex (which extended MutableVertex extended BasicVertex). Classes which wish to have application specific data structures (ie. not a TreeMapI, EdgeI,E) may need to extend either MutableVertex or BasicVertex. Unfortunately VertexRange extends ArrayListVertex, and there are other places where the assumption is that vertex classes are either Vertex, or at least MutableVertex. Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
Great, so I can I base GIRAPH-10 and GIRAPH-47 after this one! About my use case: I'm implementing a batch path query engine where the queries are XPath-like queries. Each query is a stack of conditions, each element assigned to a depth of the path traversal. A message is a stack. Not each vertex is a starting vertex for the path traversals, so it could be nice if at the first superstep the starting vertices could already have the messages in their inbox. This way the message processing (and path traversal processing with it) could be fully transparent (vertices just have to iterate throw messages without caring about parsing, start vertices etc etc). Currently, at first superstep, each vertex has to get che query from the conf, parse it into a stack and discard it afterwards, as soon as it discovers it is not a starting vertex for a path traversal. Outside, I know who's a starting vertex and could just set the inbox for those vertices accordingly. Hope this helps, Claudio On Wed, Nov 2, 2011 at 9:14 AM, Jake Mannix (Commented) (JIRA) j...@apache.org wrote: [ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13141994#comment-13141994 ] Jake Mannix commented on GIRAPH-36: --- Excellent, looks good to me. +1 for committing we can iterate from here is the main point. The sooner this gets in, the fewer other patches out there get broken. ;) Ensure that subclassing BasicVertex is possible by user apps Key: GIRAPH-36 URL: https://issues.apache.org/jira/browse/GIRAPH-36 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.70.0 Reporter: Jake Mannix Assignee: Jake Mannix Priority: Blocker Fix For: 0.70.0 Attachments: GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff.warnings Original assumptions in Giraph were that all users would subclass Vertex (which extended MutableVertex extended BasicVertex). Classes which wish to have application specific data structures (ie. not a TreeMapI, EdgeI,E) may need to extend either MutableVertex or BasicVertex. Unfortunately VertexRange extends ArrayListVertex, and there are other places where the assumption is that vertex classes are either Vertex, or at least MutableVertex. Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira -- Claudio Martella claudio.marte...@gmail.com
[jira] [Created] (GIRAPH-71) SequenceFileVertexInputFormat missing license header; rat fails
SequenceFileVertexInputFormat missing license header; rat fails --- Key: GIRAPH-71 URL: https://issues.apache.org/jira/browse/GIRAPH-71 Project: Giraph Issue Type: Bug Components: lib Affects Versions: 0.70.0 Reporter: Jakob Homan Assignee: Jakob Homan Fix For: 0.70.0 SequenceFileVertexInputFormat needs the Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-71) SequenceFileVertexInputFormat missing license header; rat fails
[ https://issues.apache.org/jira/browse/GIRAPH-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated GIRAPH-71: -- Attachment: GIRAPH-71.patch quick patch. SequenceFileVertexInputFormat missing license header; rat fails --- Key: GIRAPH-71 URL: https://issues.apache.org/jira/browse/GIRAPH-71 Project: Giraph Issue Type: Bug Components: lib Affects Versions: 0.70.0 Reporter: Jakob Homan Assignee: Jakob Homan Fix For: 0.70.0 Attachments: GIRAPH-71.patch SequenceFileVertexInputFormat needs the Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-71) SequenceFileVertexInputFormat missing license header; rat fails
[ https://issues.apache.org/jira/browse/GIRAPH-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142569#comment-13142569 ] Jake Mannix commented on GIRAPH-71: --- +1 from me, not sure why it passed for me, I was sure I'd run the full mvn clean install which runs rat too, but then when I tried re-cloning again this morning, it failed. Thanks Jakob! SequenceFileVertexInputFormat missing license header; rat fails --- Key: GIRAPH-71 URL: https://issues.apache.org/jira/browse/GIRAPH-71 Project: Giraph Issue Type: Bug Components: lib Affects Versions: 0.70.0 Reporter: Jakob Homan Assignee: Jakob Homan Fix For: 0.70.0 Attachments: GIRAPH-71.patch SequenceFileVertexInputFormat needs the Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[
https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142573#comment-13142573
]
Jakob Homan commented on GIRAPH-36:
---
Great work, Jake. One thing I notice while reading through Vertex is muddling
of the state the Vertex is responsible for on a per-application basis and the
state Giraph manages for the vertex. I think we may being ill-served by
inheritance here and should instead rely on composition to hold this state.
I'm thinking that messages in/out, edge state and mutation, facilities for
sending messages, current superstep, etc. Would it be better in the long term
to move these out to a context object (ala MR)? This would simplify Vertex
significantly, make it much easier to test (by mocking out the dependency) and
future proof the evolution of Vertex as there will be fewer moving parts to
keep track of.
Does changing compute and {pre|post}{Superstep|Application} took their external
state as parameters with sound like a good approach to explore?
Ensure that subclassing BasicVertex is possible by user apps
Key: GIRAPH-36
URL: https://issues.apache.org/jira/browse/GIRAPH-36
Project: Giraph
Issue Type: Improvement
Components: graph
Affects Versions: 0.70.0
Reporter: Jake Mannix
Assignee: Jake Mannix
Priority: Blocker
Fix For: 0.70.0
Attachments: GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff,
GIRAPH-36.diff.warnings
Original assumptions in Giraph were that all users would subclass Vertex
(which extended MutableVertex extended BasicVertex). Classes which wish to
have application specific data structures (ie. not a TreeMapI, EdgeI,E)
may need to extend either MutableVertex or BasicVertex. Unfortunately
VertexRange extends ArrayListVertex, and there are other places where the
assumption is that vertex classes are either Vertex, or at least
MutableVertex.
Let's make sure the internal APIs allow for BasicVertex to be the base class.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-71) SequenceFileVertexInputFormat missing license header; rat fails
[ https://issues.apache.org/jira/browse/GIRAPH-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-71. --- Resolution: Fixed I've committed this. Resolving as fixed. SequenceFileVertexInputFormat missing license header; rat fails --- Key: GIRAPH-71 URL: https://issues.apache.org/jira/browse/GIRAPH-71 Project: Giraph Issue Type: Bug Components: lib Affects Versions: 0.70.0 Reporter: Jakob Homan Assignee: Jakob Homan Fix For: 0.70.0 Attachments: GIRAPH-71.patch SequenceFileVertexInputFormat needs the Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-71) SequenceFileVertexInputFormat missing license header; rat fails
[ https://issues.apache.org/jira/browse/GIRAPH-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142578#comment-13142578 ] Hudson commented on GIRAPH-71: -- Integrated in Giraph-trunk-Commit #23 (See [https://builds.apache.org/job/Giraph-trunk-Commit/23/]) GIRAPH-71. SequenceFileVertexInputFormat missing license header; rat fails. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1196814 Files : * /incubator/giraph/trunk/CHANGELOG * /incubator/giraph/trunk/src/main/java/org/apache/giraph/lib/SequenceFileVertexInputFormat.java SequenceFileVertexInputFormat missing license header; rat fails --- Key: GIRAPH-71 URL: https://issues.apache.org/jira/browse/GIRAPH-71 Project: Giraph Issue Type: Bug Components: lib Affects Versions: 0.70.0 Reporter: Jakob Homan Assignee: Jakob Homan Fix For: 0.70.0 Attachments: GIRAPH-71.patch SequenceFileVertexInputFormat needs the Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142581#comment-13142581 ] Claudio Martella commented on GIRAPH-36: Hi Jakob, as a matter of fact, I believe, part of your proposal is addressed by GIRAPH-47. Now that Jake has committed this work, I'll sync and complete that patch to the current trunk (I was out of office for conference the whole last week). Can GIRAPH-47 be a start for what you have in mind? Ensure that subclassing BasicVertex is possible by user apps Key: GIRAPH-36 URL: https://issues.apache.org/jira/browse/GIRAPH-36 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.70.0 Reporter: Jake Mannix Assignee: Jake Mannix Priority: Blocker Fix For: 0.70.0 Attachments: GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff.warnings Original assumptions in Giraph were that all users would subclass Vertex (which extended MutableVertex extended BasicVertex). Classes which wish to have application specific data structures (ie. not a TreeMapI, EdgeI,E) may need to extend either MutableVertex or BasicVertex. Unfortunately VertexRange extends ArrayListVertex, and there are other places where the assumption is that vertex classes are either Vertex, or at least MutableVertex. Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-64) Create VertexRunner to make it easier to run users' computations
[
https://issues.apache.org/jira/browse/GIRAPH-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142593#comment-13142593
]
Jake Mannix commented on GIRAPH-64:
---
I'd be happy to review this too (I wrote the original form of bin/mahout's
MahoutDriver program, although it's gone and evolved over time from there) is
there a way to automagically have ReviewBoard create a review for this patch,
so we can see it in a GUI? My brain has atrophied greatly as IDE's have
advanced. ;P
Create VertexRunner to make it easier to run users' computations
Key: GIRAPH-64
URL: https://issues.apache.org/jira/browse/GIRAPH-64
Project: Giraph
Issue Type: New Feature
Reporter: Jakob Homan
Assignee: Jakob Homan
Attachments: GIRAPH-64.patch
Currently, if a user wants to implement a Giraph algorithm by extending
{{Vertex}} they must also write all the boilerplate around the {{Tool}}
interface and bundle it with the Giraph jar (or get Giraph on the classpath
and playing nice with the implementation). For example, what is included in
the PageRankBenchmark and what Kohei has done:
https://github.com/smly/java-Giraph-LabelPropagation It would be better if
we had perhaps a Vertex implementation to be subclassed that already had all
the standard Tooling included such that all one had to run would be (assuming
the Giraph jar was already on the classpath):
{noformat}hadoop jar my-awesome-vertex.jar my.awesome.vertex -i jazz_input -o
jazz_output -if org.apache.giraph.lib.in.text.adjacency-list.LongDoubleDouble
-of org.apache.giraph.lib.out.text.adjacency-list.LongDoubleDouble{noformat}
This wouldn't work with every algorithm, but would be useful in a large
number of cases.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-64) Create VertexRunner to make it easier to run users' computations
[
https://issues.apache.org/jira/browse/GIRAPH-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142608#comment-13142608
]
Jakob Homan commented on GIRAPH-64:
---
Not that I know of. I'm allergic to rb, but I've never seen one. In Hive land
we have to manually create an instance for each issue. I just tried to create
one for this and, while Giraph exists as a target, I get a 500 exception
(something broke! - gee, that's helpful).
Create VertexRunner to make it easier to run users' computations
Key: GIRAPH-64
URL: https://issues.apache.org/jira/browse/GIRAPH-64
Project: Giraph
Issue Type: New Feature
Reporter: Jakob Homan
Assignee: Jakob Homan
Attachments: GIRAPH-64.patch
Currently, if a user wants to implement a Giraph algorithm by extending
{{Vertex}} they must also write all the boilerplate around the {{Tool}}
interface and bundle it with the Giraph jar (or get Giraph on the classpath
and playing nice with the implementation). For example, what is included in
the PageRankBenchmark and what Kohei has done:
https://github.com/smly/java-Giraph-LabelPropagation It would be better if
we had perhaps a Vertex implementation to be subclassed that already had all
the standard Tooling included such that all one had to run would be (assuming
the Giraph jar was already on the classpath):
{noformat}hadoop jar my-awesome-vertex.jar my.awesome.vertex -i jazz_input -o
jazz_output -if org.apache.giraph.lib.in.text.adjacency-list.LongDoubleDouble
-of org.apache.giraph.lib.out.text.adjacency-list.LongDoubleDouble{noformat}
This wouldn't work with every algorithm, but would be useful in a large
number of cases.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142615#comment-13142615 ] Jakob Homan commented on GIRAPH-36: --- bq. Can GIRAPH-47 be a start for what you have in mind? Yeah, that's close to what I'm hoping for. I'll comment there. bq. I can certainly imagine this being very nice and clean. Inheritance is going to be brittle over the long term, so the cleaner we can make this now, the easier it will be to attract a user base confident in investing in the new platform. This is particularly true given how much we're at the mercy of Java's poor generic system; users already have to grok things like DoubleDoubleFloat in type names. Ensure that subclassing BasicVertex is possible by user apps Key: GIRAPH-36 URL: https://issues.apache.org/jira/browse/GIRAPH-36 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.70.0 Reporter: Jake Mannix Assignee: Jake Mannix Priority: Blocker Fix For: 0.70.0 Attachments: GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff, GIRAPH-36.diff.warnings Original assumptions in Giraph were that all users would subclass Vertex (which extended MutableVertex extended BasicVertex). Classes which wish to have application specific data structures (ie. not a TreeMapI, EdgeI,E) may need to extend either MutableVertex or BasicVertex. Unfortunately VertexRange extends ArrayListVertex, and there are other places where the assumption is that vertex classes are either Vertex, or at least MutableVertex. Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: GIRAPH-64 Create VertexRunner to make it easier to run users' computations
Crap. I may have forgot to add some files. Let me check when I get home in an hour. Sorry. Sent from my iPhone On Nov 2, 2011, at 7:43 PM, Jake Mannix jake.man...@gmail.com wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2691/ Does this have the whole diff? Seems like it's missing stuff. Only has GraphMapper.java changes and pom.xml... - Jake On November 3rd, 2011, 2:29 a.m., Avery Ching wrote: Review request for giraph and Jake Mannix. By Avery Ching. *Updated 2011-11-03 02:29:54* Description This is filed on behalf of Jakob Homan. Testing Don't know. *Bugs: * GIRAPH-64 https://issues.apache.org/jira/browse/GIRAPH-64 Diffs - http://svn.apache.org/repos/asf/incubator/giraph/trunk/pom.xml (1196921) - http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java (1196921) View Diff https://reviews.apache.org/r/2691/diff/
