[jira] [Created] (GIRAPH-79) Change the menu layout of the site
Change the menu layout of the site -- Key: GIRAPH-79 URL: https://issues.apache.org/jira/browse/GIRAPH-79 Project: Giraph Issue Type: Task Components: site Reporter: Hyunsik Choi The current site has the basic menu layout generated by maven site plugin. This layout is restricted to embrace new contents. I would like to suggest the following menu layout. http://people.apache.org/~hyunsik/giraph/site/index.html Although the layout includes most existing contents, it has two addition categories, Giraph and Documentation. I think that this layout is simpler and is good to add new contents. Anyone has any other suggestions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-79) Change the menu layout of the site
[ https://issues.apache.org/jira/browse/GIRAPH-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyunsik Choi updated GIRAPH-79: --- Attachment: GIRAPH-79_1.patch Change the menu layout of the site -- Key: GIRAPH-79 URL: https://issues.apache.org/jira/browse/GIRAPH-79 Project: Giraph Issue Type: Task Components: site Reporter: Hyunsik Choi Labels: site Attachments: GIRAPH-79_1.patch The current site has the basic menu layout generated by maven site plugin. This layout is restricted to embrace new contents. I would like to suggest the following menu layout. http://people.apache.org/~hyunsik/giraph/site/index.html Although the layout includes most existing contents, it has two addition categories, Giraph and Documentation. I think that this layout is simpler and is good to add new contents. Anyone has any other suggestions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-79) Change the menu layout of the site
[ https://issues.apache.org/jira/browse/GIRAPH-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13149295#comment-13149295 ] Gianmarco De Francisci Morales commented on GIRAPH-79: -- It looks nice. But why keep project reports in the home page? Are they so important for new users? To me they look confusing, I would put them somewhere else deeper in the site. Change the menu layout of the site -- Key: GIRAPH-79 URL: https://issues.apache.org/jira/browse/GIRAPH-79 Project: Giraph Issue Type: Task Components: site Reporter: Hyunsik Choi Labels: site Attachments: GIRAPH-79_1.patch The current site has the basic menu layout generated by maven site plugin. This layout is restricted to embrace new contents. I would like to suggest the following menu layout. http://people.apache.org/~hyunsik/giraph/site/index.html Although the layout includes most existing contents, it has two addition categories, Giraph and Documentation. I think that this layout is simpler and is good to add new contents. Anyone has any other suggestions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-79) Change the menu layout of the site
[ https://issues.apache.org/jira/browse/GIRAPH-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13149304#comment-13149304 ] Hyunsik Choi commented on GIRAPH-79: Gianmarco, I misunderstood your mention. You did not the remove of the project reports. I'm agree that the report is placed deeper in the site. Change the menu layout of the site -- Key: GIRAPH-79 URL: https://issues.apache.org/jira/browse/GIRAPH-79 Project: Giraph Issue Type: Task Components: site Reporter: Hyunsik Choi Labels: site Attachments: GIRAPH-79_1.patch The current site has the basic menu layout generated by maven site plugin. This layout is restricted to embrace new contents. I would like to suggest the following menu layout. http://people.apache.org/~hyunsik/giraph/site/index.html Although the layout includes most existing contents, it has two addition categories, Giraph and Documentation. I think that this layout is simpler and is good to add new contents. Anyone has any other suggestions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-11) Improve the graph distribution of Giraph
[ https://issues.apache.org/jira/browse/GIRAPH-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13149412#comment-13149412 ] Hyunsik Choi commented on GIRAPH-11: Avery, I'm sorry for delaying the review. Now, I'm digging your patch. That looks great! Based on this work, we can consider some advanced graph partitioner based on the number of edge-cuts on graph partitions. I need about one more day for more investigation because the patch is somewhat complicated for me :) Besides, for the deeper review, I would like to execute the some tests and trace them. Your patch needs the rebase. Could you rebase the patch? Thank you :) Improve the graph distribution of Giraph Key: GIRAPH-11 URL: https://issues.apache.org/jira/browse/GIRAPH-11 Project: Giraph Issue Type: Improvement Affects Versions: 0.70.0 Reporter: Avery Ching Assignee: Avery Ching Attachments: GIRAPH-11.diff Currently, Giraph assumes that the data from the VertexInputFormat is sorted. If the user data is not sorted by the vertex id, they must first run a MapReduce or Pig job to generate a sorted dataset. This is often a bit inconvenient. Giraph graph partitioning is currently range based and there are some advantages and disadvantages of this approach. The proposal of this JIRA would be to allow for both range and hash based partitioning and provide more flexibility to the user. Design goals for the graph distribution: * Allow vertices to be unordered or unordered * Ability to repartition * Select the partitioning scheme based on user needs (i.e. hash or range based) * Ability to provide user-specific hints about partitions Hash-based partitioning * Good vertex balancing across ranges for random data * Bad at vertex id locality Range-based partitioning * Good at vertex id locality * Ability to split ranges easily * Can cause hotspots for hot ranges -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-75) Create sections on how to get involved and how to generate patches on website
[ https://issues.apache.org/jira/browse/GIRAPH-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated GIRAPH-75: -- Attachment: GIRAPH-75-2.patch Updated patch based on Claudio's feedback. Will commit this. Create sections on how to get involved and how to generate patches on website - Key: GIRAPH-75 URL: https://issues.apache.org/jira/browse/GIRAPH-75 Project: Giraph Issue Type: Improvement Reporter: Jakob Homan Assignee: Jakob Homan Attachments: GIRAPH-75-2.patch, GIRAPH-75.patch We've had several questions lately on how to get started. It would be good to document this on the site. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: better way to update site?
Cool. I've got ahead and deleted the generated site from the repo and copied in the latest version (post GIRAPH-75). Thanks.
[jira] [Commented] (GIRAPH-75) Create sections on how to get involved and how to generate patches on website
[ https://issues.apache.org/jira/browse/GIRAPH-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13149431#comment-13149431 ] Jakob Homan commented on GIRAPH-75: --- bq. How about making a separate page for these sections? Sounds good. I'll comment on GIRAPH-79. Create sections on how to get involved and how to generate patches on website - Key: GIRAPH-75 URL: https://issues.apache.org/jira/browse/GIRAPH-75 Project: Giraph Issue Type: Improvement Reporter: Jakob Homan Assignee: Jakob Homan Fix For: 0.70.0 Attachments: GIRAPH-75-2.patch, GIRAPH-75.patch We've had several questions lately on how to get started. It would be good to document this on the site. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-75) Create sections on how to get involved and how to generate patches on website
[ https://issues.apache.org/jira/browse/GIRAPH-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13149432#comment-13149432 ] Hudson commented on GIRAPH-75: -- Integrated in Giraph-trunk-Commit #28 (See [https://builds.apache.org/job/Giraph-trunk-Commit/28/]) GIRAPH-75. Create sections on how to get involved and how to generate patches on website. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1201596 Files : * /incubator/giraph/trunk/CHANGELOG * /incubator/giraph/trunk/src/site/xdoc/index.xml Create sections on how to get involved and how to generate patches on website - Key: GIRAPH-75 URL: https://issues.apache.org/jira/browse/GIRAPH-75 Project: Giraph Issue Type: Improvement Reporter: Jakob Homan Assignee: Jakob Homan Fix For: 0.70.0 Attachments: GIRAPH-75-2.patch, GIRAPH-75.patch We've had several questions lately on how to get started. It would be good to document this on the site. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-79) Change the menu layout of the site
[ https://issues.apache.org/jira/browse/GIRAPH-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-79. --- Resolution: Fixed Assignee: Hyunsik Choi I've committed this and updated the site. Thanks, Hyunsik! Change the menu layout of the site -- Key: GIRAPH-79 URL: https://issues.apache.org/jira/browse/GIRAPH-79 Project: Giraph Issue Type: Task Components: site Reporter: Hyunsik Choi Assignee: Hyunsik Choi Labels: site Attachments: GIRAPH-79_1.patch, GIRAPH-79_2.patch The current site has the basic menu layout generated by maven site plugin. This layout is restricted to embrace new contents. I would like to suggest the following menu layout. http://people.apache.org/~hyunsik/giraph/site/index.html Although the layout includes most existing contents, it has two addition categories, Giraph and Documentation. I think that this layout is simpler and is good to add new contents. Anyone has any other suggestions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: better way to update site?
+1 This way is the best for us :) -- Hyunsik Choi On Mon, Nov 14, 2011 at 12:48 PM, Jakob Homan jgho...@gmail.com wrote: Cool. I've got ahead and deleted the generated site from the repo and copied in the latest version (post GIRAPH-75). Thanks.
Re: better way to update site?
I've added a page to the wiki with instructions on how I did it: https://cwiki.apache.org/confluence/display/GIRAPH/Committer+notes On Sun, Nov 13, 2011 at 8:18 PM, Hyunsik Choi hyun...@apache.org wrote: +1 This way is the best for us :) -- Hyunsik Choi On Mon, Nov 14, 2011 at 12:48 PM, Jakob Homan jgho...@gmail.com wrote: Cool. I've got ahead and deleted the generated site from the repo and copied in the latest version (post GIRAPH-75). Thanks.
Re: better way to update site?
Thank you for the nice instruction. I've updated the rsync command for group permission. -- Hyunsik Choi On Mon, Nov 14, 2011 at 1:19 PM, Jakob Homan jgho...@gmail.com wrote: I've added a page to the wiki with instructions on how I did it: https://cwiki.apache.org/confluence/display/GIRAPH/Committer+notes On Sun, Nov 13, 2011 at 8:18 PM, Hyunsik Choi hyun...@apache.org wrote: +1 This way is the best for us :) -- Hyunsik Choi On Mon, Nov 14, 2011 at 12:48 PM, Jakob Homan jgho...@gmail.com wrote: Cool. I've got ahead and deleted the generated site from the repo and copied in the latest version (post GIRAPH-75). Thanks.
Re: Review Request: GIRAPH-11 : Improve the graph distribution of Giraph
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2788/ --- (Updated 2011-11-14 06:56:19.251685) Review request for giraph. Changes --- Updated the diff as per Hyunsik's request to build against recent trunk changes. While I was waiting I added some fixed and additions as well. Upgrade ZooKeeper to 3.3.3 from 3.3.1. Fixed bug in PseudoRandomVertexInputFormat.java where the edges are not fully added (hasEdge is not the right place to look for the edge). Fixed bug in BasicRPCCommunications when putting to a local inPartitionMap Added counter for last checkpointed superstep Master should refresh the progress every 60 seconds while waiting for workers to ensure that the job isn't killed Fixed bugs in vertexCounter, finishedVertexCoutner, edgeCounter, and sentMessages counter not resetting every update (just cumultatively being added). Add additional helpful status messages for debugging. Turned off speculative execution for Giraph (not a good idea). Added analysis of the partition balancing for debugging Summary --- Warning: This is a very large change! Vertex ranges no longer exist. A generic partitioner handles the division of vertex ids to partitions. As a default, there is a HashPartitioner and a HashRangePartitioner that will use the hashCode of a Java object to decide which partition to place the vertex. Developers can write their own algorithm to determine how to change the partitioning as well as implement the assignment of partitions to workers. All vertices loaded from the input split are sent to the owner of the partition rather than loaded locally. This eliminates the constraint that the vertices must be ordered in the input split. The checkpoint format has been changed to suit the new partition style. Checkpoints are now a lot simpler. The master will assign partitions and the workers will only load their own partitions from the checkpoint. Unfortunately, the vertex range implementation was baked into almost every aspect of the code (hence the ridiculous size of this diff). But now it should be flexible to support several different graph partitioning schemes (i.e. hash-based, hash-ranged-based, and for special cases, fully ranged-based). Sorry for the long delay, but this way pretty involved. This addresses bug GIRAPH-11. https://issues.apache.org/jira/browse/GIRAPH-11 Diffs (updated) - http://svn.apache.org/repos/asf/incubator/giraph/trunk/pom.xml 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/benchmark/PseudoRandomVertexInputFormat.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/RPCCommunications.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/ServerInterface.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/WorkerCommunications.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/GeneratedVertexInputFormat.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/GeneratedVertexReader.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/MaxAggregator.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/MinAggregator.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/SimpleMutateGraphVertex.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/SimpleSuperstepVertex.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/SuperstepBalancer.java 1201607 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/SuperstepHashPartitioner.java PRE-CREATION http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/VerifyMessage.java PRE-CREATION http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/AutoBalancer.java 1201607
[jira] [Commented] (GIRAPH-11) Improve the graph distribution of Giraph
[ https://issues.apache.org/jira/browse/GIRAPH-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13149471#comment-13149471 ] Hyunsik Choi commented on GIRAPH-11: Thank you for rebase. Improve the graph distribution of Giraph Key: GIRAPH-11 URL: https://issues.apache.org/jira/browse/GIRAPH-11 Project: Giraph Issue Type: Improvement Affects Versions: 0.70.0 Reporter: Avery Ching Assignee: Avery Ching Attachments: GIRAPH-11.diff Currently, Giraph assumes that the data from the VertexInputFormat is sorted. If the user data is not sorted by the vertex id, they must first run a MapReduce or Pig job to generate a sorted dataset. This is often a bit inconvenient. Giraph graph partitioning is currently range based and there are some advantages and disadvantages of this approach. The proposal of this JIRA would be to allow for both range and hash based partitioning and provide more flexibility to the user. Design goals for the graph distribution: * Allow vertices to be unordered or unordered * Ability to repartition * Select the partitioning scheme based on user needs (i.e. hash or range based) * Ability to provide user-specific hints about partitions Hash-based partitioning * Good vertex balancing across ranges for random data * Bad at vertex id locality Range-based partitioning * Good at vertex id locality * Ability to split ranges easily * Can cause hotspots for hot ranges -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
