[GitHub] spark pull request: Spark-5854 personalized page rank
Github user jegonzal commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r29521247
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -103,8 +132,14 @@ object PageRank extends Logging {
// that didn't receive a message. Requires a shuffle for
broadcasting updated ranks to the
// edge partitions.
prevRankGraph = rankGraph
+ val rPrb = if (personalized) {
+(src: VertexId ,id: VertexId) = resetProb * delta(src,id)
+ } else {
+(src: VertexId, id: VertexId) = resetProb
+ }
+
rankGraph = rankGraph.joinVertices(rankUpdates) {
-(id, oldRank, msgSum) = resetProb + (1.0 - resetProb) * msgSum
+(id, oldRank, msgSum) = rPrb(src,id) + (1.0 - resetProb) * msgSum
--- End diff --
This all looks correct but I have a minor concern that the extra function
call and branching might increase overhead if the hotspot optimizations don't
inline. Do we have a sense as to the performance cost of this change? An
alternative, less elegant solution would be to have two code paths for lines
141 and 142 depending on whether personalization is enabled.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user jegonzal commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-98196331 Overall this looks great! I apologize for the delayed response. I am going to go ahead and merge this now and then we can tune the performance in a later pull request. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-98199444 Thanks Joey, I appreciate it. I can see your concern w/r/t the branching. If I can get some HW and time, I'll see if I notice a performance regression with the change. Cheers, Dan Sent from my iPhone On May 1, 2015, at 11:22 AM, Joey notificati...@github.com wrote: Overall this looks great! I apologize for the delayed response. I am going to go ahead and merge this now and then we can tune the performance in a later pull request. â Reply to this email directly or view it on GitHub. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4774 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-96816965 @jegonzal does this algorithm look correct to you? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-96769570 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-95256308 @jegonzal does this algorithm look correct to you? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94025941 OK, I'll update w/r/t the comments today. I'd appreciate it if someone took a glance at the algorithm; it's as specified in the referred paper, but another set of eyes would be very, very helpful. On Wed, Apr 15, 2015 at 2:43 PM, Ankur Dave notificati...@github.com wrote: This looks good aside from the comments. I didn't verify the algorithm; it would be good if @jegonzal https://github.com/jegonzal could take a look. â Reply to this email directly or view it on GitHub https://github.com/apache/spark/pull/4774#issuecomment-93577536. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94093081 [Test build #30507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30507/consoleFull) for PR 4774 at commit [`d6cebac`](https://github.com/apache/spark/commit/d6cebac881756938bfad7b10bc9674083715a2c1). * This patch **does not merge cleanly**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94121865 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30511/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94121860 [Test build #30511 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30511/consoleFull) for PR 4774 at commit [`8b907db`](https://github.com/apache/spark/commit/8b907db18ddc2ae5c0d16380e824c15e76183567). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. * This patch does not change any dependencies. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94107018 [Test build #30509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30509/consoleFull) for PR 4774 at commit [`2c20e5d`](https://github.com/apache/spark/commit/2c20e5d64749c533bc5fdf7304ed175cef686644). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94107117 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30509/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94107110 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30507/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94107116 [Test build #30509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30509/consoleFull) for PR 4774 at commit [`2c20e5d`](https://github.com/apache/spark/commit/2c20e5d64749c533bc5fdf7304ed175cef686644). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` case class Params(` * `sealed abstract class Node extends Serializable ` * `sealed trait Split extends Serializable ` * `final class CategoricalSplit(` * `final class ContinuousSplit(override val featureIndex: Int, val threshold: Double) extends Split ` * `trait DecisionTreeModel ` * This patch does not change any dependencies. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94107104 [Test build #30507 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30507/consoleFull) for PR 4774 at commit [`d6cebac`](https://github.com/apache/spark/commit/d6cebac881756938bfad7b10bc9674083715a2c1). * This patch **passes all tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-94112401 [Test build #30511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30511/consoleFull) for PR 4774 at commit [`8b907db`](https://github.com/apache/spark/commit/8b907db18ddc2ae5c0d16380e824c15e76183567). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-93562171 Is this going to get merged at some point? On Tue, Mar 31, 2015 at 10:51 AM, Yusup notificati...@github.com wrote: +1 â Reply to this email directly or view it on GitHub https://github.com/apache/spark/pull/4774#issuecomment-88186888. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-93562703 @dwmclary This doesn't merge, so would at least need a rebase. It's up to @ankurdave and @jegonzal et al. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r28465870
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -133,7 +167,29 @@ object PageRank extends Logging {
* containing the normalized weight.
*/
def runUntilConvergence[VD: ClassTag, ED: ClassTag](
- graph: Graph[VD, ED], tol: Double, resetProb: Double = 0.15):
Graph[Double, Double] =
+ graph: Graph[VD, ED], tol: Double, resetProb: Double = 0.15):
Graph[Double, Double] =
--- End diff --
style: indent parameter lists by 4 spaces
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r28465105
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -89,6 +114,10 @@ object PageRank extends Logging {
// Set the vertex attributes to the initial pagerank values
.mapVertices( (id, attr) = resetProb )
+val personalized = srcId isDefined
+val src:VertexId = srcId.getOrElse(-1L)
+def delta(u:VertexId, v:VertexId):Double = { if (u == v) 1.0 else 0.0 }
--- End diff --
Minor style comment: there should be a space after the colon here and on
the line above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r28465270
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -133,7 +167,29 @@ object PageRank extends Logging {
* containing the normalized weight.
*/
def runUntilConvergence[VD: ClassTag, ED: ClassTag](
- graph: Graph[VD, ED], tol: Double, resetProb: Double = 0.15):
Graph[Double, Double] =
+ graph: Graph[VD, ED], tol: Double, resetProb: Double = 0.15):
Graph[Double, Double] =
+ {
+ runUntilConvergenceWithOptions(graph, tol, resetProb)
+ }
+
+ /**
+ * Run a dynamic version of PageRank returning a graph with vertex
attributes containing the
+ * PageRank and edge attributes containing the normalized edge weight.
+ *
+ * @tparam VD the original vertex attribute (not used)
+ * @tparam ED the original edge attribute (not used)
+ *
+ * @param graph the graph on which to compute PageRank
+ * @param tol the tolerance allowed at convergence (smaller = more
accurate).
+ * @param resetProb the random reset probability (alpha)
+ * @param srcId the source vertex for a Personalized Page Rank (optional)
+ *
+ * @return the graph containing with each vertex containing the PageRank
and each edge
+ * containing the normalized weight.
+ */
+ def runUntilConvergenceWithOptions[VD: ClassTag, ED: ClassTag](
+ graph: Graph[VD, ED], tol: Double, resetProb: Double = 0.15,
+ srcId: Option[VertexId] = None): Graph[Double, Double] =
--- End diff --
style: inconsistent indent
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-93577536 This looks good aside from the comments. I didn't verify the algorithm; it would be good if @jegonzal could take a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r28465288
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -148,6 +204,10 @@ object PageRank extends Logging {
.mapVertices( (id, attr) = (0.0, 0.0) )
.cache()
+val personalized = srcId isDefined
--- End diff --
style: prefer `srcId.isDefined`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r28465211
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -104,7 +133,12 @@ object PageRank extends Logging {
// edge partitions.
prevRankGraph = rankGraph
rankGraph = rankGraph.joinVertices(rankUpdates) {
+if (personalized) {
--- End diff --
It would be better to move the conditional outside the closure (i.e., `if
(personalized) { rankGraph.joinVertices ... } else { ...} `) to avoid a branch
at each vertex.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r28465575
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -170,8 +241,16 @@ object PageRank extends Logging {
val initialMessage = resetProb / (1.0 - resetProb)
// Execute a dynamic version of Pregel.
-Pregel(pagerankGraph, initialMessage, activeDirection =
EdgeDirection.Out)(
- vertexProgram, sendMessage, messageCombiner)
- .mapVertices((vid, attr) = attr._1)
+if (personalized) {
+ Pregel(pagerankGraph, initialMessage, activeDirection =
EdgeDirection.Out)(
+personalizedVertexProgram, sendMessage, messageCombiner)
+.mapVertices((vid, attr) = attr._1)
+} else {
+ Pregel(pagerankGraph, initialMessage, activeDirection =
EdgeDirection.Out)(
+vertexProgram, sendMessage, messageCombiner)
+.mapVertices((vid, attr) = attr._1)
+}
--- End diff --
I think you could simplify this to something like
```scala
val vp = if (personalized) personalizedVertexProgram else vertexProgram
Pregel(pagerankGraph, initialMessage, activeDirection = EdgeDirection.Out)(
vp, sendMessage, messageCombiner)
.mapVertices((vid, attr) = attr._1)
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user xperian commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-88186888 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-83669688 Good to merge? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81538617 [Test build #28641 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28641/consoleFull) for PR 4774 at commit [`fba0edd`](https://github.com/apache/spark/commit/fba0eddf3f905b5485585785feaeb082de4b3eb2). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81538624 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28641/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81829391 OK, that should fix the binary incompatibility on the vertexProgram. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81831186 [Test build #28661 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28661/consoleFull) for PR 4774 at commit [`d711677`](https://github.com/apache/spark/commit/d71167712ff8da3707214751c07b0932bfc5bb72). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81847916 [Test build #28664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28664/consoleFull) for PR 4774 at commit [`7d00c23`](https://github.com/apache/spark/commit/7d00c2386d643aa3d4fb248cfcbe5c7183bfaeba). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81831781 [Test build #28661 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28661/consoleFull) for PR 4774 at commit [`d711677`](https://github.com/apache/spark/commit/d71167712ff8da3707214751c07b0932bfc5bb72). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81831805 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28661/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81886679 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28664/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81886657 [Test build #28664 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28664/consoleFull) for PR 4774 at commit [`7d00c23`](https://github.com/apache/spark/commit/7d00c2386d643aa3d4fb248cfcbe5c7183bfaeba). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81482802 [Test build #28641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28641/consoleFull) for PR 4774 at commit [`fba0edd`](https://github.com/apache/spark/commit/fba0eddf3f905b5485585785feaeb082de4b3eb2). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-81480270 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-78824623 OK, got 'em. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r26365904
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -148,11 +204,20 @@ object PageRank extends Logging {
.mapVertices( (id, attr) = (0.0, 0.0) )
.cache()
+val personalized = if (srcId isDefined) true else false
+val src:VertexId = srcId.getOrElse(-1L)
--- End diff --
add a space after :
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r26365902
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -148,11 +204,20 @@ object PageRank extends Logging {
.mapVertices( (id, attr) = (0.0, 0.0) )
.cache()
+val personalized = if (srcId isDefined) true else false
--- End diff --
this is just
```
val personalized = srcId.isDefined
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/4774#discussion_r26362827
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala
---
@@ -74,10 +76,35 @@ object PageRank extends Logging {
*
* @return the graph containing with each vertex containing the PageRank
and each edge
* containing the normalized weight.
+ */
+
+ def run[VD: ClassTag, ED: ClassTag](graph: Graph[VD, ED], numIter: Int,
+resetProb: Double = 0.15): Graph[Double, Double] =
+ {
+runWithOptions(graph, numIter, resetProb)
+ }
+
+ /**
+ * Run PageRank for a fixed number of iterations returning a graph
+ * with vertex attributes containing the PageRank and edge
+ * attributes the normalized edge weight.
+ *
+ * @tparam VD the original vertex attribute (not used)
+ * @tparam ED the original edge attribute (not used)
+ *
+ * @param graph the graph on which to compute PageRank
+ * @param numIter the number of iterations of PageRank to run
+ * @param resetProb the random reset probability (alpha)
+ * @param srcId the source vertex for a Personalized Page Rank (optional)
+ *
+ * @return the graph containing with each vertex containing the PageRank
and each edge
+ * containing the normalized weight.
*
*/
- def run[VD: ClassTag, ED: ClassTag](
- graph: Graph[VD, ED], numIter: Int, resetProb: Double = 0.15):
Graph[Double, Double] =
+
--- End diff --
The extra space line is redundant. Could you remove all extra spaces
between comments and methods in this file?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-78804717 Whitespace removed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-78766347 OK, that should be a reasonable solution. Thanks for the advice @rxin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-78614976 I certainly agree that binary compatibility matters. I think it's mainly a question of which is more desirable: fewer repeated LOC or binary compatibility. I can write new methods for each of the approaches to PPR, but it would require repeating a lot of the pregel boilerplate. If you feel strongly that's a better approach, I'm happy to make the change. On Thu, Mar 12, 2015 at 1:40 PM, Sean Owen notificati...@github.com wrote: Yes but binary compatibility also matters. I can't see the Jenkins output anymore, but I recall thinking it was probably legitimate. Can you write this differently without optional args maybe? â Reply to this email directly or view it on GitHub https://github.com/apache/spark/pull/4774#issuecomment-78606657. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-78616511 Also you don't necessarily need to duplicate a lot of code. You can have one Pregel function calling another. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-78616278 Sorry binary compatibility is not a preference. It is a requirement here ... otherwise it is a huge hassle for people to upgrade, which leads to more fragmentation in versions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-78606657 Yes but binary compatibility also matters. I can't see the Jenkins output anymore, but I recall thinking it was probably legitimate. Can you write this differently without optional args maybe? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-78594489 Does anyone have a comment on this MiMa failure? The fact that PageRankSuite passes illustrates that it's source compatible. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-77434500 I think this change actually introduces binary incompatability even though you have only added optional args. It may be source compatible. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-77436963 OK, thanks Sean, that was my reading of it too. On Thu, Mar 5, 2015 at 11:32 AM, Sean Owen notificati...@github.com wrote: I think this change actually introduces binary incompatability even though you have only added optional args. It may be source compatible. â Reply to this email directly or view it on GitHub https://github.com/apache/spark/pull/4774#issuecomment-77434500. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-77433149 I'm not really sure what to do about this MiMa error. Suggestions? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-77398055 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-77410205 jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-77410688 [Test build #28301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28301/consoleFull) for PR 4774 at commit [`44ada8e`](https://github.com/apache/spark/commit/44ada8e02bd2d307a17ee5186e636877d25b3e03). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-77427253 [Test build #28301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28301/consoleFull) for PR 4774 at commit [`44ada8e`](https://github.com/apache/spark/commit/44ada8e02bd2d307a17ee5186e636877d25b3e03). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-77427282 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28301/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-76252508 Cool! ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
GitHub user dwmclary opened a pull request: https://github.com/apache/spark/pull/4774 Spark-5854 personalized page rank Here's a modification to PageRank which does personalized PageRank. The approach is basically similar to that outlined by Bahmani et al. from 2010 (http://arxiv.org/pdf/1006.2880.pdf). I'm sure this needs tuning up or other considerations, so let me know how I can improve this. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dwmclary/spark SPARK-5854-Personalized-PageRank Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/4774.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4774 commit 6dc2c29b6d4ab6eb5ceb34b0f6e135843dd6cdd9 Author: Dan McClary dan.mccl...@gmail.com Date: 2015-02-24T07:24:06Z initial implementation of personalized page rank commit a5609420b897df357079a7cbe3dfaf155f72f7b7 Author: Dan McClary dan.mccl...@gmail.com Date: 2015-02-25T22:30:54Z rolled PPR into pregel code for PageRank commit b67ac693896e2d6c8d080f107e0e86343a522387 Author: Dan McClary dan.mccl...@gmail.com Date: 2015-02-25T22:43:59Z updated tolerance on chain PPR commit 1ffed95b640c3933976ddfd2ecce1a7fa60c83a3 Author: Dan McClary dan.mccl...@gmail.com Date: 2015-02-25T22:49:28Z updated tolerance on chain PPR commit 44ada8e02bd2d307a17ee5186e636877d25b3e03 Author: Dan McClary dan.mccl...@gmail.com Date: 2015-02-25T22:51:40Z updated tolerance on chain PPR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-76086384 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Spark-5854 personalized page rank
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4774#issuecomment-76120556 cc Ankur @ankurdave --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
