[GitHub] incubator-spark pull request: SPARK-1078: Replace lift-json with j...

2014-02-24 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/582#issuecomment-35920859 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] incubator-spark pull request: MLLIB-25: Implicit ALS runs out of m...

2014-02-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/629#issuecomment-35776515 Yup, good idea. I also cherry picked it into branch-0.9. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] incubator-spark pull request: MLLIB-25: Implicit ALS runs out of m...

2014-02-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/629#issuecomment-35771699 Thanks. I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: MLLIB-25: Implicit ALS runs out of m...

2014-02-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/incubator-spark/pull/629#discussion_r9956743 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala --- @@ -212,7 +212,7 @@ class ALS private (var numBlocks: Int, var rank

[GitHub] incubator-spark pull request: MLLIB-25: Implicit ALS runs out of m...

2014-02-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/incubator-spark/pull/629#discussion_r9956734 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala --- @@ -212,7 +212,7 @@ class ALS private (var numBlocks: Int, var rank

[GitHub] incubator-spark pull request: MLLIB-22. Support negative implicit ...

2014-02-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/500#issuecomment-35596136 Merged. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-post your

[GitHub] incubator-spark pull request: MLLIB-24: url of "Collaborative Filt...

2014-02-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/619#issuecomment-35592012 Thanks. I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35469927 Yes, I agree for primitive types as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35469499 This discussion should probably have happened on the mailing list, but here you go. In general I think we should favor explicit type declaration in public

[GitHub] incubator-spark pull request: MLLIB-24: url of "Collaborative Filt...

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/619#discussion_r9853229 Need to wrap the line so it fits within 100 chars. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To

[GitHub] incubator-spark pull request: url error

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/619#issuecomment-35461371 Do you mind creating a JIRA ticket and update the pull request title to include that? --- If your project is set up for it, you can reply to this email and have

[GitHub] incubator-spark pull request: SPARK-1106: check key name and ident...

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/617#issuecomment-35459852 Ok I merged this. Thanks guys. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: SPARK-1106: check key name and ident...

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/617#issuecomment-35459609 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: [Proposal] Adding sparse data suppor...

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/575#issuecomment-35445845 @fommil maybe that changed in 2.10.x entirely given the addition of value classes, and maybe Breeze is very careful in its implicit usage, but often implicits in

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35431514 I made one comment on the ambiguity of Double. Other than that, LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9840393 We should probably rename this JDouble or just use java.lang.Double in the codebase. --- If your project is set up for it, you can reply to this email and have

[GitHub] incubator-spark pull request: SPARK-1098: Minor cleanup of ClassTa...

2014-02-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/604#issuecomment-35349097 Thanks. I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Added Java API for AsyncRDDActions w...

2014-02-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/607#issuecomment-35327669 Thanks for submitting this. One discussion we need to have before merging this is on the long term support for the async scala APIs. When I added those, they were

[GitHub] incubator-spark pull request: MLI-2: Start adding k-fold cross val...

2014-02-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/572#discussion_r9805702 Basically you want to make sure it is obvious that this returns a tuple (which can also be done through explicit type declaration but probably simpler this way

[GitHub] incubator-spark pull request: MLI-2: Start adding k-fold cross val...

2014-02-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/572#discussion_r9802309 ```scala (1 to numFolds).map { fold => val sampler = new BernoulliSampler[T]((fold-1)/foldsF,fold/foldsF, complement = false) val train =

[GitHub] incubator-spark pull request: Clean up and added a few java api me...

2014-02-13 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/402#issuecomment-35061130 Great. @ScrapCodes can you mark the scala *With's as deprecated in 1.0.0, and remove the java apis from this pr?

[GitHub] incubator-spark pull request: [Application-UI] Reporting waiting s...

2014-02-13 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/123#issuecomment-35059482 ping @kayousterhout

[GitHub] incubator-spark pull request: Clean up and added a few java api me...

2014-02-13 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/402#issuecomment-35059461 I agree with you. I actually think we should consider deprecating them for Scala as well. The same sentiment is also echoed by @markhamstra who initially wrote

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-13 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/593#issuecomment-35029540 Closing since it's been merged.

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-13 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/incubator-spark/pull/593

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/593#issuecomment-34954956 Ok I merged this using the new script!

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-12 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/incubator-spark/pull/592

[GitHub] incubator-spark pull request: SPARK-1085: Fix Jenkins pull request...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/590#issuecomment-34953037 Closing this one since it has been merged.

[GitHub] incubator-spark pull request: SPARK-1085: Fix Jenkins pull request...

2014-02-12 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/incubator-spark/pull/590

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-12 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/incubator-spark/pull/593 SPARK-1088: Create a script for running tests so we can have version specific testing on Jenkins (branch-0.9) This is for branch-0.9. #592 is for master branch (1.0). You can merge

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/592#issuecomment-34952038 Yea. It just ran a script similar to this, except it is not a "script", just a sequence of commands defined in jenkins.

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/592#issuecomment-34949375 As suggested in #590.

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/592#issuecomment-34948410 I will submit another one for branch-0.9 if this looks good.

[GitHub] incubator-spark pull request: SPARK-1088: Create a script for runn...

2014-02-12 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/incubator-spark/pull/592 SPARK-1088: Create a script for running tests so we can have version specific testing on Jenkins. @pwendell You can merge this pull request into a Git repository by running: $ git pull

[GitHub] incubator-spark pull request: SPARK-1085: Fix Jenkins pull request...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/590#issuecomment-34947355 That's a good idea. I will submit a PR.

[GitHub] incubator-spark pull request: SPARK-1076: [Fix #578] add @transien...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/591#issuecomment-34935532 Ok I'm merging this one. Thanks.

[GitHub] incubator-spark pull request: Ported hadoopClient jar for < 1.0.1 ...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/584#issuecomment-34923271 We can test this once #590 is in.

[GitHub] incubator-spark pull request: SPARK-1085: Fix Jenkins pull request...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/590#issuecomment-34923019 @aarondav @marmbrus i'm no sbt expert

[GitHub] incubator-spark pull request: SPARK-1085: Fix Jenkins pull request...

2014-02-12 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/incubator-spark/pull/590 SPARK-1085: Fix Jenkins pull request builder for branch-0.9 (scalastyle command not found) Added a dummy scalastyle task to sbt. https://spark-project.atlassian.net/browse/SPARK-1085

[GitHub] incubator-spark pull request: Adding RDD unique self cross product

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/587#issuecomment-34916300 Thanks for submitting this. Just curious, what is the advantage of this over rdd.cartesian(rdd), i.e. just use cartesian to join itself?

[GitHub] incubator-spark pull request: SPARK-1076: Convert Int to Long to a...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/589#issuecomment-34901501 Thanks. Merged.

[GitHub] incubator-spark pull request: Support MiMa for reporting binary co...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-34850552 It's probably good to have the result sent to the dev list at some point.

[GitHub] incubator-spark pull request: Support MiMa for reporting binary co...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-34849894 Do you have an example of the report?

[GitHub] incubator-spark pull request: SPARK-1073 Keep GitHub pull request ...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/574#issuecomment-34849557 @pwendell comment? Would be great to have this merged soon.

[GitHub] incubator-spark pull request: SPARK-1021 sortByKey() shouldn't lau...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/379#issuecomment-34849506 @ash211 mind closing this for now? We should revisit this when we find a solution.

[GitHub] incubator-spark pull request: SPARK-1076: zipWithIndex and zipWith...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/578#issuecomment-34848995 Thanks. Merging this. We might need to add Java / Python APIs too ... but that can be done in a later PR.

[GitHub] incubator-spark pull request: SPARK-1076: zipWithIndex and zipWith...

2014-02-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/578#issuecomment-34847266 Jenkins, retest this please.

[GitHub] incubator-spark pull request: SPARK-1076: zipWithIndex and zipWith...

2014-02-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/578#issuecomment-34846154 lgtm. will merge once jenkins is happy

[GitHub] incubator-spark pull request: Ported hadoopClient jar for < 1.0.1 ...

2014-02-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/584#issuecomment-34844952 BTW Jenkins desire assertiveness. By asking him whether you could do this, he wouldn't do it ...

[GitHub] incubator-spark pull request: Ported hadoopClient jar for < 1.0.1 ...

2014-02-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/584#issuecomment-34844932 It's because branch-0.9 doesn't have the scalastyle stuff ...

[GitHub] incubator-spark pull request: Ported hadoopClient jar for < 1.0.1 ...

2014-02-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/584#issuecomment-34844882 Jenkins, retest this please.

[GitHub] incubator-spark pull request: Adding assignRanks and assignUniqueI...

2014-02-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/578#issuecomment-34844284 BTW - the new dev rules require us to create a JIRA for this, and update the PR description to include a link to the JIRA.

[GitHub] incubator-spark pull request: SPARK-1072 Use binary search when ne...

2014-02-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/571#issuecomment-34818238 Thanks. I've merged this.

[GitHub] incubator-spark pull request: SPARK-1075 Fix doc in the Spark Stre...

2014-02-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/577#issuecomment-34817976 Thanks. I've merged this.

[GitHub] incubator-spark pull request: SPARK-1078: Replace lift-json with j...

2014-02-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/582#issuecomment-34801246 Jenkins, add to whitelist.

[GitHub] incubator-spark pull request: "in the source DStream" rather than ...

2014-02-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/579#issuecomment-34729446 Thanks. Merged.

[GitHub] incubator-spark pull request: SPARK-1072 Use binary search when ne...

2014-02-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/571#issuecomment-34607475 Our Jenkins style checker actually worked!!!

[GitHub] incubator-spark pull request: MLI-2: Add k-fold cross validation t...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/572#issuecomment-34606676 Jenkins, add to whitelist.

[GitHub] incubator-spark pull request: SPARK-1058, Fix Style Errors and Add...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/567#issuecomment-34603519 Thanks! I'm merging this now.

[GitHub] incubator-spark pull request: SPARK-1058, Fix Style Errors and Add...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/567#issuecomment-34603004 Jenkins, retest this please.

[GitHub] incubator-spark pull request: Support negative implicit input in A...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/500#issuecomment-34601206 Thanks, @mengxr and @MLnick for reviewing this. @srowen this is no longer mergeable due to the style update pr. Do you mind updating the PR so it can be

[GitHub] incubator-spark pull request: new MLlib documentation for optimiza...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/566#issuecomment-34591185 Thanks. I've merged this.

[GitHub] incubator-spark pull request: SPARK-1058, Fix Style Errors and Add...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/567#issuecomment-34589963 Thanks, @ScrapCodes, for doing this! I just went through all your changes. I made some comments to improve the readability of some line wraps. Often you want to

[GitHub] incubator-spark pull request: new MLlib documentation for optimiza...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/566#issuecomment-34588467 Thanks for rebasing this. There's a couple small conflicts, likely from the style cleanup pull request. Do you mind updating this so it can be merged cleanly?

[GitHub] incubator-spark pull request: [SPARK-1038] Add more fields in Json...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/551#issuecomment-34588354 LGTM. Merging this now. Thanks!

[GitHub] incubator-spark pull request: Fixes bug where merges won't close a...

2014-02-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/569#issuecomment-34588246 Ok I merged this PR using the script in this PR. Let's see ...

[GitHub] incubator-spark pull request: [SPARK-1038] Add more fields in Json...

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/551#issuecomment-34567984 Thanks. I left some comments to improve readability of the code.

[GitHub] incubator-spark pull request: [WIP] SPARK-1067: Default log4j init...

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/560#issuecomment-34567839 Oops I didn't realize the WIP in title. Feel free to revert if necessary.

[GitHub] incubator-spark pull request: [SPARK-1060] startJettyServer should...

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/556#issuecomment-34567822 Thanks. I merged this in master.

[GitHub] incubator-spark pull request: Added example Python code for sort

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/562#issuecomment-34567786 Thanks. Merged this in master & branch-0.9.

[GitHub] incubator-spark pull request: [WIP] SPARK-1067: Default log4j init...

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/560#issuecomment-34567772 Ok merged in master & branch-0.9.

[GitHub] incubator-spark pull request: SPARK-1066: Add developer scripts to...

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/565#issuecomment-34567423 Ok I merged it. Thanks!

[GitHub] incubator-spark pull request: new MLlib documentation for optimiza...

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/563#issuecomment-34565306 Maybe you can create a patch, and apply that patch directly on master? The changes look pretty easy to apply to me, so it shouldn't be too hard.

[GitHub] incubator-spark pull request: new MLlib documentation for optimiza...

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/563#issuecomment-34559900 (And then submit a new PR and close this one)

[GitHub] incubator-spark pull request: new MLlib documentation for optimiza...

2014-02-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/563#issuecomment-34559894 To play it safe, you can always create a new branch and do the rebase there so it doesn't change your current branch.