Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35962751
Also one other thing, can you look through the Java programming guide and
the Quick Start guide in the `docs/` folder and make sure they have valid Java
code now
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019661
--- Diff:
core/src/main/java/org/apache/spark/api/java/function/DoubleFunction.java ---
@@ -15,13 +15,10 @@
* limitations under the License.
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35962439
Hey Prashant, I did another pass through this and it looks quite good, but
I have a few comments:
* Is there a way to add the Java 8 test projects to the
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019552
--- Diff:
core/src/main/java/org/apache/spark/api/java/function/PairFlatMapFunction.java
---
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019494
--- Diff:
streaming/src/test/java/org/apache/spark/streaming/JavaAPISuite.java ---
@@ -718,7 +717,7 @@ public void testPairFlatMap() {
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019414
--- Diff: project/SparkBuild.scala ---
@@ -116,10 +121,10 @@ object SparkBuild extends Build {
lazy val examples = Project("examples", file("exa
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019378
--- Diff: project/SparkBuild.scala ---
@@ -88,6 +88,11 @@ object SparkBuild extends Build {
}
lazy val hadoopClient = if (hadoopVersion.s
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019362
--- Diff: project/SparkBuild.scala ---
@@ -88,6 +88,11 @@ object SparkBuild extends Build {
}
lazy val hadoopClient = if (hadoopVersion.s
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019296
--- Diff: java8-tests/src/test/java/org/apache/spark/Java8APISuite.java ---
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundatio
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019275
--- Diff: java8-tests/src/test/java/org/apache/spark/Java8APISuite.java ---
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundatio
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019117
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -117,19 +118,19 @@ trait JavaRDDLike[T, This <: JavaRDDLike[T, This]]
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019102
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -82,15 +82,16 @@ trait JavaRDDLike[T, This <: JavaRDDLike[T, This]]
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019064
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaPairRDD.scala
---
@@ -699,8 +699,14 @@ object JavaPairRDD {
}
implicit
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10019048
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaPairRDD.scala
---
@@ -699,8 +699,14 @@ object JavaPairRDD {
}
implicit
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10018267
--- Diff:
core/src/main/java/org/apache/spark/api/java/function/DoubleFunction.java ---
@@ -15,13 +15,10 @@
* limitations under the License.
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r10018257
--- Diff:
core/src/main/java/org/apache/spark/api/java/function/DoubleFunction.java ---
@@ -15,13 +15,10 @@
* limitations under the License.
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35959525
Hey Prashant, one other thing, can you add the Java 8 tests in the Maven
build too? Is there a way to only optionally run them there?
---
If your project is set
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35886773
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35886774
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12830/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35884716
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35884717
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35884071
Did another pass and things look okay, atleast from my side.
---
If your project is set up for it, you can reply to this email and have your
reply appear on G
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35877170
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12828/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35877169
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35875417
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35875416
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35874707
I messed up rebase :( and had to flatten all the commits.
---
If your project is set up for it, you can reply to this email and have your
reply appear on Git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35873916
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35873917
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12827/
---
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35873699
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35873697
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35826379
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post y
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35826380
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12817/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35825552
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35825553
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post yo
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35825377
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-p
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35798351
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35798352
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12807/
---
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35797496
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35797497
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35732710
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35732711
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12801/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35728699
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35728700
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35707482
These implicits are used in JavaPairRDD itself, JavaRDDLike and
JavaDStreamLike. The reason they were used is the number of occurrence for this
trivial wrap i
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35704205
Also, can you tell me where those implicits in JavaPairRDD are used? Can't
we manually do the conversions there? At the very least, the implicits should
be privat
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35704145
Hey Prashant, this looks pretty good at first glance. Can you also create a
Java 8 version of the Streaming suite?
---
If your project is set up for it, you can
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35586860
Hey Matei,
I feel this is better than before in overall. There is one thing I was not
very sure about is putting a couple of implicits in JavaPairRDD
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35521749
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35521750
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12781/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35514615
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35514616
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35501417
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35501419
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12779/
---
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35496756
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35496755
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35496679
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12777/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35496676
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user ScrapCodes commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/539#discussion_r9862681
--- Diff: project/build.properties ---
@@ -14,4 +14,4 @@
# See the License for the specific language governing permissions and
# limitatio
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35491848
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35491849
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35439917
Hey Prashant, I've looked at this more, and one final thing I'd like to do
is to see if we can reduce the number of methods and classes people have to
deal with.
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35420086
I just meant don't make the "of" methods public, since they're confusing
for users to see.
---
If your project is set up for it, you can reply to this email and
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35352183
Yes, we can remove those "of" methods, but we would still need to wrap the
lambdas. Since closure cleaner won't work with out that. So as an alternative
we ca
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35322176
@ScrapCodes so in this version, do we still need the "of" methods or can we
remove those?
---
If your project is set up for it, you can reply to this email and h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35249277
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12744/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35249276
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35247680
I hope nothing is missed out, @mateiz take a look again ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitH
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35247536
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35247537
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35242604
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12743/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35242603
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35240585
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35240586
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35218897
BTW a usage example for my test project is
https://github.com/mateiz/java8-test/blob/master/src/main/java/test/Main.java.
This is what I'd like our code to look l
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35218879
BTW, I've looked into this myself, and created a short project at
https://github.com/mateiz/java8-test to show how an RDD-like API might work in
Java. To make a l
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35206774
What exactly is the problem with the closure cleaner? In the Java API
(JavaRDD and such), we can always wrap the closure into a class in Scala. Why
can't we take
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35181038
>Another problem is with API, for lambda expression we would need Interfaces
with apply and at the moment we have a lot of them as abstract classes with
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35180854
Hey Matei,
I have already tried that option, trouble is not with types but with
closure cleaner which does getResourceAsStream on a lambda expressi
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35172268
See also
http://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html#target-types-and-method-arguments
for example.
If your project is set up for
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35172231
BTW according to
http://www.oracle.com/technetwork/articles/java/lambda-1984522.html for
example, lambdas should work with method overloading, in which case Java
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35168965
Guys, I haven't looked at the full discussion on SPARK-964 yet, but is
there no way to avoid the "of" wrapper? The wrapper seems pretty awkward, and
doesn't make
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-34732078
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12676/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-34732076
Merged build finished.
84 matches
Mail list logo
