[GitHub] spark issue #14342: [SPARK-16685] Remove audit-release scripts.

2016-07-25 Thread pwendell
Github user pwendell commented on the issue: https://github.com/apache/spark/pull/14342 LGTM - I added these and I think they are dead code right now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10359] Enumerate dependencies in a file...

2015-12-25 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/10461#discussion_r48445168 --- Diff: dev/deps/spark-deps-hadoop-2.4 --- @@ -0,0 +1,185 @@ +JavaEWAH-0.3.2.jar --- End diff -- yes these are automatically generated

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/10395#issuecomment-166069156 LGTM (I downloaded your PR and did some grepping to make sure there are no references). One other thing that occured to me is someone could easily create a package

[GitHub] spark pull request: Update branch-1.6 for 1.6.0 release

2015-12-15 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/10317#issuecomment-164923664 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: Fix thread pools that cannot cache tasks in Wo...

2015-12-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/10108#issuecomment-161418651 Hey @marmbrus is actually managing the RC - it just has my name on it because some automated tooling uses my account. Ping @marmbrus. --- If your project is set up

[GitHub] spark pull request: [SPARK-12101][Core]Fix thread pools that canno...

2015-12-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/10108#issuecomment-161419476 BTW @srowen, some protocol for announcing these is probably a good idea to avoid races. I think we haven't suffered from races in the past, but mostly out of luck

[GitHub] spark pull request: [SPARK-3580][CORE] Add Consistent Method To Ge...

2015-12-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9767#issuecomment-161203603 Yeah I think it's fine to pull in - but do it quickly because an RC will go out very soon! --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-11903] Remove --skip-java-test

2015-11-23 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9924#issuecomment-159118994 Jenkins, retest this please. This LGTM - I think it's good to simply remove it. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-2365] Add IndexedRDD, an efficient upda...

2015-11-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1297#issuecomment-158851260 @josephlijia this feature has moved into a Spark package. If you want to file an issue report it's best to do it here: https://github.com/amplab/spark

[GitHub] spark pull request: [SPARK-11732] Removes some MiMa false positive...

2015-11-17 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9697#issuecomment-157469133 Yep, this LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11081] Shade Jersey and javax.rs.ws

2015-11-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9615#issuecomment-155962272 Regarding testing, the best way is to inspect the contents of the jar to make sure that the shaded version is inlined. If Spark code uses the shaded dependency

[GitHub] spark pull request: [SPARK-11081] Shade Jersey and javax.rs.ws

2015-11-10 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9615#discussion_r44503873 --- Diff: pom.xml --- @@ -2165,6 +2166,9 @@ org.eclipse.jetty:jetty-security org.eclipse.jetty:jetty-util

[GitHub] spark pull request: [SPARK-6152] Use shaded ASM5 to support closur...

2015-11-09 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9512#issuecomment-155334662 This looks fine (i.e. LGTM). However, we could also look into actually shading asm ourselves in our published artifacts, similar to how we now shade jetty and other

[GitHub] spark pull request: [SPARK-9818] Re-enable Docker tests for JDBC d...

2015-11-09 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9503#issuecomment-155341829 I only reviewed the build changes, but they look good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-7841][BUILD] Stop using retrieveManaged...

2015-11-09 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9575#issuecomment-155341204 It's hard for me to rule out that there is _no_ other reason lib_managed is used at present. I audited all the uses of it I could find in the codebase and it appears

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-05 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r44055168 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/EmittedRecordsDStream.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43980418 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala --- @@ -350,6 +349,18 @@ class PairDStreamFunctions[K, V

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43980438 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/TrackStateSpec.scala --- @@ -0,0 +1,111 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43980731 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/EmittedRecordsDStream.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43980502 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala --- @@ -350,6 +349,18 @@ class PairDStreamFunctions[K, V

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43981556 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/State.scala --- @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43982348 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala --- @@ -350,6 +349,18 @@ class PairDStreamFunctions[K, V

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43981921 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/State.scala --- @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9256#issuecomment-153975680 I took a broad pass on the public API's and left comments throughout, mostly around naming rather than core structure. What would really be helpful for me is filling

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43981603 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/State.scala --- @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43981909 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/EmittedRecordsDStream.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43982371 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala --- @@ -350,6 +349,18 @@ class PairDStreamFunctions[K, V

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43982459 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala --- @@ -350,6 +349,18 @@ class PairDStreamFunctions[K, V

[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...

2015-11-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/9256#discussion_r43982032 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/TrackStateSpec.scala --- @@ -0,0 +1,111 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-11-03 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-153469556 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-11-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-153259490 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-9926] [SPARK-10340] [SQL] Use S3 bulk l...

2015-11-02 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/8512#discussion_r43717724 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkS3Util.scala --- @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request: [SPARK-9926] [SPARK-10340] [SQL] Use S3 bulk l...

2015-11-02 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/8512#discussion_r43717704 --- Diff: core/pom.xml --- @@ -40,6 +40,11 @@ ${avro.mapred.classifier} + com.amazonaws + aws-java-sdk

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-11-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-153252094 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-11-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-153252357 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [test-maven][test-hadoop1.0][SPARK-11236][CORE...

2015-11-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-153203911 Hey @calvinjia seems okay to merge since this is just triggering other failures. Can you explain more though how this works around the mima issue - the patch seems

[GitHub] spark pull request: [test-maven][test-hadoop1.0][SPARK-11236][CORE...

2015-11-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-153208299 Got it- thanks I can merge it then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [test-hadoop1.0][SPARK-11236][CORE] Update Tac...

2015-11-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152892982 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...

2015-11-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152891750 @calvinjia can you add "[test-hadoop1.0]" to the title of this PR and then retest it? That will run the tests with hadoop 1. See more info here:

[GitHub] spark pull request: [test-hadoop1.0][SPARK-11236][CORE] Update Tac...

2015-11-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152923143 Hm - can you try also adding [test-maven] - might be better to test with maven. On Sun, Nov 1, 2015 at 9:40 PM, Calvin Jia <notificati...@github.com>

[GitHub] spark pull request: [test-maven][test-hadoop1.0][SPARK-11236][CORE...

2015-11-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152929052 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...

2015-10-29 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-152415593 @haoyuan hey HY - can you not merge build related patches without asking for feedback from one of the build maintainers (me or @srowen)? This patch makes changes

[GitHub] spark pull request: SPARK-2533 - Add locality levels (with tasks c...

2015-10-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9117#issuecomment-149127284 Yeah this was my thought - could we provide a summary in the stage page rather than in the stage index.I do see how an aggregated summary is significantly more

[GitHub] spark pull request: SPARK-2533 - Add locality levels (with tasks c...

2015-10-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9117#issuecomment-149125876 ping @kayousterhout to me it's the normal issue - sure it's useful for some cases, but is it worth putting in the index page? It could be better to just put a locality

[GitHub] spark pull request: [SPARK-11110] [Build] Remove transient annotat...

2015-10-14 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9126#issuecomment-148231844 ping @marmbrus for any thoughts. But I think removing them makes sense. If someone makes it a val later they will have to reason about whether it should be transient

[GitHub] spark pull request: SPARK-1537 publisher-side code and tests

2015-10-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/8744#discussion_r41951480 --- Diff: yarn/pom.xml --- @@ -164,6 +164,92 @@ + + --- End diff -- Can this say "The YARN applic

[GitHub] spark pull request: [SPARK-10930] Adds max task duration to all st...

2015-10-12 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-147493632 I see the underlying problem posed in the JIRA - it's difficult to assess duration since it currently includes the time spent waiting on dependent stages. However

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8931#issuecomment-146293403 SGTM. On Wed, Oct 7, 2015 at 11:55 AM, Reynold Xin <notificati...@github.com> wrote: > (and data size should include all the data,

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-07 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/8931#discussion_r41426589 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregate.scala --- @@ -69,6 +72,9 @@ case class TungstenAggregate

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8931#issuecomment-146285580 I think we want to have concise and consistent names here across all the metrics. Here is my proposal for naming: ``` input rows output rows

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8931#issuecomment-146288685 The reason I like accumulated memory is that it's something that should be roughly constant over multiple runs of a workload so people can get a sense of how much data

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-07 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/8931#discussion_r41426412 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregate.scala --- @@ -69,6 +72,9 @@ case class TungstenAggregate

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8931#issuecomment-146289643 BTW - one alternative would be to create an accumulator that tracks max, min, median, and total and then have it display nicely in two lines. For instance

[GitHub] spark pull request: [SPARK-10833] [BUILD] Inline, organize BSD/MIT...

2015-09-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8919#issuecomment-143837144 Hey Sean - looks good to me, but can't claim to be nearly as deep as you are on this stuff! --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: Update branch-1.5 for 1.5.1 release.

2015-09-23 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8890#issuecomment-142783383 @rxin I think you should upgrade R/pkg/DESCRIPTION - otherwise LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: Update version to 1.6.0-SNAPSHOT.

2015-09-15 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8350#issuecomment-140562180 No problem - I should have caught it earlier. Hopefully people didn't spin too many cycles on this. On Tue, Sep 15, 2015 at 12:56 AM, Sean Owen <notific

[GitHub] spark pull request: [SPARK-10511][BUILD] Reset git repository befo...

2015-09-15 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8774#issuecomment-140508440 This is perfect, thanks. LGTM. Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Update version to 1.6.0-SNAPSHOT.

2015-09-14 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8350#issuecomment-140213492 I think if you update previousVersion in MimaBuild.scala many of these should go away. I'm happy to look at the error output after doing that. --- If your project

[GitHub] spark pull request: Update version to 1.6.0-SNAPSHOT.

2015-09-14 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8350#issuecomment-140231316 I tested it locally, you'll need something like this: ``` version match { +case v if v.startsWith("1.6") => +

[GitHub] spark pull request: Update version to 1.6.0-SNAPSHOT.

2015-09-14 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8350#issuecomment-140286995 Hm - this is a test file that is in the source folder, so MIMA is complaining about changes. The error message gives you a filter you can add to the MIMA file

[GitHub] spark pull request: [SPARK-10300] [build] [tests] Add support for ...

2015-09-14 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8437#issuecomment-140288092 LGTM - I've modified that script recently enough to be familiar with how it works. This seems like a good approach and could be useful for us in the future

[GitHub] spark pull request: [SPARK-10411][SQL]Move visualization above exp...

2015-09-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8570#issuecomment-137170215 For the details link - can you use the standard drop down icon we use in other places? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-10411][SQL]Move visualization above exp...

2015-09-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8570#issuecomment-137221455 Yes - I mean the triangle image used there and in a few other places (such as showing more metrics). --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...

2015-08-30 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/7878#discussion_r38283921 --- Diff: dev/run-tests.py --- @@ -227,11 +228,32 @@ def build_spark_documentation(): os.chdir(SPARK_HOME) +def get_zinc_port

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-08-30 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-136268566 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-08-30 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-136255396 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-08-30 Thread pwendell
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/8531 [SPARK-10359] Enumerate Spark's dependencies in a file and diff against it for new pull requests DON'T MERGE ME - TESTING ON JENKINS You can merge this pull request into a Git repository

[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...

2015-08-30 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-136250200 Thanks for looking at this @vanzin. I do agree it would be a lot nicer to base things on comments, but because the comment stream isn't available as meta-data

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-08-30 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-136271740 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...

2015-08-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-135871143 Okay this seems to be passing now - any thoughts @JoshRosen or @vanzin? IMO this would be really nice since we can test build changes with either build, before

[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...

2015-08-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-135870939 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...

2015-08-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-135582563 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...

2015-08-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-135637236 ping @aarondav --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...

2015-08-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-135635256 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-135144791 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-135144862 K - I just sent a hotfix to up the timeout. On Wed, Aug 26, 2015 at 9:51 AM, Marcelo Vanzin notificati...@github.com wrote: I'm working on fixing

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-134867614 Marcelo you will need to change the timeout in the code itself for it to increase from 175 On Aug 25, 2015 11:24 PM, UCB AMPLab notificati...@github.com wrote

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-135105365 I still don't understand - why not fix the issue in a separate PR, get this passing and then merge this? It will then benefit the other PR's also that are facing

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-135103392 Okay then - I would do that separately, get this PR passing, then merge it. It is not good to merge a PR that deterministically fails jenkins. Have we done

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-135200381 Great - looks good! On Aug 26, 2015 3:53 PM, Marcelo Vanzin notificati...@github.com wrote: Yay! — Reply to this email directly or view

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-25 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-134683004 Does this PR increase test time in some way? Just wondering why this would consistently timeout when others don't. On Tue, Aug 25, 2015 at 10:43 AM, Marcelo

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-25 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-134732350 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-24 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-134422382 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-9284] [tests] Allow all tests to run wi...

2015-08-24 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7629#issuecomment-134422378 Yeah sounds good - might be good to let it run one more time just to be sure it's not affecting jenkins somehow. On Mon, Aug 24, 2015 at 4:40 PM, Marcelo

[GitHub] spark pull request: [SPARK-6196] [BUILD] Remove MapR profiles in f...

2015-08-24 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8338#issuecomment-134449642 Yes this LGTM - these are outdated and I don't even think MapR is advising their customers to use these. They are asking people to use hadoop-provided, which

[GitHub] spark pull request: SPARK-7726: Add import so Scaladoc doesn't fai...

2015-08-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8095#issuecomment-129950381 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-1517] Refactor release scripts to facil...

2015-08-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7411#issuecomment-129950164 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-1517] Refactor release scripts to facil...

2015-08-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7411#issuecomment-129954901 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-1517] Refactor release scripts to facil...

2015-08-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7411#issuecomment-129976266 test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...

2015-08-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-130156023 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-1517] Refactor release scripts to facil...

2015-08-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7411#issuecomment-130155218 Okay will merge this - I've been keeping things in a separate repo and it's much better to have it in the upstream in case others want to modify it. --- If your

[GitHub] spark pull request: [SPARK-1517] Refactor release scripts to facil...

2015-08-10 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7411#issuecomment-129708771 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-7726 Add import so Scaladoc doesn't fail...

2015-08-10 Thread pwendell
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/8095 SPARK-7726 Add import so Scaladoc doesn't fail. This is another import needed so Scala 2.11 doc generation doesn't fail. See SPARK-7726 for more detail. I tested this locally and the 2.11

[GitHub] spark pull request: SPARK-7726: Add import so Scaladoc doesn't fai...

2015-08-10 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/8095#issuecomment-129713711 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-1517] Refactor release scripts to facil...

2015-08-10 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/7411#discussion_r36684757 --- Diff: dev/create-release/release-build.sh --- @@ -0,0 +1,320 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-1517] Refactor release scripts to facil...

2015-08-10 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/7411#discussion_r36684821 --- Diff: dev/create-release/release-build.sh --- @@ -0,0 +1,320 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-1517] Refactor release scripts to facil...

2015-08-10 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7411#issuecomment-129605050 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-9593] [SQL] Fixes Hadoop shims loading

2015-08-05 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7929#issuecomment-127889147 LGTM - feel free to merge, as it is really taking a toll on our tests right now. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-9593] [SQL] Fixes Hadoop shims loading

2015-08-05 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/7929#discussion_r36273191 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/ClientWrapper.scala --- @@ -62,6 +64,52 @@ private[hive] class ClientWrapper

[GitHub] spark pull request: [SPARK-9593] [SQL] Fixes Hadoop shims loading

2015-08-05 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/7929#issuecomment-127890665 do feel free to get the comment thing hashed out with @srowen. My time zone is approaching bed time, so I have to sign off. Would be nice to get something

  1   2   3   4   5   6   7   8   9   10   >