[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
I think pip installing locally to connect to a cluster is something people
would want to use - but if people disagree then cutting down the artifacts
makes sense and I'm happy to do that in a follow up PR.

On Mon, Nov 28, 2016 at 8:04 PM Reynold Xin 
wrote:

> The use case I intended for this one was to allow installing Spark on a
> laptop against some local file system, not using pip as a distributed
> mechanism across a cluster. For that use case I think limiting this to one
> would make more sense. This doubles now the release preparation time, 
which
> already takes a while.
>
> —
> You are receiving this because you were mentioned.
>
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[rxin]

Github user rxin commented on the issue:

https://github.com/apache/spark/pull/15659
  
The use case I intended for this one was to allow installing Spark on a 
laptop against some local file system, not using pip as a distributed mechanism 
across a cluster. For that use case I think limiting this to one would make 
more sense. This doubles now the release preparation time, which already takes 
a while.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
For the same reasons we have this in the current download, we could of
course limit this to a smaller subset of you want.

On Mon, Nov 28, 2016 at 5:29 PM Reynold Xin 
wrote:

> @holdenk  why are we introducing so many
> tarballs for Python?
>
>
> [   ] pyspark-2.1.0+hadoop2.3.tar.gz  2016-11-28 20:17165M
> [TXT] pyspark-2.1.0+hadoop2.3.tar.gz.asc  2016-11-28 20:17
490 
> [   ] pyspark-2.1.0+hadoop2.3.tar.gz.md5  2016-11-28 20:17
81  
> [   ] pyspark-2.1.0+hadoop2.3.tar.gz.sha  2016-11-28 20:17
272 
> [   ] pyspark-2.1.0+hadoop2.4.tar.gz  2016-11-28 20:17166M
> [TXT] pyspark-2.1.0+hadoop2.4.tar.gz.asc  2016-11-28 20:17
490 
> [   ] pyspark-2.1.0+hadoop2.4.tar.gz.md5  2016-11-28 20:17
81  
> [   ] pyspark-2.1.0+hadoop2.4.tar.gz.sha  2016-11-28 20:17
272 
> [   ] pyspark-2.1.0+hadoop2.6.tar.gz  2016-11-28 20:17170M
> [TXT] pyspark-2.1.0+hadoop2.6.tar.gz.asc  2016-11-28 20:17
490 
> [   ] pyspark-2.1.0+hadoop2.6.tar.gz.md5  2016-11-28 20:17
81  
> [   ] pyspark-2.1.0+hadoop2.6.tar.gz.sha  2016-11-28 20:17
272 
> [   ] pyspark-2.1.0+hadoop2.7.tar.gz  2016-11-28 20:17172M
> [TXT] pyspark-2.1.0+hadoop2.7.tar.gz.asc  2016-11-28 20:17
490 
> [   ] pyspark-2.1.0+hadoop2.7.tar.gz.md5  2016-11-28 20:17
81  
> [   ] pyspark-2.1.0+hadoop2.7.tar.gz.sha  2016-11-28 20:17
272 
> [   ] pyspark-2.1.0+without.hadoop.tar.gz 2016-11-28 20:17
103M
> [TXT] pyspark-2.1.0+without.hadoop.tar.gz.asc 2016-11-28 20:17
490 
> [   ] pyspark-2.1.0+without.hadoop.tar.gz.md5 2016-11-28 20:17
123 
> [   ] pyspark-2.1.0+without.hadoop.tar.gz.sha 2016-11-28 20:17
292
>
> Can we just have one with the latest Hadoop version?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[rxin]

Github user rxin commented on the issue:

https://github.com/apache/spark/pull/15659
  
@holdenk why are we introducing so many tarballs for Python? 


```

[   ]   pyspark-2.1.0+hadoop2.3.tar.gz  2016-11-28 20:17165M
 
[TXT]   pyspark-2.1.0+hadoop2.3.tar.gz.asc  2016-11-28 20:17
490  
[   ]   pyspark-2.1.0+hadoop2.3.tar.gz.md5  2016-11-28 20:17
81   
[   ]   pyspark-2.1.0+hadoop2.3.tar.gz.sha  2016-11-28 20:17
272  
[   ]   pyspark-2.1.0+hadoop2.4.tar.gz  2016-11-28 20:17166M
 
[TXT]   pyspark-2.1.0+hadoop2.4.tar.gz.asc  2016-11-28 20:17
490  
[   ]   pyspark-2.1.0+hadoop2.4.tar.gz.md5  2016-11-28 20:17
81   
[   ]   pyspark-2.1.0+hadoop2.4.tar.gz.sha  2016-11-28 20:17
272  
[   ]   pyspark-2.1.0+hadoop2.6.tar.gz  2016-11-28 20:17170M
 
[TXT]   pyspark-2.1.0+hadoop2.6.tar.gz.asc  2016-11-28 20:17
490  
[   ]   pyspark-2.1.0+hadoop2.6.tar.gz.md5  2016-11-28 20:17
81   
[   ]   pyspark-2.1.0+hadoop2.6.tar.gz.sha  2016-11-28 20:17
272  
[   ]   pyspark-2.1.0+hadoop2.7.tar.gz  2016-11-28 20:17172M
 
[TXT]   pyspark-2.1.0+hadoop2.7.tar.gz.asc  2016-11-28 20:17
490  
[   ]   pyspark-2.1.0+hadoop2.7.tar.gz.md5  2016-11-28 20:17
81   
[   ]   pyspark-2.1.0+hadoop2.7.tar.gz.sha  2016-11-28 20:17
272  
[   ]   pyspark-2.1.0+without.hadoop.tar.gz 2016-11-28 20:17
103M 
[TXT]   pyspark-2.1.0+without.hadoop.tar.gz.asc 2016-11-28 20:17
490  
[   ]   pyspark-2.1.0+without.hadoop.tar.gz.md5 2016-11-28 20:17
123  
[   ]   pyspark-2.1.0+without.hadoop.tar.gz.sha 2016-11-28 20:17
292
```

Can we just have one with the latest Hadoop version? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[minrk]

Github user minrk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Awesome, thanks @holdenk!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Yay! :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[JoshRosen]

Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/15659
  
Agreed, so I'm going to cherry-pick this into branch-2.1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
@JoshRosen thank you so much! That's awesome. I've had some off-line chats 
about why I think we should merge this for 2.1 as well (hopefully @rxin can 
chime in here with his thoughts) but I think the general argument is that:

I'd like to argue for its inclusion in 2.1 as it seems to follow both the 
official and defacto conventions for inclusion:
Namely this PR existed before branch cut and is primarily additive, and the 
idea it's self has been proposed since 2014 in various forms. Also defacto 
we've added much of the streaming functionality between RCs during 2.0 which 
was even larger than this.

If we want to publish to PyPI for 2.2 (which I think would be awesome) - it 
would probably be best for us to have a chance to have the pip installable 
artifacts available in a prior release so that we can have an opportunity to 
work out any unexpected kinks before going ahead with full PyPI publishing 
which might reach a different audience.

Thank you so much for everyone involved @rgbkrk, @nchammas , @jhlch, 
@felixcheung, @viirya, @davies , @minrk, @mariusvniekerk - this is the PR I'm 
most excited about getting in recently :dancer: :smile: :package: 
:dancing_women: :four_leaf_clover: :+1: :tea: :cat: :cat2: :joy_cat: 
:smile_cat: :snake: #2ammytimesorryfortheemoji #pip4life


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[rgbkrk]

Github user rgbkrk commented on the issue:

https://github.com/apache/spark/pull/15659
  
🙇 Thank you everyone!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[JoshRosen]

Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/15659
  
I'm going to merge this to master now and will look into cherry-picking to 
branch-2.1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
cc @jkbradley / @davies / @JoshRosen / @rxin - it seems like this should be 
an ok state to merge if one of you has the bandwidth? I'd really like to get 
this in before we cut RC1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
I'v added the extra exclusion rules for extra safety in packaging  :) cc 
@rxin if this looks good to merge?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68669/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68669 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68669/consoleFull)**
 for PR 15659 at commit 
[`e139855`](https://github.com/apache/spark/commit/e1398552469288de3829eb889fec0de2ba568f15).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68669 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68669/consoleFull)**
 for PR 15659 at commit 
[`e139855`](https://github.com/apache/spark/commit/e1398552469288de3829eb889fec0de2ba568f15).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Ping @joshrosen / @rxin :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
@JoshRosen - yes since we ship the jars with them we want people to be able 
to install the correct package for the hadoop distribution they are running 
with/against.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68638/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68638 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68638/consoleFull)**
 for PR 15659 at commit 
[`d753d80`](https://github.com/apache/spark/commit/d753d8094e5483e0da7577a85c0c2ed182de3e34).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68638 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68638/consoleFull)**
 for PR 15659 at commit 
[`d753d80`](https://github.com/apache/spark/commit/d753d8094e5483e0da7577a85c0c2ed182de3e34).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[JoshRosen]

Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/15659
  
> I think we will want to keep the release-build tagging so that the 
artifacts are correct for the different hadoop versions.

Ah, so the idea is that we'll make separate PyPi artifacts for each Hadoop 
version and want that to be reflected in the version shown in Python? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68619/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68620/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68619 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68619/consoleFull)**
 for PR 15659 at commit 
[`dd243a2`](https://github.com/apache/spark/commit/dd243a2aeed82921efad484ea382a251002b1fa7).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68620 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68620/consoleFull)**
 for PR 15659 at commit 
[`df5a3f9`](https://github.com/apache/spark/commit/df5a3f92731329ca19f28ca52e6511258156268e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68620 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68620/consoleFull)**
 for PR 15659 at commit 
[`df5a3f9`](https://github.com/apache/spark/commit/df5a3f92731329ca19f28ca52e6511258156268e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68619 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68619/consoleFull)**
 for PR 15659 at commit 
[`dd243a2`](https://github.com/apache/spark/commit/dd243a2aeed82921efad484ea382a251002b1fa7).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
@JoshRosen: So I took a look at doing that on the flight and I think we 
will want to keep the release-build tagging so that the artifacts are correct 
for the different hadoop versions, but we can use release-tags for when we are 
updating the source. Just writing it down to make sure we are on the same page.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Thanks @JoshRosen I'll take a crack at that this weekend :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[JoshRosen]

Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/15659
  
I just tested this out end-to-end on my Mac and it worked great. I have 
only one minor concern that I'd like to fix regarding the `version.py` file:

Today, the `./dev/create-release/release-tag.sh` script has special logic 
for updating the `R` package version files by editing them in-place using 
`sed`; I think it would make sense to move the `sed` logic that you currently 
have in `release-build.sh` into here so that all rewriting of files takes place 
outside of the actual build process itself when creating release (in other 
words, we'll be running `release-build.sh` on a git tag whose files already 
have the right versions edited in rather than doing it at compile-time). I 
think this is important in order to have all of the versions be consistent in 
the git tag itself, otherwise I think we'll hit problems when folks are 
reviewing the release artifacts.

Therefore, I think we should move those `sed` lines right below 
https://github.com/apache/spark/blob/1386fd28daf798bf152606f4da30a36223d75d18/dev/create-release/release-tag.sh#L77

In addition, it might be nice to do a `from pyspark.version import 
__version__` in `python/pyspark/__init__.py` so that end users can then write 
`import pyspark; pyspark.__version__`, which is a nice idiom that works for 
many other Python packages.

Besides this one piece of feedback, I think the rest looks good to go.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68570/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68570 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68570/consoleFull)**
 for PR 15659 at commit 
[`3345eb9`](https://github.com/apache/spark/commit/3345eb9bd3f0a165ae1d20d76e5d36e88e5512cd).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
and or @rxin?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68570 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68570/consoleFull)**
 for PR 15659 at commit 
[`3345eb9`](https://github.com/apache/spark/commit/3345eb9bd3f0a165ae1d20d76e5d36e88e5512cd).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
So @joshrosen or @davies now that I've updated the shell scripts does this 
look good to go?

I'll make a follow up to maybe add shellcheck to our standard linting if 
it's something we want to enforce more generally.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Jenkins retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[minrk]

Github user minrk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Sounds great!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68557/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68557 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68557/consoleFull)**
 for PR 15659 at commit 
[`3345eb9`](https://github.com/apache/spark/commit/3345eb9bd3f0a165ae1d20d76e5d36e88e5512cd).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68557 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68557/consoleFull)**
 for PR 15659 at commit 
[`3345eb9`](https://github.com/apache/spark/commit/3345eb9bd3f0a165ae1d20d76e5d36e88e5512cd).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
So the simplest thing sounds like updating the error message, for now I'll 
avoid changing the maven build since that might have some unintended collateral 
effects and this change is currently primarily additive.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68554/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68554 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68554/consoleFull)**
 for PR 15659 at commit 
[`2904998`](https://github.com/apache/spark/commit/290499848cf9d65fa35e8488f11531091a014081).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68553/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68553 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68553/consoleFull)**
 for PR 15659 at commit 
[`587c0eb`](https://github.com/apache/spark/commit/587c0ebc71ff587e48ec8e71728069d155694dd2).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[minrk]

Github user minrk commented on the issue:

https://github.com/apache/spark/pull/15659
  
> Any ideas on how we could follow up and support that install pattern?

To go all the way to the two commands I expected to work based on Python 
tradition:

```bash
./build/mvn -DskipTests clean package
cd python && pip install .
```

I believe you would need to copy (or hardlink or symlink-to-absolute-path) 
the jars into the Python directory during `build\mvn`, since `pip` copies the 
whole `python` directory to a temp location before running your setup.py.

One step short of this would be to allow the `stage-jars` to be an explicit 
step, rather than a temporary staging in setup.py that cleans up after itself. 
Then you would have one small extra step and the full command would be:

```bash
./build/mvn
cd python
python setup.py stage_jars # ./stage-jars.sh, whatever you prefer
pip install .
```

and the error message when jars are missing could point to the extra step.

Yet another option, and the least work from what you have now, could be to 
put the full sequence in your missing-jars error message:

> If you are installing pyspark from the spark source, you must build spark 
and run sdist first:
> ```
> ./build/mvn -DskipTests clean package
> cd python
> python setup.py sdist
> pip install dist/*.tar.gz
> ```

I think most people wouldn't discover that without very explicit help, 
because they would need to know about both the temporary symlinks and pip's 
tempdir to work it out.

Not being able to do `pip install .` is a bit odd for a Python package, as 
that is the standard command for installing any Python package from source 
(it's not `python setup.py install` anymore). But if you provide something 
copy/pasteable when people do try it, you are in pretty good shape.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Thanks @JoshRosen for taking time to review this. The factoured out fix is 
at https://github.com/apache/spark/pull/15860 :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68554 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68554/consoleFull)**
 for PR 15659 at commit 
[`2904998`](https://github.com/apache/spark/commit/290499848cf9d65fa35e8488f11531091a014081).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
@minrk thanks :) Any ideas on how we could follow up and support that 
install pattern?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68553 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68553/consoleFull)**
 for PR 15659 at commit 
[`587c0eb`](https://github.com/apache/spark/commit/587c0ebc71ff587e48ec8e71728069d155694dd2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
@joshrosen Is `shellcheck` intended to be part of our standard linting?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[minrk]

Github user minrk commented on the issue:

https://github.com/apache/spark/pull/15659
  
I downloaded this branch and ran this to test (macOS, Python 3.5):

```bash
build/mvn -DskipTests clean package
cd python
python setup.py sdist
pip install dist/*.tar.gz
```

and everything worked great. Yay!

What didn't work (which I tried first) was:

pip install .

which is what I'm used to for installing a Python package from source, 
because pip copies the python package directory to a temp location before 
running the installation. Since this happens before symlinking the jars, they 
don't get found. I'm not sure how big of a deal that is, since it won't affect 
stable-release installation, but it is idiomatic for Python projects.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Yay! Thanks @JoshRosen :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[JoshRosen]

Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/15659
  
I can aim to take a review pass this afternoon or this weekend.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Maybe @jkbradley ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68532/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68532 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68532/consoleFull)**
 for PR 15659 at commit 
[`210c9d4`](https://github.com/apache/spark/commit/210c9d4fef8c5aca7eddd00b8dc0aa9f7b37bb18).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Thanks @davies :)

I think at this point we've probably got things pretty well covered 
(@viirya & @nchammas also have done a lot of review and @minrk  took a look at 
the initial PR / direction for sanity before hand).

If you or @rxin would feel comfortable merging this that would be great - 
but if you want another person to take a look at some more details any 
suggestions on who you have in mind for that?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68532 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68532/consoleFull)**
 for PR 15659 at commit 
[`210c9d4`](https://github.com/apache/spark/commit/210c9d4fef8c5aca7eddd00b8dc0aa9f7b37bb18).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[davies]

Github user davies commented on the issue:

https://github.com/apache/spark/pull/15659
  
This looks good to me in general, but did not check all the details.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Also like if there is any testing or anything I can do or coordinate to get 
done that would help y'all feel comfortable with this please let me know :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
So @davies does this look ok? It would be nice to get this in before RC1 is 
cut so that it can have the pip artifacts for people to test as well. (cc 
@rxin).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68490/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68490 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68490/consoleFull)**
 for PR 15659 at commit 
[`8d74672`](https://github.com/apache/spark/commit/8d7467205bc352107aea3149193352efff10a3a7).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68490 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68490/consoleFull)**
 for PR 15659 at commit 
[`8d74672`](https://github.com/apache/spark/commit/8d7467205bc352107aea3149193352efff10a3a7).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68447/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68447 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68447/consoleFull)**
 for PR 15659 at commit 
[`7001f90`](https://github.com/apache/spark/commit/7001f90889b38d214bd0b279bd5cd6aa343bc4b3).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68447 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68447/consoleFull)**
 for PR 15659 at commit 
[`7001f90`](https://github.com/apache/spark/commit/7001f90889b38d214bd0b279bd5cd6aa343bc4b3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Jenkins retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68419/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68419 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68419/consoleFull)**
 for PR 15659 at commit 
[`7001f90`](https://github.com/apache/spark/commit/7001f90889b38d214bd0b279bd5cd6aa343bc4b3).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
Jenkins retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68419 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68419/consoleFull)**
 for PR 15659 at commit 
[`7001f90`](https://github.com/apache/spark/commit/7001f90889b38d214bd0b279bd5cd6aa343bc4b3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68407/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68407 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68407/consoleFull)**
 for PR 15659 at commit 
[`7001f90`](https://github.com/apache/spark/commit/7001f90889b38d214bd0b279bd5cd6aa343bc4b3).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
re-ping to @davies & @rxin 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68407 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68407/consoleFull)**
 for PR 15659 at commit 
[`7001f90`](https://github.com/apache/spark/commit/7001f90889b38d214bd0b279bd5cd6aa343bc4b3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68372/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68372 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68372/consoleFull)**
 for PR 15659 at commit 
[`49fc6db`](https://github.com/apache/spark/commit/49fc6db5152a0d567de0b0fa66f87a53268be8bd).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68370/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15659
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68370 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68370/consoleFull)**
 for PR 15659 at commit 
[`23109a4`](https://github.com/apache/spark/commit/23109a44c25b06ec3fefc4938978cf9d76f13e7c).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
ping @davies / @rxin / @mateiz - I'm getting a flight tomorrow if you 
happen to have any comments beforehand I can try and address them during the 
flight :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68372 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68372/consoleFull)**
 for PR 15659 at commit 
[`49fc6db`](https://github.com/apache/spark/commit/49fc6db5152a0d567de0b0fa66f87a53268be8bd).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15659
  
**[Test build #68370 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68370/consoleFull)**
 for PR 15659 at commit 
[`23109a4`](https://github.com/apache/spark/commit/23109a44c25b06ec3fefc4938978cf9d76f13e7c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

[holdenk]

Github user holdenk commented on the issue:

https://github.com/apache/spark/pull/15659
  
@davies - no this does also address SPARK-18129 take a look at the changes 
inside of `dev/create-release/release-build.sh` :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org