[GitHub] spark issue #18388: [SPARK-21175] Reject OpenBlocks when memory shortage on ...

2017-06-27 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18388 Haven't looked at the path in detail yet. High level questions/thoughts. So you say the memory usage is by the netty chunks, so my assumption is this is during the actual transfer? failing

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18436 **[Test build #78699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78699/testReport)** for PR 18436 at commit

[GitHub] spark issue #18431: [SPARK-21224][R] Specify a schema by using a DDL-formatt...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18431 **[Test build #78700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78700/testReport)** for PR 18431 at commit

[GitHub] spark issue #18406: [SPARK-21195] Automatically register new metrics from so...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18406 **[Test build #78696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78696/testReport)** for PR 18406 at commit

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17758 **[Test build #78695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78695/testReport)** for PR 17758 at commit

[GitHub] spark issue #18418: [SPARK-19104][SQL] Lambda variables in ExternalMapToCata...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18418 **[Test build #78701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78701/testReport)** for PR 18418 at commit

[GitHub] spark pull request #18368: [SPARK-21102][SQL] Make refresh resource command ...

2017-06-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18368#discussion_r124299127 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -230,11 +230,24 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18436 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18418: [SPARK-19104][SQL] Lambda variables in ExternalMapToCata...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18418 **[Test build #78698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78698/testReport)** for PR 18418 at commit

[GitHub] spark issue #18418: [SPARK-19104][SQL] Lambda variables in ExternalMapToCata...

2017-06-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18418 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18388: [SPARK-21175] Reject OpenBlocks when memory shortage on ...

2017-06-27 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/18388 Thanks a lot for quick reply :) Yes, this patch doesn't guarantee avoiding the OOM on shuffle service when all reducers are opening the blocks at the same time. But we can alleviate this

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18436 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124212354 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Instance.scala --- @@ -27,3 +27,28 @@ import org.apache.spark.ml.linalg.Vector * @param

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124272854 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -798,77 +798,184 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124258933 --- Diff: mllib/src/test/scala/org/apache/spark/ml/optim/IterativelyReweightedLeastSquaresSuite.scala --- @@ -43,7 +43,7 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124268761 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -798,77 +798,160 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124261785 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -339,15 +364,16 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124271108 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -578,6 +578,79 @@ class

[GitHub] spark issue #18431: [SPARK-21224][R] Specify a schema by using a DDL-formatt...

2017-06-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18431 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124272947 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -798,77 +798,184 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124281406 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -961,14 +1007,30 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124235464 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -303,6 +317,17 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124265572 --- Diff: mllib/src/test/scala/org/apache/spark/ml/optim/IterativelyReweightedLeastSquaresSuite.scala --- @@ -169,29 +169,29 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124236849 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -440,13 +479,13 @@ object

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124234628 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -168,6 +179,9 @@ private[regression] trait

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124233224 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -406,6 +437,14 @@ object

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124269044 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -798,77 +798,184 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124262306 --- Diff: mllib/src/test/scala/org/apache/spark/ml/optim/IterativelyReweightedLeastSquaresSuite.scala --- @@ -156,7 +156,7 @@ class

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17758 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18368: [SPARK-21102][SQL] Make refresh resource command less ag...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18368 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18368: [SPARK-21102][SQL] Make refresh resource command less ag...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18368 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78703/ Test FAILed. ---

[GitHub] spark issue #18368: [SPARK-21102][SQL] Make refresh resource command less ag...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18368 **[Test build #78703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78703/testReport)** for PR 18368 at commit

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18436 **[Test build #78697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78697/testReport)** for PR 18436 at commit

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18436 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78697/ Test FAILed. ---

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-06-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r124237318 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -944,15 +983,22 @@ class

[GitHub] spark issue #18418: [SPARK-19104][SQL] Lambda variables in ExternalMapToCata...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18418 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78698/ Test FAILed. ---

[GitHub] spark issue #18418: [SPARK-19104][SQL] Lambda variables in ExternalMapToCata...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18418 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17758: [SPARK-20460][SPARK-21144][SQL] Make it more cons...

2017-06-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r124293746 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -185,8 +183,21 @@ case class DataSource( }

[GitHub] spark issue #18406: [SPARK-21195] Automatically register new metrics from so...

2017-06-27 Thread robert3005
Github user robert3005 commented on the issue: https://github.com/apache/spark/pull/18406 This is to facilitate using metrics in libraries that integrate in spark. Since spark already has metric reporting infrastructure and lets you register sources with it it seems natural extension

[GitHub] spark issue #18388: [SPARK-21175] Reject OpenBlocks when memory shortage on ...

2017-06-27 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/18388 >So an alternative to this is limiting the number of blocks each reducer is fetching at once Is it relevant to `spark.reducer.maxSizeInFlight` ? Breaking `OpenBlocks` into more

[GitHub] spark issue #18410: [SPARK-20971][SS] purge metadata log in FileStreamSource

2017-06-27 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/18410 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18436: [SPARK-20073][SQL] Prints an explicit warning mes...

2017-06-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18436#discussion_r124302536 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -299,7 +308,7 @@ class Column(val expr: Expression) extends Logging {

[GitHub] spark issue #18410: [SPARK-20971][SS] purge metadata log in FileStreamSource

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18410 **[Test build #78704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78704/testReport)** for PR 18410 at commit

[GitHub] spark issue #18368: [SPARK-21102][SQL] Make refresh resource command less ag...

2017-06-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18368 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18369: [SPARK-21155][WEBUI] Add (? running tasks) into Spark UI...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18369 **[Test build #78707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78707/testReport)** for PR 18369 at commit

[GitHub] spark issue #18431: [SPARK-21224][R] Specify a schema by using a DDL-formatt...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18431 **[Test build #78711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78711/testReport)** for PR 18431 at commit

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18439 cc @holdenk and @shaneknapp, what do you think about this change? I at least see `Running PySpark packaging tests` is passed for three builds above. --- If your project is set up for

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17758 **[Test build #78717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78717/testReport)** for PR 17758 at commit

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78717/ Test FAILed. ---

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18426 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78718/ Test FAILed. ---

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18439 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18426 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18439 So, is this resolved now without this chsnge? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17758 **[Test build #78709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78709/testReport)** for PR 17758 at commit

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78709/ Test PASSed. ---

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-27 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/15821 okie dokie. how about this i install pyarrow in the py3k conda environment right now... once that's done, we can remove the pyarrow test from run-pip-tests and add it to the regular

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18439 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78715/ Test FAILed. ---

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18439 I'm also in favour of backing out the original PR until we can resolve the issue, but I'll defer to @JoshRosen on this. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18439 If @shaneknapp has the bandwidth to update conda (or is ok with me updating conda) I would feel more comfortable doing that than turning of SSL verification. --- If your project is set up for it,

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18439 **[Test build #78712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78712/testReport)** for PR 18439 at commit

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18426 **[Test build #78718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78718/testReport)** for PR 18426 at commit

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/18426 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/18439 thanks for closing this... @holdenk and/or @bryancutler will be updating the tests and removing the call to install pyarrow from run-pip-tests some time this afternoon. On Tue,

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-27 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/15821 anyways: installing pyarrow right now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15821 I can do the test updating assuming that @BryanCutler is traveling. I've got a webinar this afternoon but I can do it after I'm done with that. Also I don't think its the wild card issue

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-27 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/15821 yeah, i think you're right. however, upgrading to a new version of conda on a live environment does indeed scare me a little bit. :) w/the new jenkins, i'll have a staging server

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-27 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15821 Thanks for doing this @shaneknapp and @holdenk! I'm about to hop on a plane but should be online later this afternoon. I can switch out the pyarrow tests then if it still needs to be

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78712/ Test PASSed. ---

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18426 **[Test build #78722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78722/testReport)** for PR 18426 at commit

[GitHub] spark issue #18176: [SPARK-20952] Make TaskContext an InheritableTheadLocal

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18176 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78719/ Test FAILed. ---

[GitHub] spark issue #17293: [SPARK-19950][SQL] Fix to ignore nullable when df.load()...

2017-06-27 Thread JasonMWhite
Github user JasonMWhite commented on the issue: https://github.com/apache/spark/pull/17293 This issue causes a lot of headaches for us when picking up parquet datasets. To get around this issue, we write the schema alongside the parquet files in a side-band, and then when loading,

[GitHub] spark issue #18176: [SPARK-20952] Make TaskContext an InheritableTheadLocal

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18176 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18176: [SPARK-20952] Make TaskContext an InheritableTheadLocal

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18176 **[Test build #78719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78719/testReport)** for PR 18176 at commit

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18426 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18426 **[Test build #78720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78720/testReport)** for PR 18426 at commit

[GitHub] spark issue #18369: [SPARK-21155][WEBUI] Add (? running tasks) into Spark UI...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18369 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78707/ Test PASSed. ---

[GitHub] spark issue #18369: [SPARK-21155][WEBUI] Add (? running tasks) into Spark UI...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18369 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18431: [SPARK-21224][R] Specify a schema by using a DDL-formatt...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18431 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18431: [SPARK-21224][R] Specify a schema by using a DDL-formatt...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18431 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78711/ Test PASSed. ---

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-27 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/15821 done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-27 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/15821 ``` (py3k)-bash-4.1$ pip install pyarrow Requirement already satisfied: pyarrow in /home/anaconda/envs/py3k/lib/python3.4/site-packages Requirement already satisfied: six>=1.0.0 in

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18439 **[Test build #78715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78715/testReport)** for PR 18439 at commit

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18439 **[Test build #78714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78714/testReport)** for PR 18439 at commit

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18439 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78714/ Test FAILed. ---

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/18439 i've installed pyarrow 0.4.0 on the workers in the py3k conda environment, and would not like to disable SSL. i do not approve of this PR, sorry. --- If your project is set up for

[GitHub] spark issue #18439: [HOTFIX] Disable SSL verification to install pyarrow wit...

2017-06-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18439 Ok nvm. I assume we don't want to fix it in this way and going to unblock other PRs soon. I am closing this. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #18439: [HOTFIX] Disable SSL verification to install pyar...

2017-06-27 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/18439 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18426 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78720/ Test FAILed. ---

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78724/ Test PASSed. ---

[GitHub] spark issue #18426: [SPARK-21216][SS] Hive strategies missed in Structured S...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18426 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12414: [SPARK-14657][SPARKR][ML] RFormula w/o intercept should ...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12414 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18443: [SPARK-21231] Disable installing pyarrow during run pip ...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18443 **[Test build #78733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78733/testReport)** for PR 18443 at commit

[GitHub] spark issue #12414: [SPARK-14657][SPARKR][ML] RFormula w/o intercept should ...

2017-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12414 **[Test build #78742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78742/testReport)** for PR 12414 at commit

[GitHub] spark issue #12414: [SPARK-14657][SPARKR][ML] RFormula w/o intercept should ...

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12414 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78742/ Test FAILed. ---

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78744/ Test FAILed. ---

[GitHub] spark pull request #18268: [SPARK-21054] [SQL] Reset Command support reset s...

2017-06-27 Thread ericsahit
Github user ericsahit commented on a diff in the pull request: https://github.com/apache/spark/pull/18268#discussion_r124445231 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/SetCommand.scala --- @@ -150,12 +150,21 @@ object SetCommand { * This

  1   2   3   4   5   6   7   8   >