[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-23 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r134847542 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2040,4 +2040,83 @@ class SQLQuerySuite extends

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-23 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r134845802 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2040,4 +2040,83 @@ class SQLQuerySuite extends

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16774 **[Test build #81047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81047/testReport)** for PR 16774 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-23 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r134845628 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1509,4 +1509,84 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-23 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r134844461 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,67 @@ +/* + *

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-23 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r134843683 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,67 @@ +/* + *

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16774 **[Test build #81046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81046/testReport)** for PR 16774 at commit

[GitHub] spark pull request #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default ...

2017-08-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19021 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-23 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r134842881 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1509,4 +1509,84 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-23 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r134842831 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1509,4 +1509,84 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #18645: [SPARK-14280][BUILD][WIP] Update change-version.sh and p...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18645 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81037/ Test FAILed. ---

[GitHub] spark issue #18645: [SPARK-14280][BUILD][WIP] Update change-version.sh and p...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18645 **[Test build #81037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81037/testReport)** for PR 18645 at commit

[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19021 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18645: [SPARK-14280][BUILD][WIP] Update change-version.sh and p...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18645 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-23 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r134842420 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -359,6 +359,18 @@ case class

[GitHub] spark issue #16087: [SPARK-18656][SQL] Memory efficient implementation of th...

2017-08-23 Thread poplav
Github user poplav commented on the issue: https://github.com/apache/spark/pull/16087 @barrybecker4 Following the trail of this issue...What is the solution for many columns rather than the workaround for chunking? Thanks! --- If your project is set up for it, you can reply to this

[GitHub] spark issue #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarchy to ma...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18958 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarchy to ma...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18958 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81038/ Test PASSed. ---

[GitHub] spark issue #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarchy to ma...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18958 **[Test build #81038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81038/testReport)** for PR 18958 at commit

[GitHub] spark issue #18964: [SPARK-21701][CORE] Enable RPC client to use ` SO_RCVBUF...

2017-08-23 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18964 > I also notice that Spark RPC by default uses java native serialization, even a verifying endpoint exist or not request would cost 1K of payload size, not to mention some other real logic

[GitHub] spark issue #18281: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18281 **[Test build #81045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81045/testReport)** for PR 18281 at commit

[GitHub] spark issue #19027: [SPARK-19165][PYTHON][SQL] PySpark APIs using columns as...

2017-08-23 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19027 I like this approach @HyukjinKwon :D! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18368: [SPARK-21102][SQL] Refresh command is too aggressive in ...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18368 Set. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18368: [SPARK-21102][SQL] Refresh command is too aggressive in ...

2017-08-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18368 Hi, @gatorsmile . Could you set @aokolnychyi as an `Assignee` and `Fix Versions` at https://issues.apache.org/jira/browse/SPARK-21102? Thanks! --- If your project is set up for it, you

[GitHub] spark issue #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading remote r...

2017-08-23 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18962 Sorry, you are right we talked about both there. I was just thinking of the hadoop not supporting it case. Still agree to split out to another jira. --- If your project is set up for it, you

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19018 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19018 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81041/ Test PASSed. ---

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19018 **[Test build #81041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81041/testReport)** for PR 19018 at commit

[GitHub] spark issue #18317: [SPARK-21113][CORE] Read ahead input stream to amortize ...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18317 **[Test build #81044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81044/testReport)** for PR 18317 at commit

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-23 Thread sitalkedia
Github user sitalkedia closed the pull request at: https://github.com/apache/spark/pull/18317 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-23 Thread sitalkedia
GitHub user sitalkedia reopened a pull request: https://github.com/apache/spark/pull/18317 [SPARK-21113][CORE] Read ahead input stream to amortize disk IO cost … Profiling some of our big jobs, we see that around 30% of the time is being spent in reading the spill files from

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19016 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81040/ Test PASSed. ---

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19016 **[Test build #81040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81040/testReport)** for PR 19016 at commit

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19016 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading remote r...

2017-08-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18962 > If we don't then we need to conditionalize and only do for for HDFS or other hadoop supported filesystems. I thought in the previous discussion we chose to treat that as a separate issue

[GitHub] spark issue #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Param Val...

2017-08-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/17849 Thanks @holdenk! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18281: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18281 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18281: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18281 **[Test build #81043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81043/testReport)** for PR 18281 at commit

[GitHub] spark issue #18281: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18281 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81043/ Test FAILed. ---

[GitHub] spark issue #18281: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18281 **[Test build #81043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81043/testReport)** for PR 18281 at commit

[GitHub] spark issue #18317: [SPARK-21113][CORE] Read ahead input stream to amortize ...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18317 **[Test build #81042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81042/testReport)** for PR 18317 at commit

[GitHub] spark issue #18317: [SPARK-21113][CORE] Read ahead input stream to amortize ...

2017-08-23 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/18317 @jiangxb1987 - Made a change to wrap the read ahead input stream around the compressed input stream so that we can amortize the cost of decompression as well --- If your project is set up for

[GitHub] spark pull request #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading r...

2017-08-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18962#discussion_r134821799 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -330,19 +332,21 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-23 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r134821455 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,288 @@ +/* + * Licensed under the Apache License,

[GitHub] spark issue #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading remote r...

2017-08-23 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18962 also I just realized on our original discussion of "Spark on YARN doesn't support upload remote resources from http, https or ftp server" I think the feature to download without this change

[GitHub] spark pull request #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading r...

2017-08-23 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/18962#discussion_r134820252 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -330,19 +332,21 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19018 **[Test build #81041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81041/testReport)** for PR 19018 at commit

[GitHub] spark pull request #18581: [SPARK-21289][SQL][ML] Supports custom line separ...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18581#discussion_r134817030 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1195,4 +1196,46 @@ class CSVSuite extends

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19016 **[Test build #81040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81040/testReport)** for PR 19016 at commit

[GitHub] spark pull request #19016: [SPARK-21805][SPARKR] Disable R vignettes code on...

2017-08-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19016#discussion_r134816715 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -27,6 +27,17 @@ vignette: > limitations under the License. --> +```{r setup,

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19018 jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateO...

2017-08-23 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19029#discussion_r134816423 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -438,6 +438,10 @@ private[ml] object SummaryBuilderImpl extends Logging {

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/19016 Sure - change LGTM. Lets see if @HyukjinKwon has any more comments ? If not we can merge to master, branch-2.2 and then do some more tests. --- If your project is set up for it, you can

[GitHub] spark pull request #19016: [SPARK-21805][SPARKR] Disable R vignettes code on...

2017-08-23 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/19016#discussion_r134815709 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -27,6 +27,17 @@ vignette: > limitations under the License. --> +```{r setup,

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19016 It likely best to back port this to branch-2.2 so the test can pass with matching API in a released jar. Then we can see everything passing (except having to hand edit the version

[GitHub] spark pull request #18581: [SPARK-21289][SQL][ML] Supports custom line separ...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18581#discussion_r134814393 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -32,7 +32,9 @@ import

[GitHub] spark pull request #18581: [SPARK-21289][SQL][ML] Supports custom line separ...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18581#discussion_r134813986 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMOptions.scala --- @@ -41,11 +41,15 @@ private[libsvm] class

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81036/ Test FAILed. ---

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81036/testReport)** for PR 18581 at commit

[GitHub] spark pull request #18940: [SPARK-21501] Change CacheLoader to limit entries...

2017-08-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18940 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18940: [SPARK-21501] Change CacheLoader to limit entries based ...

2017-08-23 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18940 +1, I'm going to merge as it appears all comments addressed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #19022: [Spark-21807][SQL]The getAliasedConstraints funct...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19022#discussion_r134807546 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionSetSuite.scala --- @@ -210,4 +210,13 @@ class

[GitHub] spark pull request #19013: [SPARK-21728][core] Allow SparkSubmit to use Logg...

2017-08-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19013#discussion_r134807428 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -202,7 +209,7 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19022 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81039/ Test FAILed. ---

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19022 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19022 **[Test build #81039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81039/testReport)** for PR 19022 at commit

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19022 **[Test build #81039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81039/testReport)** for PR 19022 at commit

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19022 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarchy to ma...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18958 **[Test build #81038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81038/testReport)** for PR 18958 at commit

[GitHub] spark pull request #18266: [SPARK-20427][SQL] Read JDBC table use custom sch...

2017-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18266#discussion_r134797984 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -768,6 +769,25 @@ object JdbcUtils extends

[GitHub] spark pull request #18910: [SPARK-21694][MESOS] Support Mesos CNI network la...

2017-08-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/18910#discussion_r134795818 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/config.scala --- @@ -70,4 +70,19 @@ package object config {

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/19016 Ah I see. Yeah the failed tests makes sense. We can also try to submit a custom tar.gz to r-hub to test it with the PDF and a different version number ? --- If your project is set up for it, you

[GitHub] spark issue #18910: [SPARK-21694][MESOS] Support Mesos CNI network labels

2017-08-23 Thread susanxhuynh
Github user susanxhuynh commented on the issue: https://github.com/apache/spark/pull/18910 @srowen Sean, would you like to review this PR? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #19027: [SPARK-19165][PYTHON][SQL] PySpark APIs using columns as...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19027 Cool looks to me like a very reasonable fix. Could we perhaps add a test for numpy.bool_ or numpy.float_ (that it should fail)? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #18730: [SPARK-21527][CORE] Use buffer limit in order to ...

2017-08-23 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18730#discussion_r134786732 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -63,6 +65,19 @@ private[spark] class ChunkedByteBuffer(var chunks:

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19016 I think that is caused by the way I build the source package - I use the option --no-build-vignettes so the source package I test with doesn't have the PDF in it Similarly for the

[GitHub] spark issue #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarchy to ma...

2017-08-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18958 LGTM except some minor comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-23 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18730 It would be great to benchmark this improvement, otherwise we are not sure there is no regression. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarch...

2017-08-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18958#discussion_r134780644 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java --- @@ -505,18 +511,12 @@ public void

[GitHub] spark pull request #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarch...

2017-08-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18958#discussion_r134780277 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java --- @@ -95,19 +81,32 @@ public void close() {

[GitHub] spark pull request #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarch...

2017-08-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18958#discussion_r134779475 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java --- @@ -307,64 +306,69 @@ public void update(int ordinal,

[GitHub] spark issue #19016: [SPARK-21805][SPARKR] Disable R vignettes code on Window...

2017-08-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/19016 Thanks @felixcheung ! Are the warnings about the missing PDF unavoidable ? I see something like ``` checking package vignettes in 'inst/doc' ... WARNING Package vignette without

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81033/ Test FAILed. ---

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81033/testReport)** for PR 18875 at commit

[GitHub] spark issue #18645: [SPARK-14280][BUILD][WIP] Update change-version.sh and p...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18645 **[Test build #81037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81037/testReport)** for PR 18645 at commit

[GitHub] spark issue #18964: [SPARK-21701][CORE] Enable RPC client to use ` SO_RCVBUF...

2017-08-23 Thread neoremind
Github user neoremind commented on the issue: https://github.com/apache/spark/pull/18964 @cloud-fan would you take a look of the PR, the update is very simple. Thanks very much! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81036/testReport)** for PR 18581 at commit

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19022 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19022 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81035/ Test FAILed. ---

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19022 **[Test build #81035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81035/testReport)** for PR 19022 at commit

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19022 **[Test build #81035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81035/testReport)** for PR 19022 at commit

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19022 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81034/ Test FAILed. ---

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19022 cc @gatorsmile @HyukjinKwon May you help to trigger Jenkins tests? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81034/testReport)** for PR 18581 at commit

[GitHub] spark issue #19022: [Spark-21807][SQL]The getAliasedConstraints function in ...

2017-08-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19022 We can edit the PR title and description to show the changes clearly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #19022: [Spark-21807][SQL]The getAliasedConstraints funct...

2017-08-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19022#discussion_r134761549 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ExpressionSet.scala --- @@ -67,6 +67,12 @@ class ExpressionSet protected(

<    1   2   3   4   5   >