[jira] [Commented] (SPARK-21397) Maven shade plugin adding dependency-reduced-pom.xml to base directory

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085248#comment-16085248 ] Apache Spark commented on SPARK-21397: -- User 'zuotingbing' has created a pull reques

[jira] [Assigned] (SPARK-21397) Maven shade plugin adding dependency-reduced-pom.xml to base directory

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21397: Assignee: (was: Apache Spark) > Maven shade plugin adding dependency-reduced-pom.xml t

[jira] [Assigned] (SPARK-21397) Maven shade plugin adding dependency-reduced-pom.xml to base directory

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21397: Assignee: Apache Spark > Maven shade plugin adding dependency-reduced-pom.xml to base dire

[jira] [Commented] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

2017-07-12 Thread indraneel rao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085244#comment-16085244 ] indraneel rao commented on SPARK-21391: --- [~kiszk], unfortunately it might not be po

[jira] [Created] (SPARK-21398) Data on Rest end point is not updating after first fetch

2017-07-12 Thread Shubham Gupta (JIRA)
Shubham Gupta created SPARK-21398: - Summary: Data on Rest end point is not updating after first fetch Key: SPARK-21398 URL: https://issues.apache.org/jira/browse/SPARK-21398 Project: Spark Is

[jira] [Commented] (SPARK-21360) Spark failing to query SQL Server. Query contains a column having space in where clause

2017-07-12 Thread dharani_sugumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085238#comment-16085238 ] dharani_sugumar commented on SPARK-21360: - feroz khan: Are you executing this in

[jira] [Created] (SPARK-21397) Maven shade plugin adding dependency-reduced-pom.xml to base directory

2017-07-12 Thread zuotingbing (JIRA)
zuotingbing created SPARK-21397: --- Summary: Maven shade plugin adding dependency-reduced-pom.xml to base directory Key: SPARK-21397 URL: https://issues.apache.org/jira/browse/SPARK-21397 Project: Spark

[jira] [Assigned] (SPARK-20090) Add StructType.fieldNames to Python API

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20090: Assignee: Apache Spark > Add StructType.fieldNames to Python API > ---

[jira] [Commented] (SPARK-20090) Add StructType.fieldNames to Python API

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085224#comment-16085224 ] Apache Spark commented on SPARK-20090: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-20090) Add StructType.fieldNames to Python API

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20090: Assignee: (was: Apache Spark) > Add StructType.fieldNames to Python API >

[jira] [Created] (SPARK-21396) Spark Hive Thriftserver doesn't return UDT field

2017-07-12 Thread Haopu Wang (JIRA)
Haopu Wang created SPARK-21396: -- Summary: Spark Hive Thriftserver doesn't return UDT field Key: SPARK-21396 URL: https://issues.apache.org/jira/browse/SPARK-21396 Project: Spark Issue Type: Impr

[jira] [Comment Edited] (SPARK-20703) Add an operator for writing data out

2017-07-12 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085017#comment-16085017 ] Liang-Chi Hsieh edited comment on SPARK-20703 at 7/13/17 5:28 AM: -

[jira] [Commented] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085160#comment-16085160 ] Apache Spark commented on SPARK-21376: -- User 'jerryshao' has created a pull request

[jira] [Assigned] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21376: Assignee: Apache Spark > Token is not renewed in yarn client process in cluster mode > ---

[jira] [Assigned] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21376: Assignee: (was: Apache Spark) > Token is not renewed in yarn client process in cluster

[jira] [Comment Edited] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

2017-07-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085099#comment-16085099 ] Kazuaki Ishizaki edited comment on SPARK-21391 at 7/13/17 3:42 AM:

[jira] [Commented] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

2017-07-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085099#comment-16085099 ] Kazuaki Ishizaki commented on SPARK-21391: -- [~hyukjin.kwon] I think that [SPARK

[jira] [Updated] (SPARK-21395) Spark SQL hive-thriftserver doesn't register operation log before execute sql statement

2017-07-12 Thread Chaozhong Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaozhong Yang updated SPARK-21395: --- Description: In HiveServer2, TFetchResultsReq has a member which is named as `fetchType`. If

[jira] [Created] (SPARK-21395) Spark SQL hive-thriftserver doesn't register operation log before execute sql statement

2017-07-12 Thread Chaozhong Yang (JIRA)
Chaozhong Yang created SPARK-21395: -- Summary: Spark SQL hive-thriftserver doesn't register operation log before execute sql statement Key: SPARK-21395 URL: https://issues.apache.org/jira/browse/SPARK-21395

[jira] [Updated] (SPARK-21297) Add count in 'JDBC/ODBC Server' page.

2017-07-12 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21297: --- Description: 1.Add count about 'Session Statistics' and 'SQL Statistics' in 'JDBC/ODBC Server

[jira] [Commented] (SPARK-20703) Add an operator for writing data out

2017-07-12 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085017#comment-16085017 ] Liang-Chi Hsieh commented on SPARK-20703: - Thanks [~ste...@apache.org] for voicin

[jira] [Updated] (SPARK-21297) Add count in 'JDBC/ODBC Server' page.

2017-07-12 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21297: --- Summary: Add count in 'JDBC/ODBC Server' page. (was: Add State in 'Session Statistics' table

[jira] [Resolved] (SPARK-18646) ExecutorClassLoader for spark-shell does not honor spark.executor.userClassPathFirst

2017-07-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18646. - Resolution: Fixed Assignee: Min Shen Fix Version/s: 2.3.0 > ExecutorClassLoader f

[jira] [Updated] (SPARK-21377) Jars specified with --jars or --packages are not added into AM's system classpath

2017-07-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21377: Summary: Jars specified with --jars or --packages are not added into AM's system classpath (was: A

[jira] [Commented] (SPARK-21377) Add a new configuration to extend AM classpath in yarn client mode

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084929#comment-16084929 ] Apache Spark commented on SPARK-21377: -- User 'jerryshao' has created a pull request

[jira] [Commented] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

2017-07-12 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084886#comment-16084886 ] Ruslan Dautkhanov commented on SPARK-13534: --- [~bryanc], thanks for the feedback

[jira] [Comment Edited] (SPARK-21392) Unable to infer schema when loading large Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084837#comment-16084837 ] Stuart Reynolds edited comment on SPARK-21392 at 7/12/17 10:27 PM:

[jira] [Created] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

2017-07-12 Thread Zahra (JIRA)
Zahra created SPARK-21393: - Summary: spark (pyspark) crashes unpredictably when using show() or toPandas() Key: SPARK-21393 URL: https://issues.apache.org/jira/browse/SPARK-21393 Project: Spark Issu

[jira] [Commented] (SPARK-21392) Unable to infer schema when loading large Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084837#comment-16084837 ] Stuart Reynolds commented on SPARK-21392: - I've simplified the example a little m

[jira] [Updated] (SPARK-21392) Unable to infer schema when loading large Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart Reynolds updated SPARK-21392: Description: The following boring code works {code:none} response = "mi_or_chd_5" sc = get

[jira] [Commented] (SPARK-12559) Cluster mode doesn't work with --packages

2017-07-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084776#comment-16084776 ] Marcelo Vanzin commented on SPARK-12559: [~skonto], could you look at whether the

[jira] [Comment Edited] (SPARK-12559) Cluster mode doesn't work with --packages

2017-07-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084776#comment-16084776 ] Marcelo Vanzin edited comment on SPARK-12559 at 7/12/17 9:49 PM: --

[jira] [Comment Edited] (SPARK-21392) Unable to infer schema when loading large Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084837#comment-16084837 ] Stuart Reynolds edited comment on SPARK-21392 at 7/12/17 10:27 PM:

[jira] [Updated] (SPARK-21392) Unable to infer schema when loading large Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart Reynolds updated SPARK-21392: Description: The following boring code works {code:none} response = "mi_or_chd_5" sc = get

[jira] [Commented] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084821#comment-16084821 ] Hyukjin Kwon commented on SPARK-21392: -- I think this is unrelated with that JIRA ^ t

[jira] [Comment Edited] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

2017-07-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084689#comment-16084689 ] Hyukjin Kwon edited comment on SPARK-21393 at 7/12/17 10:12 PM: ---

[jira] [Assigned] (SPARK-21394) Reviving broken callable objects in UDF in PySpark

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21394: Assignee: Apache Spark > Reviving broken callable objects in UDF in PySpark >

[jira] [Commented] (SPARK-21394) Reviving broken callable objects in UDF in PySpark

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084787#comment-16084787 ] Apache Spark commented on SPARK-21394: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-21394) Reviving broken callable objects in UDF in PySpark

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21394: Assignee: (was: Apache Spark) > Reviving broken callable objects in UDF in PySpark > -

[jira] [Commented] (SPARK-21374) Reading globbed paths from S3 into DF doesn't work if filesystem caching is disabled

2017-07-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084764#comment-16084764 ] Shixiong Zhu commented on SPARK-21374: -- Yeah, org.apache.spark.deploy.SparkHadoopUti

[jira] [Comment Edited] (SPARK-21378) Spark Poll timeout when specific offsets are passed

2017-07-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084772#comment-16084772 ] Shixiong Zhu edited comment on SPARK-21378 at 7/12/17 9:49 PM:

[jira] [Commented] (SPARK-21378) Spark Poll timeout when specific offsets are passed

2017-07-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084772#comment-16084772 ] Shixiong Zhu commented on SPARK-21378: -- bq. Digging deeper shows that there's an ass

[jira] [Reopened] (SPARK-12559) Cluster mode doesn't work with --packages

2017-07-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-12559: > Cluster mode doesn't work with --packages > - > >

[jira] [Updated] (SPARK-21378) Spark Poll timeout when specific offsets are passed

2017-07-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21378: - Component/s: (was: Spark Core) DStreams > Spark Poll timeout when specific o

[jira] [Resolved] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

2017-07-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21391. -- Resolution: Cannot Reproduce I can't reproduce this against the current master as described in

[jira] [Created] (SPARK-21394) Reviving broken callable objects in UDF in PySpark

2017-07-12 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-21394: Summary: Reviving broken callable objects in UDF in PySpark Key: SPARK-21394 URL: https://issues.apache.org/jira/browse/SPARK-21394 Project: Spark Issue Type

[jira] [Commented] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084697#comment-16084697 ] Hyukjin Kwon commented on SPARK-21392: -- Would you mind running {{outcome.show}} and

[jira] [Updated] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

2017-07-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21393: - Affects Version/s: (was: 2.2.1) > spark (pyspark) crashes unpredictably when using show() or

[jira] [Commented] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084707#comment-16084707 ] Dongjoon Hyun commented on SPARK-21392: --- It seems to be a different issue. SPARK-16

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

2017-07-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084689#comment-16084689 ] Hyukjin Kwon commented on SPARK-21393: -- Would you mind sharing your codes? I want to

[jira] [Commented] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084601#comment-16084601 ] Stuart Reynolds commented on SPARK-21392: - Done. I'm simply trying to build a ta

[jira] [Updated] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart Reynolds updated SPARK-21392: Description: The following boring code works {code:none} response = "mi_or_chd_5"

[jira] [Updated] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart Reynolds updated SPARK-21392: Description: The following boring code works {code:none} response = "mi_or_chd_5"

[jira] [Updated] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart Reynolds updated SPARK-21392: Description: The following boring code works {code:python} response = "mi_or_chd_5"

[jira] [Updated] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart Reynolds updated SPARK-21392: Description: The following boring code works {code:python} response = "mi_or_chd_5"

[jira] [Updated] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart Reynolds updated SPARK-21392: Description: The following boring code works {code:python} response = "mi_or_chd_5"

[jira] [Commented] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084583#comment-16084583 ] Sean Owen commented on SPARK-21392: --- Can you format this so it's readable? I don't unde

[jira] [Issue Comment Deleted] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators such as OneVsRest

2017-07-12 Thread Ajay Saini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Saini updated SPARK-21221: --- Comment: was deleted (was: Note: In order for python persistence of OneVsRest inside a CrossValidato

[jira] [Assigned] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2017-07-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14280: -- Assignee: Josh Rosen > Update change-version.sh and pom.xml to add Scala 2.12 profiles > -

[jira] [Assigned] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2017-07-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14280: -- Assignee: (was: Josh Rosen) > Update change-version.sh and pom.xml to add Scala 2.12 profi

[jira] [Assigned] (SPARK-14650) Compile Spark REPL for Scala 2.12

2017-07-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14650: -- Assignee: (was: Josh Rosen) > Compile Spark REPL for Scala 2.12 >

[jira] [Resolved] (SPARK-14438) Cross-publish Breeze for Scala 2.12

2017-07-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-14438. Resolution: Fixed > Cross-publish Breeze for Scala 2.12 > --- > >

[jira] [Assigned] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2017-07-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14280: -- Assignee: (was: Josh Rosen) > Update change-version.sh and pom.xml to add Scala 2.12 profi

[jira] [Resolved] (SPARK-14519) Cross-publish Kafka for Scala 2.12

2017-07-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-14519. Resolution: Fixed > Cross-publish Kafka for Scala 2.12 > -- > >

[jira] [Comment Edited] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084460#comment-16084460 ] Saisai Shao edited comment on SPARK-21376 at 7/12/17 6:31 PM: -

[jira] [Commented] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084460#comment-16084460 ] Saisai Shao commented on SPARK-21376: - I'm referrring to o.a.s.deploy.yarn.Client thi

[jira] [Commented] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-12 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084458#comment-16084458 ] Thomas Graves commented on SPARK-21376: --- so you are referring to the org.apache.spa

[jira] [Updated] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators such as OneVsRest

2017-07-12 Thread Ajay Saini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Saini updated SPARK-21221: --- Summary: CrossValidator and TrainValidationSplit Persist Nested Estimators such as OneVsRest (was: C

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

2017-07-12 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084451#comment-16084451 ] Ye Zhou commented on SPARK-18085: - I want to add my own testing experience with the codes

[jira] [Created] (SPARK-21392) Unable to infer schema when loading Parquet file

2017-07-12 Thread Stuart Reynolds (JIRA)
Stuart Reynolds created SPARK-21392: --- Summary: Unable to infer schema when loading Parquet file Key: SPARK-21392 URL: https://issues.apache.org/jira/browse/SPARK-21392 Project: Spark Issue

[jira] [Commented] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

2017-07-12 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084390#comment-16084390 ] Bryan Cutler commented on SPARK-13534: -- Hi [~tagar], the {{ArrowSerializer}} doesn't

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

2017-07-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084369#comment-16084369 ] Marcelo Vanzin commented on SPARK-18085: [~kanjilal] the code is pretty much all

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

2017-07-12 Thread Saikat Kanjilal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084361#comment-16084361 ] Saikat Kanjilal commented on SPARK-18085: - [~vanzin] I would be interested in hel

[jira] [Commented] (SPARK-20703) Add an operator for writing data out

2017-07-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084355#comment-16084355 ] Steve Loughran commented on SPARK-20703: this has just added a whole new stack tr

[jira] [Comment Edited] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

2017-07-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084333#comment-16084333 ] Kazuaki Ishizaki edited comment on SPARK-21391 at 7/12/17 5:19 PM:

[jira] [Comment Edited] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

2017-07-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084333#comment-16084333 ] Kazuaki Ishizaki edited comment on SPARK-21391 at 7/12/17 5:19 PM:

[jira] [Commented] (SPARK-21380) Join with Columns thinks inner join is cross join even when aliased

2017-07-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084274#comment-16084274 ] Dongjoon Hyun commented on SPARK-21380: --- I see. I agree your point about that warni

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

2017-07-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084277#comment-16084277 ] Reynold Xin commented on SPARK-18085: - You should email dev@ to notify the list about

[jira] [Commented] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084270#comment-16084270 ] Saisai Shao commented on SPARK-21376: - Hi [~tgraves], it is the local yarn launcher p

[jira] [Commented] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

2017-07-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084333#comment-16084333 ] Kazuaki Ishizaki commented on SPARK-21391: -- This program works with the master.

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

2017-07-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084306#comment-16084306 ] Kazuaki Ishizaki commented on SPARK-21390: -- Another interesting results with Spa

[jira] [Comment Edited] (SPARK-21390) Dataset filter api inconsistency

2017-07-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084306#comment-16084306 ] Kazuaki Ishizaki edited comment on SPARK-21390 at 7/12/17 5:09 PM:

[jira] [Updated] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

2017-07-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18085: Summary: SPIP: Better History Server scalability for many / large applications (was: Better Histor

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2017-07-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084265#comment-16084265 ] Marcelo Vanzin commented on SPARK-18085: Sure, if it's just a matter of adding th

[jira] [Updated] (SPARK-18085) Better History Server scalability for many / large applications

2017-07-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18085: --- Labels: SPIP (was: ) > Better History Server scalability for many / large applications > ---

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

2017-07-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084266#comment-16084266 ] Kazuaki Ishizaki commented on SPARK-21390: -- Thank you for reporting this. I can

[jira] [Created] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

2017-07-12 Thread indraneel rao (JIRA)
indraneel rao created SPARK-21391: - Summary: Cannot convert a Seq of Map whose value type is again a seq, into a dataset Key: SPARK-21391 URL: https://issues.apache.org/jira/browse/SPARK-21391 Projec

[jira] [Updated] (SPARK-21390) Dataset filter api inconsistency

2017-07-12 Thread Gheorghe Gheorghe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gheorghe Gheorghe updated SPARK-21390: -- Description: Hello everybody, I've encountered a strange situation with the spark-she

[jira] [Updated] (SPARK-21390) Dataset filter api inconsistency

2017-07-12 Thread Gheorghe Gheorghe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gheorghe Gheorghe updated SPARK-21390: -- Description: Hello everybody, I've encountered a strange situation with the spark-she

[jira] [Created] (SPARK-21390) Dataset filter api inconsistency

2017-07-12 Thread Gheorghe Gheorghe (JIRA)
Gheorghe Gheorghe created SPARK-21390: - Summary: Dataset filter api inconsistency Key: SPARK-21390 URL: https://issues.apache.org/jira/browse/SPARK-21390 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18646) ExecutorClassLoader for spark-shell does not honor spark.executor.userClassPathFirst

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084193#comment-16084193 ] Apache Spark commented on SPARK-18646: -- User 'jiangxb1987' has created a pull reques

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

2017-07-12 Thread Arun Achuthan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084169#comment-16084169 ] Arun Achuthan commented on SPARK-18838: --- We are facing an issue where randomly some

[jira] [Created] (SPARK-21389) ALS recommendForAll optimization uses Native BLAS

2017-07-12 Thread Peng Meng (JIRA)
Peng Meng created SPARK-21389: - Summary: ALS recommendForAll optimization uses Native BLAS Key: SPARK-21389 URL: https://issues.apache.org/jira/browse/SPARK-21389 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-21376) Token is not renewed in yarn client process in cluster mode

2017-07-12 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084075#comment-16084075 ] Thomas Graves commented on SPARK-21376: --- Can you please clarify the title and descr

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-07-12 Thread Joseph Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084069#comment-16084069 ] Joseph Wang commented on SPARK-20307: - Fantastic function to add. It would be nice to

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084058#comment-16084058 ] Apache Spark commented on SPARK-20307: -- User 'yanboliang' has created a pull request

[jira] [Resolved] (SPARK-18619) Make QuantileDiscretizer/Bucketizer/StringIndexer inherit from HasHandleInvalid

2017-07-12 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-18619. - Resolution: Fixed Assignee: zhengruifeng Fix Version/s: 2.3.0 > Make QuantileDisc

[jira] [Commented] (SPARK-21388) GBT inherit from HasStepSize & LInearSVC/Binarizer from HasThreshold

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083974#comment-16083974 ] Apache Spark commented on SPARK-21388: -- User 'zhengruifeng' has created a pull reque

[jira] [Assigned] (SPARK-21388) GBT inherit from HasStepSize & LInearSVC/Binarizer from HasThreshold

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21388: Assignee: Apache Spark > GBT inherit from HasStepSize & LInearSVC/Binarizer from HasThresh

[jira] [Assigned] (SPARK-21388) GBT inherit from HasStepSize & LInearSVC/Binarizer from HasThreshold

2017-07-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21388: Assignee: (was: Apache Spark) > GBT inherit from HasStepSize & LInearSVC/Binarizer fro

  1   2   >