[jira] [Created] (SPARK-21074) Parquet files are read fully even though only count() is requested

Michael Spector created SPARK-21074: --- Summary: Parquet files are read fully even though only count() is requested Key: SPARK-21074 URL: https://issues.apache.org/jira/browse/SPARK-21074 Project:

[jira] [Updated] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 吴志龙 updated SPARK-21075: Description: cd $spark_home ./dev/make-distribution.sh --name custom-spark --tgz -Psparkr -Phadoop-2.6 -Phive

[jira] [Updated] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 吴志龙 updated SPARK-21075: Description: cd $spark_home/spark-2.2.0-rc4 ./dev/make-distribution.sh --name custom-spark --tgz -Psparkr

[jira] [Updated] (SPARK-21073) Support map_keys and map_values functions in DataSet

[ https://issues.apache.org/jira/browse/SPARK-21073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated SPARK-21073: -- Summary: Support map_keys and map_values functions in DataSet (was: Support map_keys and

[jira] [Updated] (SPARK-19975) Add map_keys and map_values functions to Python

[ https://issues.apache.org/jira/browse/SPARK-19975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-19975: - Issue Type: Improvement (was: Bug) > Add map_keys and map_values functions to Python >

[jira] [Resolved] (SPARK-21073) Support map_keys and map_values functions in DataSet

[ https://issues.apache.org/jira/browse/SPARK-21073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21073. -- Resolution: Duplicate The proposed changes look a subset of SPARK-19975 > Support map_keys

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047701#comment-16047701 ] 吴志龙 commented on SPARK-21075: - I was using it： java version "1.8.0_131" Apache Maven 3.3.9 The problem that

[jira] [Assigned] (SPARK-21066) LibSVM load just one input file

[ https://issues.apache.org/jira/browse/SPARK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21066: Assignee: (was: Apache Spark) > LibSVM load just one input file >

[jira] [Assigned] (SPARK-21066) LibSVM load just one input file

[ https://issues.apache.org/jira/browse/SPARK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21066: Assignee: Apache Spark > LibSVM load just one input file >

[jira] [Updated] (SPARK-21076) R dapply doesn't return array or raw columns when array have different length

[ https://issues.apache.org/jira/browse/SPARK-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Yang updated SPARK-21076: Description: still have this issue when input data is an array column not having the same length on each

[jira] [Updated] (SPARK-21076) R dapply doesn't return array or raw columns when array have different length

[ https://issues.apache.org/jira/browse/SPARK-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Yang updated SPARK-21076: Description: Calling SparkR::dapplyCollect with R functions that return dataframes produces an error.

[jira] [Updated] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 吴志龙 updated SPARK-21075: Description: cd $spark_home ./dev/make-distribution.sh --name custom-spark --tgz -Psparkr -Phadoop-2.6 -Phive

[jira] [Assigned] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

[ https://issues.apache.org/jira/browse/SPARK-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20920: - Assignee: Sean Owen > ForkJoinPool pools are leaked when writing hive tables with many

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047706#comment-16047706 ] Sean Owen commented on SPARK-21075: --- You're not using Java 8 if you get that message. Your Maven is

[jira] [Assigned] (SPARK-21073) Support map_keys and map_values functions in DataSet

[ https://issues.apache.org/jira/browse/SPARK-21073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21073: Assignee: Apache Spark > Support map_keys and map_values functions in DataSet >

[jira] [Commented] (SPARK-21073) Support map_keys and map_values functions in DataSet

[ https://issues.apache.org/jira/browse/SPARK-21073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047579#comment-16047579 ] Apache Spark commented on SPARK-21073: -- User 'darionyaphet' has created a pull request for this

[jira] [Assigned] (SPARK-21073) Support map_keys and map_values functions in DataSet

[ https://issues.apache.org/jira/browse/SPARK-21073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21073: Assignee: (was: Apache Spark) > Support map_keys and map_values functions in DataSet

[jira] [Commented] (SPARK-21066) LibSVM load just one input file

[ https://issues.apache.org/jira/browse/SPARK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047723#comment-16047723 ] Apache Spark commented on SPARK-21066: -- User 'darionyaphet' has created a pull request for this

[jira] [Created] (SPARK-21076) R dapply doesn't return array or raw columns when array have different length

Xu Yang created SPARK-21076: --- Summary: R dapply doesn't return array or raw columns when array have different length Key: SPARK-21076 URL: https://issues.apache.org/jira/browse/SPARK-21076 Project: Spark

[jira] [Commented] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

[ https://issues.apache.org/jira/browse/SPARK-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047759#comment-16047759 ] kavn qin commented on SPARK-19878: -- I'm so sorry to have seen this so far. Thanks for your

[jira] [Created] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

吴志龙 created SPARK-21075: --- Summary: spark 2.2 mvn [error] javac: invalid source release: 1.8 Key: SPARK-21075 URL: https://issues.apache.org/jira/browse/SPARK-21075 Project: Spark Issue Type: Question

[jira] [Resolved] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21075. --- Resolution: Not A Problem 2.2 requires Java 8 > spark 2.2 mvn [error] javac: invalid source

[jira] [Resolved] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

[ https://issues.apache.org/jira/browse/SPARK-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20920. --- Resolution: Fixed Fix Version/s: 2.1.2 2.3.0 2.2.1

[jira] [Updated] (SPARK-21073) Support map_keys and map_values in DataSet

[ https://issues.apache.org/jira/browse/SPARK-21073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated SPARK-21073: -- Summary: Support map_keys and map_values in DataSet (was: Support map_keys and map_values in

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047710#comment-16047710 ] 吴志龙 commented on SPARK-21075: - [root@kafka-test-50-123 ~]# mvn -version Java HotSpot(TM) 64-Bit Server VM

[jira] [Commented] (SPARK-21077) Cannot access public files over S3 protocol

[ https://issues.apache.org/jira/browse/SPARK-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047828#comment-16047828 ] Sean Owen commented on SPARK-21077: --- I think this is a Hadoop or AWS SDK issue, not Spark. > Cannot

[jira] [Created] (SPARK-21078) JobHistory applications synchronized is invalid

fangfengbin created SPARK-21078: --- Summary: JobHistory applications synchronized is invalid Key: SPARK-21078 URL: https://issues.apache.org/jira/browse/SPARK-21078 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047849#comment-16047849 ] Michel Lemay edited comment on SPARK-21021 at 6/13/17 1:04 PM: --- Yes, as a

[jira] [Comment Edited] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047849#comment-16047849 ] Michel Lemay edited comment on SPARK-21021 at 6/13/17 1:04 PM: --- Yes, as a

[jira] [Created] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

Maria created SPARK-21079: - Summary: ANALYZE TABLE fails to calculate totalSize for a partitioned table Key: SPARK-21079 URL: https://issues.apache.org/jira/browse/SPARK-21079 Project: Spark Issue

[jira] [Commented] (SPARK-21077) Cannot access public files over S3 protocol

[ https://issues.apache.org/jira/browse/SPARK-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047836#comment-16047836 ] Ciprian Tomoiaga commented on SPARK-21077: -- I thought so too, but they said the AWS-SDK

[jira] [Created] (SPARK-21077) Cannot access public files over S3 protocol

Ciprian Tomoiaga created SPARK-21077: Summary: Cannot access public files over S3 protocol Key: SPARK-21077 URL: https://issues.apache.org/jira/browse/SPARK-21077 Project: Spark Issue

[jira] [Commented] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047849#comment-16047849 ] Michel Lemay commented on SPARK-21021: -- Yes, as a workaround, we do a

[jira] [Commented] (SPARK-21078) JobHistory applications synchronized is invalid

[ https://issues.apache.org/jira/browse/SPARK-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047860#comment-16047860 ] Sean Owen commented on SPARK-21078: --- I agree there's a problem here, which might or might not actually

[jira] [Comment Edited] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047849#comment-16047849 ] Michel Lemay edited comment on SPARK-21021 at 6/13/17 1:06 PM: --- Yes, as a

[jira] [Resolved] (SPARK-21064) Fix the default value bug in NettyBlockTransferServiceSuite

[ https://issues.apache.org/jira/browse/SPARK-21064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21064. --- Resolution: Fixed Fix Version/s: 2.1.2 2.3.0 2.2.1

[jira] [Commented] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047815#comment-16047815 ] jin xing commented on SPARK-21021: -- I think the reason of the incompatibility is that the

[jira] [Assigned] (SPARK-21039) Use treeAggregate instead of aggregate in DataFrame.stat.bloomFilter

[ https://issues.apache.org/jira/browse/SPARK-21039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21039: - Assignee: Lovasoa Priority: Minor (was: Major) > Use treeAggregate instead of aggregate in

[jira] [Resolved] (SPARK-21060) Css style about paging function is error in the executor page.

[ https://issues.apache.org/jira/browse/SPARK-21060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21060. --- Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Resolved by

[jira] [Updated] (SPARK-21065) Spark Streaming concurrentJobs + StreamingJobProgressListener conflict

[ https://issues.apache.org/jira/browse/SPARK-21065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Dutrow updated SPARK-21065: --- Component/s: Web UI > Spark Streaming concurrentJobs + StreamingJobProgressListener conflict >

[jira] [Comment Edited] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047849#comment-16047849 ] Michel Lemay edited comment on SPARK-21021 at 6/13/17 1:01 PM: --- Yes, as a

[jira] [Resolved] (SPARK-21039) Use treeAggregate instead of aggregate in DataFrame.stat.bloomFilter

[ https://issues.apache.org/jira/browse/SPARK-21039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21039. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18263

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA, because there are many tasks found and needed to be done here.

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048427#comment-16048427 ] Zhenhua Wang commented on SPARK-21079: -- [~tejasp] Thanks for the explanation! [~mbasmanova] Would

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Summary: Support statistics collection and cardinality estimation for partitioned tables (was:

[jira] [Updated] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21079: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > ANALYZE TABLE fails to calculate

[jira] [Updated] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20986: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > Reset table's statistics after

[jira] [Updated] (SPARK-15616) CatalogRelation should fallback to HDFS size of partitions that are involved in Query if statistics are not available.

[ https://issues.apache.org/jira/browse/SPARK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-15616: - Affects Version/s: 2.3.0 Issue Type: Sub-task (was: Improvement)

[jira] [Commented] (SPARK-21077) Cannot access public files over S3 protocol

[ https://issues.apache.org/jira/browse/SPARK-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048470#comment-16048470 ] Hyukjin Kwon commented on SPARK-21077: -- I also think it is not a Spark issue at least and it looks

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048270#comment-16048270 ] Sean Owen commented on SPARK-21082: --- I don't see how this would interact with, for example, data

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA as an umbrella ticket, because there are a few tasks found and

[jira] [Updated] (SPARK-16669) Partition pruning for metastore relation size estimates for better join selection.

[ https://issues.apache.org/jira/browse/SPARK-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-16669: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > Partition pruning for metastore

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA, because there are a few tasks found and needed to be done

[jira] [Commented] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048424#comment-16048424 ] Apache Spark commented on SPARK-20379: -- User 'vanzin' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048384#comment-16048384 ] Dongjoon Hyun edited comment on SPARK-21075 at 6/13/17 8:46 PM: Please do

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048432#comment-16048432 ] Maria commented on SPARK-21079: --- [~ZenWzh], yes, I have a fix and will try to submit a PR. > ANALYZE TABLE

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Affects Version/s: 2.3.0 Issue Type: Improvement (was: Sub-task)

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20379: Assignee: Apache Spark > Allow setting SSL-related passwords through env variables >

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20379: Assignee: (was: Apache Spark) > Allow setting SSL-related passwords through env

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048441#comment-16048441 ] Zhenhua Wang commented on SPARK-21079: -- [~mbasmanova] Great~ > ANALYZE TABLE fails to calculate

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048381#comment-16048381 ] Tejas Patil commented on SPARK-21079: - [~ZenWzh] The reason why unit tests won't catch this is

[jira] [Issue Comment Deleted] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maria updated SPARK-21079: -- Comment: was deleted (was: [~ZenWzh], I'm using partitioned table created by Hive. The data is stored in DWRF

[jira] [Updated] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominic Ricard updated SPARK-21067: --- Description: After upgrading our Thrift cluster to 2.1.1, we ran into an issue where CTAS

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048384#comment-16048384 ] Dongjoon Hyun commented on SPARK-21075: --- Please do `jps` and check whether `Zinc` is running or

[jira] [Commented] (SPARK-21078) JobHistory applications synchronized is invalid

[ https://issues.apache.org/jira/browse/SPARK-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048650#comment-16048650 ] fangfengbin commented on SPARK-21078: - [~sowen], this actually cause a problem in practice, when

[jira] [Comment Edited] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048653#comment-16048653 ] DjvuLee edited comment on SPARK-21082 at 6/14/17 3:15 AM: -- [~srowen] This

[jira] [Resolved] (SPARK-19753) Remove all shuffle files on a host in case of slave lost of fetch failure

[ https://issues.apache.org/jira/browse/SPARK-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19753. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18150

[jira] [Comment Edited] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048647#comment-16048647 ] yuhao yang edited comment on SPARK-21086 at 6/14/17 5:22 AM: - Sounds good.

[jira] [Commented] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048720#comment-16048720 ] Apache Spark commented on SPARK-21085: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Commented] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

[ https://issues.apache.org/jira/browse/SPARK-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048723#comment-16048723 ] yuhao yang commented on SPARK-21087: I'd like to work on this if my

[jira] [Created] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

Joseph K. Bradley created SPARK-21086: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting Key: SPARK-21086 URL: https://issues.apache.org/jira/browse/SPARK-21086

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048653#comment-16048653 ] DjvuLee commented on SPARK-21082: - [~srowen] This situation occurred when the partition number is larger

[jira] [Updated] (SPARK-20602) Adding LBFGS optimizer and Squared_hinge loss for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-20602: --- Summary: Adding LBFGS optimizer and Squared_hinge loss for LinearSVC (was: Adding LBFGS as

[jira] [Assigned] (SPARK-19753) Remove all shuffle files on a host in case of slave lost of fetch failure

[ https://issues.apache.org/jira/browse/SPARK-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19753: --- Assignee: Sital Kedia > Remove all shuffle files on a host in case of slave lost of fetch

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048698#comment-16048698 ] yuhao yang commented on SPARK-20988: Eh.. I was trying to add the squared_hinge loss to LinearSVC and

[jira] [Comment Edited] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048647#comment-16048647 ] yuhao yang edited comment on SPARK-21086 at 6/14/17 5:12 AM: - Sounds good.

[jira] [Assigned] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21085: Assignee: Apache Spark (was: Xiao Li) > Failed to read the partitioned table created by

[jira] [Assigned] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21085: Assignee: Xiao Li (was: Apache Spark) > Failed to read the partitioned table created by

[jira] [Created] (SPARK-21088) CrossValidator, TrainValidationSplit should preserve all models after fitting: Python

Joseph K. Bradley created SPARK-21088: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting: Python Key: SPARK-21088 URL: https://issues.apache.org/jira/browse/SPARK-21088

[jira] [Updated] (SPARK-21088) CrossValidator, TrainValidationSplit should preserve all models after fitting: Python

[ https://issues.apache.org/jira/browse/SPARK-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21088: -- Component/s: PySpark > CrossValidator, TrainValidationSplit should preserve all models

[jira] [Created] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

Joseph K. Bradley created SPARK-21087: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala Key: SPARK-21087 URL: https://issues.apache.org/jira/browse/SPARK-21087

[jira] [Assigned] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20986: --- Assignee: Lianhui Wang > Reset table's statistics after PruneFileSourcePartitions rule. >

[jira] [Resolved] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20986. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18205

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048593#comment-16048593 ] Vincent commented on SPARK-20988: - opps. I have finished the conversion part, but there are still other

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048612#comment-16048612 ] 吴志龙 commented on SPARK-21075: - ok,thanks > spark 2.2 mvn [error] javac: invalid source release: 1.8 >

[jira] [Commented] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048647#comment-16048647 ] yuhao yang commented on SPARK-21086: Sounds good. About the default path for saving different models,

[jira] [Commented] (SPARK-20602) Adding LBFGS optimizer and Squared_hinge loss for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048663#comment-16048663 ] yuhao yang commented on SPARK-20602: Combining this with SPARK-20348. Support squared hinge loss (L2

[jira] [Resolved] (SPARK-20348) Support squared hinge loss (L2 loss) for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang resolved SPARK-20348. Resolution: Duplicate Combine it with SPARK-20602 and resolve this as duplicate. > Support

[jira] [Created] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

Xiao Li created SPARK-21089: --- Summary: Table properties are not shown in DESC EXTENDED/FORMATTED Key: SPARK-21089 URL: https://issues.apache.org/jira/browse/SPARK-21089 Project: Spark Issue Type:

[jira] [Updated] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21089: Target Version/s: 2.2.0 > Table properties are not shown in DESC EXTENDED/FORMATTED >

[jira] [Assigned] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21090: Assignee: Apache Spark > Optimize the unified memory manager code >

[jira] [Commented] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048731#comment-16048731 ] Apache Spark commented on SPARK-21090: -- User '10110346' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21090: Assignee: (was: Apache Spark) > Optimize the unified memory manager code >

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048654#comment-16048654 ] DjvuLee commented on SPARK-21082: - My idea is try to consider the BlockManger information when scheduling

[jira] [Assigned] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21089: Assignee: Apache Spark (was: Xiao Li) > Table properties are not shown in DESC

[jira] [Assigned] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21089: Assignee: Xiao Li (was: Apache Spark) > Table properties are not shown in DESC

[jira] [Commented] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048717#comment-16048717 ] Apache Spark commented on SPARK-21089: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Created] (SPARK-21090) Optimize the unified memory manager code

liuxian created SPARK-21090: --- Summary: Optimize the unified memory manager code Key: SPARK-21090 URL: https://issues.apache.org/jira/browse/SPARK-21090 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-21090: Description: 1.In *acquireStorageMemory*, when the MemoryMode is OFF_HEAP ,the *maxMemory* should be

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048489#comment-16048489 ] Seth Hendrickson commented on SPARK-20988: -- I've already started it a bit. Would you mind doing

  1   2   >