[jira] [Commented] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2018-03-20 Thread kaushik srinivas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405887#comment-16405887 ] kaushik srinivas commented on SPARK-19185: -- same issue found with kafka spark streaming 010.

[jira] [Comment Edited] (SPARK-20964) Make some keywords reserved along with the ANSI/SQL standard

2018-03-20 Thread Alex Ott (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405962#comment-16405962 ] Alex Ott edited comment on SPARK-20964 at 3/20/18 8:35 AM: --- Just want to add

[jira] [Commented] (SPARK-20964) Make some keywords reserved along with the ANSI/SQL standard

2018-03-20 Thread Alex Ott (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405962#comment-16405962 ] Alex Ott commented on SPARK-20964: -- Just want to add another example of query that is rejected by

[jira] [Created] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2018-03-20 Thread zuotingbing (JIRA)
zuotingbing created SPARK-23745: --- Summary: Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped Key: SPARK-23745 URL: https://issues.apache.org/jira/browse/SPARK-23745

[jira] [Commented] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406050#comment-16406050 ] Apache Spark commented on SPARK-23745: -- User 'zuotingbing' has created a pull request for this

[jira] [Assigned] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23745: Assignee: (was: Apache Spark) > Remove the directories of the

[jira] [Assigned] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23745: Assignee: Apache Spark > Remove the directories of the “hive.downloaded.resources.dir”

[jira] [Updated] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2018-03-20 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23745: Attachment: 2018-03-20_164832.png > Remove the directories of the “hive.downloaded.resources.dir”

[jira] [Commented] (SPARK-23513) java.io.IOException: Expected 12 fields, but got 5 for row :Spark submit error

2018-03-20 Thread Narsireddy AVula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406164#comment-16406164 ] Narsireddy AVula commented on SPARK-23513: -- Seems provided information is not sufficient  to

[jira] [Comment Edited] (SPARK-20964) Make some keywords reserved along with the ANSI/SQL standard

2018-03-20 Thread Alex Ott (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405962#comment-16405962 ] Alex Ott edited comment on SPARK-20964 at 3/20/18 8:36 AM: --- Just want to add

[jira] [Updated] (SPARK-23691) Use sql_conf util in PySpark tests where possible

2018-03-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23691: - Fix Version/s: 2.3.1 > Use sql_conf util in PySpark tests where possible >

[jira] [Commented] (SPARK-16872) Include Gaussian Naive Bayes Classifier

2018-03-20 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406128#comment-16406128 ] zhengruifeng commented on SPARK-16872: -- I think both 1) a new GNB estimator and 2) current NB

[jira] [Assigned] (SPARK-23542) The exists action shoule be further optimized in logical plan

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23542: Assignee: Apache Spark > The exists action shoule be further optimized in logical plan >

[jira] [Commented] (SPARK-23542) The exists action shoule be further optimized in logical plan

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406172#comment-16406172 ] Apache Spark commented on SPARK-23542: -- User 'KaiXinXiaoLei' has created a pull request for this

[jira] [Assigned] (SPARK-23542) The exists action shoule be further optimized in logical plan

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23542: Assignee: (was: Apache Spark) > The exists action shoule be further optimized in

[jira] [Updated] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2018-03-20 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23745: Description:   when start the HiveThriftServer2, we create some  directories for

[jira] [Updated] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2018-03-20 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23745: Description: !2018-03-20_164832.png!   when start the HiveThriftServer2, we create some

[jira] [Updated] (SPARK-23542) The exists action shoule be further optimized in logical plan

2018-03-20 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-23542: -- Description: The optimized logical plan of query '*select * from tt1 where exists (select * 

[jira] [Updated] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

2018-03-20 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23745: Description: !2018-03-20_164832.png!   when start the HiveThriftServer2, we create some  

[jira] [Commented] (SPARK-16745) Spark job completed however have to wait for 13 mins (data size is small)

2018-03-20 Thread Sujit Kumar Mahapatra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406153#comment-16406153 ] Sujit Kumar Mahapatra commented on SPARK-16745: --- +1. Getting similar issue with standalone

[jira] [Created] (SPARK-23746) HashMap UserDefinedType giving cast exception in Spark 1.6.2 while implementing UDAF

2018-03-20 Thread Izhar Ahmed (JIRA)
Izhar Ahmed created SPARK-23746: --- Summary: HashMap UserDefinedType giving cast exception in Spark 1.6.2 while implementing UDAF Key: SPARK-23746 URL: https://issues.apache.org/jira/browse/SPARK-23746

[jira] [Created] (SPARK-23753) [Performance] Group By Push Down through Join

2018-03-20 Thread Ioana Delaney (JIRA)
Ioana Delaney created SPARK-23753: - Summary: [Performance] Group By Push Down through Join Key: SPARK-23753 URL: https://issues.apache.org/jira/browse/SPARK-23753 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23574) SinglePartition in data source V2 scan

2018-03-20 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23574. - Resolution: Fixed Assignee: Jose Torres Fix Version/s: 2.4.0 > SinglePartition

[jira] [Resolved] (SPARK-23737) Scala API documentation leads to nonexistent pages for sources

2018-03-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23737. -- Resolution: Duplicate > Scala API documentation leads to nonexistent pages for sources >

[jira] [Commented] (SPARK-6190) create LargeByteBuffer abstraction for eliminating 2GB limit on blocks

2018-03-20 Thread Matthew Porter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406875#comment-16406875 ] Matthew Porter commented on SPARK-6190: --- Experiencing similar frustrations to Brian, we have well

[jira] [Created] (SPARK-23754) StopIterator exception in Python UDF results in partial result

2018-03-20 Thread Li Jin (JIRA)
Li Jin created SPARK-23754: -- Summary: StopIterator exception in Python UDF results in partial result Key: SPARK-23754 URL: https://issues.apache.org/jira/browse/SPARK-23754 Project: Spark Issue

[jira] [Updated] (SPARK-23754) StopIterator exception in Python UDF results in partial result

2018-03-20 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-23754: --- Description: {code:java} df = spark.range(0, 1000) from pyspark.sql.functions import udf def foo(x):

[jira] [Created] (SPARK-23755) [Performance] Distinct elimination

2018-03-20 Thread Ioana Delaney (JIRA)
Ioana Delaney created SPARK-23755: - Summary: [Performance] Distinct elimination Key: SPARK-23755 URL: https://issues.apache.org/jira/browse/SPARK-23755 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-23754) StopIterator exception in Python UDF results in partial result

2018-03-20 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-23754: --- Description: Reproduce: {code:java} df = spark.range(0, 1000) from pyspark.sql.functions import udf def

[jira] [Created] (SPARK-23758) MLlib 2.4 Roadmap

2018-03-20 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-23758: - Summary: MLlib 2.4 Roadmap Key: SPARK-23758 URL: https://issues.apache.org/jira/browse/SPARK-23758 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-20697) MSCK REPAIR TABLE resets the Storage Information for bucketed hive tables.

2018-03-20 Thread Abhishek Madav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Madav updated SPARK-20697: --- Priority: Critical (was: Major) > MSCK REPAIR TABLE resets the Storage Information for

[jira] [Updated] (SPARK-23690) VectorAssembler should have handleInvalid to handle columns with null values

2018-03-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23690: -- Shepherd: Joseph K. Bradley > VectorAssembler should have handleInvalid to handle

[jira] [Commented] (SPARK-18813) MLlib 2.2 Roadmap

2018-03-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407108#comment-16407108 ] Joseph K. Bradley commented on SPARK-18813: --- I just linked the roadmap for 2.4 (since we did

[jira] [Created] (SPARK-23759) Unable to bind Spark2 history server to specific host name / IP

2018-03-20 Thread Felix (JIRA)
Felix created SPARK-23759: - Summary: Unable to bind Spark2 history server to specific host name / IP Key: SPARK-23759 URL: https://issues.apache.org/jira/browse/SPARK-23759 Project: Spark Issue

[jira] [Commented] (SPARK-10884) Support prediction on single instance for regression and classification related models

2018-03-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407163#comment-16407163 ] Joseph K. Bradley commented on SPARK-10884: --- I know a lot of people are watching this, so I'm

[jira] [Updated] (SPARK-20697) MSCK REPAIR TABLE resets the Storage Information for bucketed hive tables.

2018-03-20 Thread Abhishek Madav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Madav updated SPARK-20697: --- Affects Version/s: 2.2.0 2.2.1 2.3.0 > MSCK

[jira] [Resolved] (SPARK-23500) Filters on named_structs could be pushed into scans

2018-03-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23500. - Resolution: Fixed Assignee: Henry Robinson Fix Version/s: 2.4.0 > Filters on

[jira] [Updated] (SPARK-23758) MLlib 2.4 Roadmap

2018-03-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23758: -- Description: h1. Roadmap process This roadmap is a master list for MLlib improvements

[jira] [Commented] (SPARK-23686) Make better usage of org.apache.spark.ml.util.Instrumentation

2018-03-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407205#comment-16407205 ] Joseph K. Bradley commented on SPARK-23686: --- This will be useful! Synced offline: we'll split

[jira] [Commented] (SPARK-23739) Spark structured streaming long running problem

2018-03-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407112#comment-16407112 ] Marco Gaido commented on SPARK-23739: - Can you provide some more info about how you are getting this

[jira] [Assigned] (SPARK-23750) [Performance] Inner Join Elimination based on Informational RI constraints

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23750: Assignee: Apache Spark > [Performance] Inner Join Elimination based on Informational RI

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-20 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407247#comment-16407247 ] Darek commented on SPARK-23534: --- https://github.com/Azure/azure-storage-java 7.0 will only work with

[jira] [Updated] (SPARK-23455) Default Params in ML should be saved separately

2018-03-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23455: -- Shepherd: Joseph K. Bradley > Default Params in ML should be saved separately >

[jira] [Commented] (SPARK-23513) java.io.IOException: Expected 12 fields, but got 5 for row :Spark submit error

2018-03-20 Thread abel-sun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407355#comment-16407355 ] abel-sun commented on SPARK-23513: -- Can you provide some more error message![~Fray] >

[jira] [Assigned] (SPARK-23759) Unable to bind Spark2 history server to specific host name / IP

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23759: Assignee: (was: Apache Spark) > Unable to bind Spark2 history server to specific host

[jira] [Commented] (SPARK-23759) Unable to bind Spark2 history server to specific host name / IP

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407288#comment-16407288 ] Apache Spark commented on SPARK-23759: -- User 'felixalbani' has created a pull request for this

[jira] [Assigned] (SPARK-23759) Unable to bind Spark2 history server to specific host name / IP

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23759: Assignee: Apache Spark > Unable to bind Spark2 history server to specific host name / IP

[jira] [Commented] (SPARK-23751) Kolmogorov-Smirnoff test Python API in pyspark.ml

2018-03-20 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407302#comment-16407302 ] Weichen Xu commented on SPARK-23751: I will work on this. :) > Kolmogorov-Smirnoff test Python API

[jira] [Assigned] (SPARK-23750) [Performance] Inner Join Elimination based on Informational RI constraints

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23750: Assignee: (was: Apache Spark) > [Performance] Inner Join Elimination based on

[jira] [Commented] (SPARK-23750) [Performance] Inner Join Elimination based on Informational RI constraints

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407248#comment-16407248 ] Apache Spark commented on SPARK-23750: -- User 'ioana-delaney' has created a pull request for this

[jira] [Updated] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

2018-03-20 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franck Tago updated SPARK-23519: Description: 1- create and populate a hive table  . I did this in a hive cli session .[ not that

[jira] [Comment Edited] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

2018-03-20 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389150#comment-16389150 ] Franck Tago edited comment on SPARK-23519 at 3/21/18 3:29 AM: -- Any updates

[jira] [Created] (SPARK-23760) CodegenContext.withSubExprEliminationExprs should save/restore CSE state correctly

2018-03-20 Thread Kris Mok (JIRA)
Kris Mok created SPARK-23760: Summary: CodegenContext.withSubExprEliminationExprs should save/restore CSE state correctly Key: SPARK-23760 URL: https://issues.apache.org/jira/browse/SPARK-23760 Project:

[jira] [Updated] (SPARK-23749) Avoid Hive.get() to compatible with different Hive metastore

2018-03-20 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23749: Description: {noformat} 18/03/15 22:34:46 WARN Hive: Failed to register all functions.

[jira] [Commented] (SPARK-23760) CodegenContext.withSubExprEliminationExprs should save/restore CSE state correctly

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407466#comment-16407466 ] Apache Spark commented on SPARK-23760: -- User 'rednaxelafx' has created a pull request for this

[jira] [Assigned] (SPARK-23760) CodegenContext.withSubExprEliminationExprs should save/restore CSE state correctly

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23760: Assignee: (was: Apache Spark) > CodegenContext.withSubExprEliminationExprs should

[jira] [Assigned] (SPARK-23760) CodegenContext.withSubExprEliminationExprs should save/restore CSE state correctly

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23760: Assignee: Apache Spark > CodegenContext.withSubExprEliminationExprs should save/restore

[jira] [Commented] (SPARK-20709) spark-shell use proxy-user failed

2018-03-20 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407334#comment-16407334 ] KaiXinXIaoLei commented on SPARK-20709: --- [~ffbin] [~srowen] i also meet this problem. Can u tell me

[jira] [Commented] (SPARK-19208) MultivariateOnlineSummarizer performance optimization

2018-03-20 Thread Teng Peng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407451#comment-16407451 ] Teng Peng commented on SPARK-19208: --- [~timhunter] Has the Jira ticket been opened? I believe this would

[jira] [Comment Edited] (SPARK-19208) MultivariateOnlineSummarizer performance optimization

2018-03-20 Thread Teng Peng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407451#comment-16407451 ] Teng Peng edited comment on SPARK-19208 at 3/21/18 4:44 AM: [~timhunter] Has

[jira] [Commented] (SPARK-23499) Mesos Cluster Dispatcher should support priority queues to submit drivers

2018-03-20 Thread Pascal GILLET (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406796#comment-16406796 ] Pascal GILLET commented on SPARK-23499: --- [~susanxhuynh] Certainly, none of the proposed solutions

[jira] [Assigned] (SPARK-21898) Feature parity for KolmogorovSmirnovTest in MLlib

2018-03-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-21898: - Assignee: Weichen Xu > Feature parity for KolmogorovSmirnovTest in MLlib >

[jira] [Created] (SPARK-23751) Kolmogorov-Smirnoff test Python API in pyspark.ml

2018-03-20 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-23751: - Summary: Kolmogorov-Smirnoff test Python API in pyspark.ml Key: SPARK-23751 URL: https://issues.apache.org/jira/browse/SPARK-23751 Project: Spark

[jira] [Resolved] (SPARK-21898) Feature parity for KolmogorovSmirnovTest in MLlib

2018-03-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-21898. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 19108

[jira] [Updated] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-03-20 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-23715: -- Description: This produces the expected answer: {noformat}

[jira] [Updated] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

2018-03-20 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franck Tago updated SPARK-23519: Component/s: SQL > Create View Commands Fails with The view output (col1,col1) contains >

[jira] [Updated] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-03-20 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-23715: -- Description: This produces the expected answer: {noformat}

[jira] [Updated] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-03-20 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-23715: -- Description: This produces the expected answer: {noformat}

[jira] [Created] (SPARK-23752) [Performance] Existential Subquery to Inner Join

2018-03-20 Thread Ioana Delaney (JIRA)
Ioana Delaney created SPARK-23752: - Summary: [Performance] Existential Subquery to Inner Join Key: SPARK-23752 URL: https://issues.apache.org/jira/browse/SPARK-23752 Project: Spark Issue

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-03-20 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406836#comment-16406836 ] Bruce Robbins commented on SPARK-23715: --- A fix to this requires some ugly hacking of the implicit

[jira] [Updated] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-03-20 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-23715: -- Description: This produces the expected answer: {noformat}

[jira] [Assigned] (SPARK-23749) Avoid Hive.get() to compatible with different Hive metastore

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23749: Assignee: (was: Apache Spark) > Avoid Hive.get() to compatible with different Hive

[jira] [Commented] (SPARK-23749) Avoid Hive.get() to compatible with different Hive metastore

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406724#comment-16406724 ] Apache Spark commented on SPARK-23749: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23749) Avoid Hive.get() to compatible with different Hive metastore

2018-03-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23749: Assignee: Apache Spark > Avoid Hive.get() to compatible with different Hive metastore >

[jira] [Created] (SPARK-23750) [Performance] Inner Join Elimination based on Informational RI constraints

2018-03-20 Thread Ioana Delaney (JIRA)
Ioana Delaney created SPARK-23750: - Summary: [Performance] Inner Join Elimination based on Informational RI constraints Key: SPARK-23750 URL: https://issues.apache.org/jira/browse/SPARK-23750

[jira] [Commented] (SPARK-23737) Scala API documentation leads to nonexistent pages for sources

2018-03-20 Thread Alexander Bessonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406591#comment-16406591 ] Alexander Bessonov commented on SPARK-23737: Oh, thanks. Linked them. > Scala API

[jira] [Created] (SPARK-23748) Support select from temp tables

2018-03-20 Thread Jose Torres (JIRA)
Jose Torres created SPARK-23748: --- Summary: Support select from temp tables Key: SPARK-23748 URL: https://issues.apache.org/jira/browse/SPARK-23748 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-23747) Add EpochCoordinator unit tests

2018-03-20 Thread Jose Torres (JIRA)
Jose Torres created SPARK-23747: --- Summary: Add EpochCoordinator unit tests Key: SPARK-23747 URL: https://issues.apache.org/jira/browse/SPARK-23747 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-23749) Avoid Hive.get() to compatible with different Hive metastore

2018-03-20 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-23749: --- Summary: Avoid Hive.get() to compatible with different Hive metastore Key: SPARK-23749 URL: https://issues.apache.org/jira/browse/SPARK-23749 Project: Spark

[jira] [Created] (SPARK-23756) [Performance] Redundant join elimination

2018-03-20 Thread Ioana Delaney (JIRA)
Ioana Delaney created SPARK-23756: - Summary: [Performance] Redundant join elimination Key: SPARK-23756 URL: https://issues.apache.org/jira/browse/SPARK-23756 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2018-03-20 Thread Ioana Delaney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406909#comment-16406909 ] Ioana Delaney commented on SPARK-19842: --- I opened several performance JIRAs to show the benefits of

[jira] [Created] (SPARK-23757) [Performance] Star schema detection improvements

2018-03-20 Thread Ioana Delaney (JIRA)
Ioana Delaney created SPARK-23757: - Summary: [Performance] Star schema detection improvements Key: SPARK-23757 URL: https://issues.apache.org/jira/browse/SPARK-23757 Project: Spark Issue