[jira] [Commented] (SPARK-24043) InterpretedPredicate.eval fails if expression tree contains Nondeterministic expressions

2018-04-23 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449326#comment-16449326 ] Takeshi Yamamuro commented on SPARK-24043: -- I tried this with codegen=off; {code:java}

[jira] [Commented] (SPARK-24058) Default Params in ML should be saved separately: Python API

2018-04-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449313#comment-16449313 ] Liang-Chi Hsieh commented on SPARK-24058: - OK. I will work on this. Thanks. > Default Params in

[jira] [Resolved] (SPARK-24002) Task not serializable caused by org.apache.parquet.io.api.Binary$ByteBufferBackedBinary.getBytes

2018-04-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-24002. --- Resolution: Fixed Fix Version/s: 2.4.0 This is resolved via

[jira] [Commented] (SPARK-23589) Add interpreted execution for ExternalMapToCatalyst expression

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449290#comment-16449290 ] Apache Spark commented on SPARK-23589: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-24043) InterpretedPredicate.eval fails if expression tree contains Nondeterministic expressions

2018-04-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449281#comment-16449281 ] Bruce Robbins commented on SPARK-24043: --- [~maropu] > Do I miss any precondition? For this bug to

[jira] [Assigned] (SPARK-24061) [SS]TypedFilter is not supported in Continuous Processing

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24061: Assignee: (was: Apache Spark) > [SS]TypedFilter is not supported in Continuous

[jira] [Commented] (SPARK-24061) [SS]TypedFilter is not supported in Continuous Processing

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449274#comment-16449274 ] Apache Spark commented on SPARK-24061: -- User 'yanlin-Lynn' has created a pull request for this

[jira] [Assigned] (SPARK-24061) [SS]TypedFilter is not supported in Continuous Processing

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24061: Assignee: Apache Spark > [SS]TypedFilter is not supported in Continuous Processing >

[jira] [Created] (SPARK-24061) [SS]TypedFilter is not supported in Continuous Processing

2018-04-23 Thread Wang Yanlin (JIRA)
Wang Yanlin created SPARK-24061: --- Summary: [SS]TypedFilter is not supported in Continuous Processing Key: SPARK-24061 URL: https://issues.apache.org/jira/browse/SPARK-24061 Project: Spark

[jira] [Updated] (SPARK-24061) [SS]TypedFilter is not supported in Continuous Processing

2018-04-23 Thread Wang Yanlin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Yanlin updated SPARK-24061: Attachment: TypedFilter_error.png > [SS]TypedFilter is not supported in Continuous Processing >

[jira] [Commented] (SPARK-24059) When blacklist disable always hash to a bad local directory may cause job failure

2018-04-23 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449254#comment-16449254 ] zhoukang commented on SPARK-24059: -- When we set multiple local dir on different disk. Retry will make

[jira] [Commented] (SPARK-24060) StreamingSymmetricHashJoinHelperSuite should initialize after SparkSession creation

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449237#comment-16449237 ] Apache Spark commented on SPARK-24060: -- User 'pwoody' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24060) StreamingSymmetricHashJoinHelperSuite should initialize after SparkSession creation

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24060: Assignee: (was: Apache Spark) > StreamingSymmetricHashJoinHelperSuite should

[jira] [Assigned] (SPARK-24060) StreamingSymmetricHashJoinHelperSuite should initialize after SparkSession creation

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24060: Assignee: Apache Spark > StreamingSymmetricHashJoinHelperSuite should initialize after

[jira] [Created] (SPARK-24060) StreamingSymmetricHashJoinHelperSuite should initialize after SparkSession creation

2018-04-23 Thread Patrick Woody (JIRA)
Patrick Woody created SPARK-24060: - Summary: StreamingSymmetricHashJoinHelperSuite should initialize after SparkSession creation Key: SPARK-24060 URL: https://issues.apache.org/jira/browse/SPARK-24060

[jira] [Commented] (SPARK-24059) When blacklist disable always hash to a bad local directory may cause job failure

2018-04-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449227#comment-16449227 ] Sean Owen commented on SPARK-24059: --- If the disk has failed, retry doesn't make a lot of sense. Why

[jira] [Resolved] (SPARK-23973) Remove consecutive sorts

2018-04-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23973. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 > Remove

[jira] [Commented] (SPARK-24059) When blacklist disable always hash to a bad local directory may cause job failure

2018-04-23 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449175#comment-16449175 ] zhoukang commented on SPARK-24059: -- cc [~vanzin] [~srowen] If we add retry make sense? I will work on

[jira] [Created] (SPARK-24059) When blacklist disable always hash to a bad local directory may cause job failure

2018-04-23 Thread zhoukang (JIRA)
zhoukang created SPARK-24059: Summary: When blacklist disable always hash to a bad local directory may cause job failure Key: SPARK-24059 URL: https://issues.apache.org/jira/browse/SPARK-24059 Project:

[jira] [Commented] (SPARK-24051) Incorrect results for certain queries using Java and Python APIs on Spark 2.3.0

2018-04-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449122#comment-16449122 ] Hyukjin Kwon commented on SPARK-24051: -- Thanks for elaborating it. The details look very helpful.

[jira] [Commented] (SPARK-23850) We should not redact username|user|url from UI by default

2018-04-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449073#comment-16449073 ] Marcelo Vanzin commented on SPARK-23850: Anyway, I'll take a stab at cleaning this up tomorrow if

[jira] [Commented] (SPARK-23850) We should not redact username|user|url from UI by default

2018-04-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449072#comment-16449072 ] Marcelo Vanzin commented on SPARK-23850: Yeah, things like {{java.vendor.url}} end up redacted

[jira] [Commented] (SPARK-23850) We should not redact username|user|url from UI by default

2018-04-23 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449054#comment-16449054 ] Thomas Graves commented on SPARK-23850: --- the url seems somewhat silly to me to, look at the

[jira] [Commented] (SPARK-23850) We should not redact username|user|url from UI by default

2018-04-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449046#comment-16449046 ] Marcelo Vanzin commented on SPARK-23850: Unless I hear back from the people CC'ed above, I'm

[jira] [Updated] (SPARK-23455) Default Params in ML should be saved separately

2018-04-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23455: -- Target Version/s: 2.4.0 > Default Params in ML should be saved separately >

[jira] [Assigned] (SPARK-23455) Default Params in ML should be saved separately

2018-04-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-23455: - Assignee: Liang-Chi Hsieh > Default Params in ML should be saved separately >

[jira] [Commented] (SPARK-24058) Default Params in ML should be saved separately: Python API

2018-04-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448994#comment-16448994 ] Joseph K. Bradley commented on SPARK-24058: --- CCing [~viirya] since you're the natural one to

[jira] [Created] (SPARK-24058) Default Params in ML should be saved separately: Python API

2018-04-23 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-24058: - Summary: Default Params in ML should be saved separately: Python API Key: SPARK-24058 URL: https://issues.apache.org/jira/browse/SPARK-24058 Project: Spark

[jira] [Updated] (SPARK-24057) put the real data type in the AssertionError message

2018-04-23 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-24057: --- Description: I had a wrong data type in one of my tests and got the following message: 

[jira] [Commented] (SPARK-24057) put the real data type in the AssertionError message

2018-04-23 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448965#comment-16448965 ] Huaxin Gao commented on SPARK-24057: I will submit a PR to fix the problem.  > put the real data

[jira] [Created] (SPARK-24057) put the real data type in the AssertionError message

2018-04-23 Thread Huaxin Gao (JIRA)
Huaxin Gao created SPARK-24057: -- Summary: put the real data type in the AssertionError message Key: SPARK-24057 URL: https://issues.apache.org/jira/browse/SPARK-24057 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-24056) Make consumer creation lazy in Kafka source for Structured streaming

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24056: Assignee: Apache Spark (was: Tathagata Das) > Make consumer creation lazy in Kafka

[jira] [Assigned] (SPARK-24056) Make consumer creation lazy in Kafka source for Structured streaming

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24056: Assignee: Tathagata Das (was: Apache Spark) > Make consumer creation lazy in Kafka

[jira] [Commented] (SPARK-24056) Make consumer creation lazy in Kafka source for Structured streaming

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448925#comment-16448925 ] Apache Spark commented on SPARK-24056: -- User 'tdas' has created a pull request for this issue:

[jira] [Commented] (SPARK-23961) pyspark toLocalIterator throws an exception

2018-04-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448884#comment-16448884 ] Dongjoon Hyun commented on SPARK-23961: --- Yep. I'm also able to observe this for all Spark 2.X (2.0

[jira] [Updated] (SPARK-23961) pyspark toLocalIterator throws an exception

2018-04-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23961: -- Affects Version/s: 2.0.2 2.1.2 2.3.0 > pyspark

[jira] [Created] (SPARK-24056) Make consumer creation lazy in Kafka source for Structured streaming

2018-04-23 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-24056: - Summary: Make consumer creation lazy in Kafka source for Structured streaming Key: SPARK-24056 URL: https://issues.apache.org/jira/browse/SPARK-24056 Project:

[jira] [Updated] (SPARK-23799) [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics

2018-04-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23799: Fix Version/s: (was: 2.3.1) > [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case

[jira] [Updated] (SPARK-24055) Add e2e test for using kubectl proxy for submitting spark jobs

2018-04-23 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-24055: --- Summary: Add e2e test for using kubectl proxy for submitting spark jobs (was: Add

[jira] [Commented] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448854#comment-16448854 ] Apache Spark commented on SPARK-24013: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Created] (SPARK-24055) Add e2e test for using kubectl proxy for submission

2018-04-23 Thread Anirudh Ramanathan (JIRA)
Anirudh Ramanathan created SPARK-24055: -- Summary: Add e2e test for using kubectl proxy for submission Key: SPARK-24055 URL: https://issues.apache.org/jira/browse/SPARK-24055 Project: Spark

[jira] [Assigned] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24013: Assignee: Apache Spark > ApproximatePercentile grinds to a halt on sorted input. >

[jira] [Assigned] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24013: Assignee: (was: Apache Spark) > ApproximatePercentile grinds to a halt on sorted

[jira] [Commented] (SPARK-24029) Set "reuse address" flag on listen sockets

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448838#comment-16448838 ] Apache Spark commented on SPARK-24029: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.10.0

2018-04-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448834#comment-16448834 ] Xiao Li commented on SPARK-23874: - [~bryanc] The fix was only merged to Arrow 0.10? Does Arrow also ship

[jira] [Commented] (SPARK-24023) Built-in SQL Functions improvement in SparkR

2018-04-23 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448835#comment-16448835 ] Shivaram Venkataraman commented on SPARK-24023: --- Thanks [~hyukjin.kwon] . +1 to what Felix

[jira] [Assigned] (SPARK-23888) speculative task should not run on a given host where another attempt is already running on

2018-04-23 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-23888: Assignee: wuyi > speculative task should not run on a given host where another attempt is

[jira] [Resolved] (SPARK-23888) speculative task should not run on a given host where another attempt is already running on

2018-04-23 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-23888. -- Resolution: Fixed Fix Version/s: (was: 2.3.0) 2.4.0 Issue

[jira] [Assigned] (SPARK-23433) java.lang.IllegalStateException: more than one active taskSet for stage

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23433: Assignee: Apache Spark > java.lang.IllegalStateException: more than one active taskSet

[jira] [Assigned] (SPARK-23433) java.lang.IllegalStateException: more than one active taskSet for stage

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23433: Assignee: (was: Apache Spark) > java.lang.IllegalStateException: more than one active

[jira] [Commented] (SPARK-23433) java.lang.IllegalStateException: more than one active taskSet for stage

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448822#comment-16448822 ] Apache Spark commented on SPARK-23433: -- User 'squito' has created a pull request for this issue:

[jira] [Updated] (SPARK-23874) Upgrade apache/arrow to 0.10.0

2018-04-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23874: -- Summary: Upgrade apache/arrow to 0.10.0 (was: Upgrade apache/arrow to 0.9.0) > Upgrade

[jira] [Resolved] (SPARK-23004) Structured Streaming raise "llegalStateException: Cannot remove after already committed or aborted"

2018-04-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23004. --- Resolution: Fixed Fix Version/s: 2.3.1 3.0.0 Issue resolved by

[jira] [Resolved] (SPARK-24053) Support add subdirectory named as user name on staging directory

2018-04-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24053. Resolution: Not A Bug Copy & paste of PR comments: {quote} > When we have multiple users

[jira] [Resolved] (SPARK-21168) KafkaRDD should always set kafka clientId.

2018-04-23 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger resolved SPARK-21168. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 19887

[jira] [Assigned] (SPARK-21168) KafkaRDD should always set kafka clientId.

2018-04-23 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger reassigned SPARK-21168: -- Assignee: liuzhaokun > KafkaRDD should always set kafka clientId. >

[jira] [Commented] (SPARK-23996) Implement the optimal KLL algorithms for quantiles in streams

2018-04-23 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448639#comment-16448639 ] Miao Wang commented on SPARK-23996: --- Thanks for your reply! I am learning the core part of the

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

2018-04-23 Thread Tavis Barr (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448634#comment-16448634 ] Tavis Barr commented on SPARK-18112: Sorry, following myself up... the next parameter, 

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

2018-04-23 Thread Tavis Barr (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448587#comment-16448587 ] Tavis Barr commented on SPARK-18112: It looks to me like this issue has actually not been fixed.  As

[jira] [Resolved] (SPARK-24024) Fix deviance calculations in GLM to handle corner cases

2018-04-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-24024. - Resolution: Fixed Fix Version/s: 2.4.0 > Fix deviance calculations in GLM to handle corner cases

[jira] [Assigned] (SPARK-24024) Fix deviance calculations in GLM to handle corner cases

2018-04-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-24024: --- Assignee: Teng Peng > Fix deviance calculations in GLM to handle corner cases >

[jira] [Resolved] (SPARK-23638) Spark on k8s: spark.kubernetes.initContainer.image has no effect

2018-04-23 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li resolved SPARK-23638. -- Resolution: Not A Problem > Spark on k8s: spark.kubernetes.initContainer.image has no effect >

[jira] [Commented] (SPARK-24051) Incorrect results for certain queries using Java API on Spark 2.3.0

2018-04-23 Thread Emlyn Corrin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448390#comment-16448390 ] Emlyn Corrin commented on SPARK-24051: -- I've managed to reproduce this in {{pyspark}}: {code} from

[jira] [Updated] (SPARK-24051) Incorrect results for certain queries using Java and Python APIs on Spark 2.3.0

2018-04-23 Thread Emlyn Corrin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emlyn Corrin updated SPARK-24051: - Summary: Incorrect results for certain queries using Java and Python APIs on Spark 2.3.0 (was:

[jira] [Commented] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448365#comment-16448365 ] Marco Gaido commented on SPARK-24013: - [~juliuszsompolski] I have been able to reproduce with

[jira] [Assigned] (SPARK-24054) Add array_position function / element_at functions

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24054: Assignee: (was: Apache Spark) > Add array_position function / element_at functions >

[jira] [Commented] (SPARK-24054) Add array_position function / element_at functions

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448276#comment-16448276 ] Apache Spark commented on SPARK-24054: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-24054) Add array_position function / element_at functions

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24054: Assignee: Apache Spark > Add array_position function / element_at functions >

[jira] [Created] (SPARK-24054) Add array_position function / element_at functions

2018-04-23 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-24054: Summary: Add array_position function / element_at functions Key: SPARK-24054 URL: https://issues.apache.org/jira/browse/SPARK-24054 Project: Spark Issue

[jira] [Comment Edited] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Juliusz Sompolski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448093#comment-16448093 ] Juliusz Sompolski edited comment on SPARK-24013 at 4/23/18 1:11 PM:

[jira] [Commented] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Juliusz Sompolski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448093#comment-16448093 ] Juliusz Sompolski commented on SPARK-24013: --- Hi [~mgaido] I tested again on current master

[jira] [Updated] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Juliusz Sompolski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-24013: -- Attachment: screenshot-1.png > ApproximatePercentile grinds to a halt on sorted input.

[jira] [Resolved] (SPARK-23589) Add interpreted execution for ExternalMapToCatalyst expression

2018-04-23 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23589. --- Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.4.0 >

[jira] [Commented] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448058#comment-16448058 ] Marco Gaido commented on SPARK-24013: - I cannot reproduce on current master. For me it was very fast

[jira] [Commented] (SPARK-24009) spark2.3.0 INSERT OVERWRITE LOCAL DIRECTORY '/home/spark/aaaaab'

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448056#comment-16448056 ] Marco Gaido commented on SPARK-24009: - I have not been able to reproduce. > spark2.3.0 INSERT

[jira] [Assigned] (SPARK-23564) the optimized logical plan about Left anti join should be further optimization

2018-04-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23564: --- Assignee: Marco Gaido > the optimized logical plan about Left anti join should be further

[jira] [Resolved] (SPARK-23564) the optimized logical plan about Left anti join should be further optimization

2018-04-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23564. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21083

[jira] [Assigned] (SPARK-7132) Add fit with validation set to spark.ml GBT

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7132: --- Assignee: Apache Spark > Add fit with validation set to spark.ml GBT >

[jira] [Assigned] (SPARK-7132) Add fit with validation set to spark.ml GBT

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7132: --- Assignee: (was: Apache Spark) > Add fit with validation set to spark.ml GBT >

[jira] [Commented] (SPARK-7132) Add fit with validation set to spark.ml GBT

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448039#comment-16448039 ] Apache Spark commented on SPARK-7132: - User 'WeichenXu123' has created a pull request for this issue:

[jira] [Commented] (SPARK-15257) Require CREATE EXTERNAL TABLE to specify LOCATION

2018-04-23 Thread ice bai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447949#comment-16447949 ] ice bai commented on SPARK-15257: - “but its {color:#FF}metadata{color} won't be deleted even when the

[jira] [Commented] (SPARK-24053) Support add subdirectory named as user name on staging directory

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447943#comment-16447943 ] Apache Spark commented on SPARK-24053: -- User 'caneGuy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24053) Support add subdirectory named as user name on staging directory

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24053: Assignee: Apache Spark > Support add subdirectory named as user name on staging

[jira] [Assigned] (SPARK-24053) Support add subdirectory named as user name on staging directory

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24053: Assignee: (was: Apache Spark) > Support add subdirectory named as user name on

[jira] [Created] (SPARK-24053) Support add subdirectory named as user name on staging directory

2018-04-23 Thread zhoukang (JIRA)
zhoukang created SPARK-24053: Summary: Support add subdirectory named as user name on staging directory Key: SPARK-24053 URL: https://issues.apache.org/jira/browse/SPARK-24053 Project: Spark

[jira] [Commented] (SPARK-20426) OneForOneStreamManager occupies too much memory.

2018-04-23 Thread ice bai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447935#comment-16447935 ] ice bai commented on SPARK-20426: - refer with the following issue:

[jira] [Commented] (SPARK-24052) Support spark version showing on environment page

2018-04-23 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447917#comment-16447917 ] zhoukang commented on SPARK-24052: -- [~jiangxb] Cloud you help review this?Thanks too much! > Support

[jira] [Assigned] (SPARK-24052) Support spark version showing on environment page

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24052: Assignee: (was: Apache Spark) > Support spark version showing on environment page >

[jira] [Commented] (SPARK-24052) Support spark version showing on environment page

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447914#comment-16447914 ] Apache Spark commented on SPARK-24052: -- User 'caneGuy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24052) Support spark version showing on environment page

2018-04-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24052: Assignee: Apache Spark > Support spark version showing on environment page >

[jira] [Commented] (SPARK-23901) Data Masking Functions

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447916#comment-16447916 ] Marco Gaido commented on SPARK-23901: - [~smilegator] [~ueshin] any comment on my questions? > Data

[jira] [Updated] (SPARK-24052) Support spark version showing on environment page

2018-04-23 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-24052: - Description: Since we may have multiple version in production cluster,we can showing some information

[jira] [Updated] (SPARK-24052) Support spark version showing on environment page

2018-04-23 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-24052: - Attachment: environment-page.png > Support spark version showing on environment page >

[jira] [Created] (SPARK-24052) Support spark version showing on environment page

2018-04-23 Thread zhoukang (JIRA)
zhoukang created SPARK-24052: Summary: Support spark version showing on environment page Key: SPARK-24052 URL: https://issues.apache.org/jira/browse/SPARK-24052 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-10697) Lift Calculation in Association Rule mining

2018-04-23 Thread Mourits de Beer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447898#comment-16447898 ] Mourits de Beer edited comment on SPARK-10697 at 4/23/18 10:16 AM: ---

[jira] [Commented] (SPARK-10697) Lift Calculation in Association Rule mining

2018-04-23 Thread Mourits de Beer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447898#comment-16447898 ] Mourits de Beer commented on SPARK-10697: - Hello - is this still in progress? We require the

[jira] [Commented] (SPARK-2750) Add Https support for Web UI

2018-04-23 Thread palash gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447880#comment-16447880 ] palash gupta commented on SPARK-2750: - I am using Spark 1.6.3 for my current project , is there any

[jira] [Updated] (SPARK-24051) Incorrect results for certain queries using Java API on Spark 2.3.0

2018-04-23 Thread Emlyn Corrin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emlyn Corrin updated SPARK-24051: - Summary: Incorrect results for certain queries using Java API on Spark 2.3.0 (was: Incorrect

[jira] [Commented] (SPARK-24051) Incorrect results for certain queries in Java API on Spark 2.3.0

2018-04-23 Thread Emlyn Corrin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447854#comment-16447854 ] Emlyn Corrin commented on SPARK-24051: -- [~hyukjin.kwon] I expanded the title a bit, but feel free to

[jira] [Updated] (SPARK-24051) Incorrect results for certain queries in Java API on Spark 2.3.0

2018-04-23 Thread Emlyn Corrin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emlyn Corrin updated SPARK-24051: - Summary: Incorrect results for certain queries in Java API on Spark 2.3.0 (was: Incorrect

  1   2   >