[jira] [Commented] (SPARK-20780) Spark Kafka10 Consumer Hangs

2018-12-06 Thread shangmin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712449#comment-16712449 ] shangmin commented on SPARK-20780: -- The root case is the kafka consumer lib version used in your

[jira] [Updated] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23580: -- Target Version/s: (was: 2.5.0) > Interpreted mode fallback should be implemented for all

[jira] [Updated] (SPARK-24360) Support Hive 3.1 metastore

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24360: -- Target Version/s: (was: 2.5.0) > Support Hive 3.1 metastore > -- >

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-12978: -- Target Version/s: (was: 2.5.0) > Skip unnecessary final group-by when input data already

[jira] [Updated] (SPARK-15117) Generate code that get a value in each compressed column from CachedBatch when DataFrame.cache() is called

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15117: -- Target Version/s: (was: 2.5.0) > Generate code that get a value in each compressed column

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712414#comment-16712414 ] Dongjoon Hyun commented on SPARK-20184: --- If this exists up to 2.4.0, could you update the `Affects

[jira] [Updated] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16196: -- Target Version/s: (was: 2.5.0) > Optimize in-memory scan performance using ColumnarBatches

[jira] [Updated] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20184: -- Target Version/s: (was: 2.5.0) > performance regression for complex/long sql when enable

[jira] [Updated] (SPARK-21318) The exception message thrown by `lookupFunction` is ambiguous.

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21318: -- Target Version/s: (was: 2.5.0) > The exception message thrown by `lookupFunction` is

[jira] [Commented] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712410#comment-16712410 ] Dongjoon Hyun commented on SPARK-23580: --- I removed the target version `2.5.0`. Please feel free to

[jira] [Updated] (SPARK-24360) Support Hive 3.1 metastore

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24360: -- Target Version/s: 3.0.0 > Support Hive 3.1 metastore > -- > >

[jira] [Created] (SPARK-26301) Consider switching from putting secret in environment variable directly to using secret reference

2018-12-06 Thread Matt Cheah (JIRA)
Matt Cheah created SPARK-26301: -- Summary: Consider switching from putting secret in environment variable directly to using secret reference Key: SPARK-26301 URL: https://issues.apache.org/jira/browse/SPARK-26301

[jira] [Commented] (SPARK-26239) Add configurable auth secret source in k8s backend

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712373#comment-16712373 ] Apache Spark commented on SPARK-26239: -- User 'mccheah' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26239) Add configurable auth secret source in k8s backend

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26239: Assignee: Apache Spark > Add configurable auth secret source in k8s backend >

[jira] [Assigned] (SPARK-26239) Add configurable auth secret source in k8s backend

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26239: Assignee: (was: Apache Spark) > Add configurable auth secret source in k8s backend >

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-12-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26188: Labels: release-notes (was: ) > Spark 2.4.0 Partitioning behavior breaks backwards compatibility >

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-12-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26188: Labels: correctness release-notes (was: release-notes) > Spark 2.4.0 Partitioning behavior breaks

[jira] [Commented] (SPARK-25401) Reorder the required ordering to match the table's output ordering for bucket join

2018-12-06 Thread David Vrba (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712350#comment-16712350 ] David Vrba commented on SPARK-25401: I was looking at it and i believe that it the class

[jira] [Updated] (SPARK-26298) Upgrade Janino version to 3.0.11

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26298: -- Issue Type: Improvement (was: Bug) > Upgrade Janino version to 3.0.11 >

[jira] [Resolved] (SPARK-26298) Upgrade Janino version to 3.0.11

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26298. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0 This is

[jira] [Issue Comment Deleted] (SPARK-26298) Upgrade Janino version to 3.0.11

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26298: -- Comment: was deleted (was: User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26300) The `checkForStreaming` mothod may be called twice in `createQuery`

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26300: Assignee: (was: Apache Spark) > The `checkForStreaming` mothod may be called twice

[jira] [Commented] (SPARK-26300) The `checkForStreaming` mothod may be called twice in `createQuery`

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712296#comment-16712296 ] Apache Spark commented on SPARK-26300: -- User '10110346' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26300) The `checkForStreaming` mothod may be called twice in `createQuery`

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26300: Assignee: Apache Spark > The `checkForStreaming` mothod may be called twice in

[jira] [Created] (SPARK-26300) The `checkForStreaming` mothod may be called twice in `createQuery`

2018-12-06 Thread liuxian (JIRA)
liuxian created SPARK-26300: --- Summary: The `checkForStreaming` mothod may be called twice in `createQuery` Key: SPARK-26300 URL: https://issues.apache.org/jira/browse/SPARK-26300 Project: Spark

[jira] [Assigned] (SPARK-26263) Throw exception when Partition column value can't be converted to user specified type

2018-12-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26263: --- Assignee: Gengliang Wang > Throw exception when Partition column value can't be converted

[jira] [Resolved] (SPARK-26263) Throw exception when Partition column value can't be converted to user specified type

2018-12-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26263. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23215

[jira] [Commented] (SPARK-26051) Can't create table with column name '22222d'

2018-12-06 Thread Xie Juntao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712279#comment-16712279 ] Xie Juntao commented on SPARK-26051: [~dkbiswal] hi, I tested in mysql. It's ok for creating a table

[jira] [Comment Edited] (SPARK-26051) Can't create table with column name '22222d'

2018-12-06 Thread Xie Juntao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712279#comment-16712279 ] Xie Juntao edited comment on SPARK-26051 at 12/7/18 2:53 AM: - [~dkbiswal]

[jira] [Updated] (SPARK-26299) [MAVEN] Shaded test jar in spark-streaming cause deploy test jar twice

2018-12-06 Thread Liu, Linhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu, Linhong updated SPARK-26299: - Description: In spark root pom, there is a test-jar execution for all sub-projects including

[jira] [Updated] (SPARK-26299) [MAVEN] Shaded test jar in spark-streaming cause deploy test jar twice

2018-12-06 Thread Liu, Linhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu, Linhong updated SPARK-26299: - Description: In spark root pom, there is a test-jar execution for all sub-projects including

[jira] [Updated] (SPARK-26299) [MAVEN] Shaded test jar in spark-streaming cause deploy test jar twice

2018-12-06 Thread Liu, Linhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu, Linhong updated SPARK-26299: - Affects Version/s: 2.3.2 > [MAVEN] Shaded test jar in spark-streaming cause deploy test jar

[jira] [Updated] (SPARK-26299) [MAVEN] Shaded test jar in spark-streaming cause deploy test jar twice

2018-12-06 Thread Liu, Linhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu, Linhong updated SPARK-26299: - Description: In spark root pom, there is a test-jar execution for all sub-projects including

[jira] [Created] (SPARK-26299) [MAVEN] Shaded test jar in spark-streaming cause deploy test jar twice

2018-12-06 Thread Liu, Linhong (JIRA)
Liu, Linhong created SPARK-26299: Summary: [MAVEN] Shaded test jar in spark-streaming cause deploy test jar twice Key: SPARK-26299 URL: https://issues.apache.org/jira/browse/SPARK-26299 Project:

[jira] [Resolved] (SPARK-26289) cleanup enablePerfMetrics parameter from BytesToBytesMap

2018-12-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26289. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23244

[jira] [Assigned] (SPARK-26289) cleanup enablePerfMetrics parameter from BytesToBytesMap

2018-12-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26289: --- Assignee: caoxuewen > cleanup enablePerfMetrics parameter from BytesToBytesMap >

[jira] [Commented] (SPARK-26295) [K8S] serviceAccountName is not set in client mode

2018-12-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712192#comment-16712192 ] Marcelo Vanzin commented on SPARK-26295: Can't SPARK-25887 be used for this? > [K8S]

[jira] [Commented] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712116#comment-16712116 ] Dongjoon Hyun commented on SPARK-25987: --- It's not directly related to this, but it seems that we

[jira] [Assigned] (SPARK-26298) Upgrade Janino version to 3.0.11

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26298: Assignee: Apache Spark > Upgrade Janino version to 3.0.11 >

[jira] [Commented] (SPARK-26298) Upgrade Janino version to 3.0.11

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712089#comment-16712089 ] Apache Spark commented on SPARK-26298: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-26298) Upgrade Janino version to 3.0.11

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712090#comment-16712090 ] Apache Spark commented on SPARK-26298: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-26298) Upgrade Janino version to 3.0.11

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26298: Assignee: (was: Apache Spark) > Upgrade Janino version to 3.0.11 >

[jira] [Resolved] (SPARK-26194) Support automatic spark.authenticate secret in Kubernetes backend

2018-12-06 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah resolved SPARK-26194. Resolution: Fixed Fix Version/s: 3.0.0 > Support automatic spark.authenticate secret in

[jira] [Created] (SPARK-26298) Upgrade Janino version to 3.0.11

2018-12-06 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-26298: - Summary: Upgrade Janino version to 3.0.11 Key: SPARK-26298 URL: https://issues.apache.org/jira/browse/SPARK-26298 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712067#comment-16712067 ] Dongjoon Hyun edited comment on SPARK-25987 at 12/6/18 10:06 PM: - Hi,

[jira] [Updated] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25987: -- Affects Version/s: 3.0.0 > StackOverflowError when executing many operations on a table with

[jira] [Commented] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712067#comment-16712067 ] Dongjoon Hyun commented on SPARK-25987: --- Hi, [~kiszk]. The situation is very similar with

[jira] [Updated] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25987: -- Description: When I execute {code:java} import org.apache.spark.sql._ import

[jira] [Updated] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25987: -- Affects Version/s: 2.4.0 > StackOverflowError when executing many operations on a table with

[jira] [Updated] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25987: -- Description: When I execute {code:java} import org.apache.spark.sql._ import

[jira] [Commented] (SPARK-22776) Increase default value of spark.sql.codegen.maxFields

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711988#comment-16711988 ] Dongjoon Hyun commented on SPARK-22776: --- Hi, [~kiszk]. Is there anything to do here? > Increase

[jira] [Closed] (SPARK-26067) Pandas GROUPED_MAP udf breaks if DF has >255 columns

2018-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-26067. - > Pandas GROUPED_MAP udf breaks if DF has >255 columns >

[jira] [Updated] (SPARK-26286) Add MAXIMUM_PAGE_SIZE_BYTES Exception unit test

2018-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26286: -- Priority: Trivial (was: Minor) > Add MAXIMUM_PAGE_SIZE_BYTES Exception unit test >

[jira] [Resolved] (SPARK-26292) Assert statement of currentPage may be not in right place

2018-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26292. --- Resolution: Not A Problem Fix Version/s: (was: 2.4.0) Please also dont' set Fix Version

[jira] [Updated] (SPARK-26294) Delete Unnecessary If statement

2018-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26294: -- Priority: Trivial (was: Minor) This is too trivial for a JIRA. > Delete Unnecessary If statement >

[jira] [Resolved] (SPARK-26236) Kafka delegation token support documentation

2018-12-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26236. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23195

[jira] [Assigned] (SPARK-26236) Kafka delegation token support documentation

2018-12-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26236: -- Assignee: Gabor Somogyi > Kafka delegation token support documentation >

[jira] [Comment Edited] (SPARK-26254) Move delegation token providers into a separate project

2018-12-06 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711516#comment-16711516 ] Gabor Somogyi edited comment on SPARK-26254 at 12/6/18 6:32 PM: I've

[jira] [Resolved] (SPARK-26213) Custom Receiver for Structured streaming

2018-12-06 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi resolved SPARK-26213. --- Resolution: Information Provided > Custom Receiver for Structured streaming >

[jira] [Resolved] (SPARK-26296) Base Spark over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26296. Resolution: Won't Fix Lack of newer JDK support is not only the fault of Scala. There's a

[jira] [Resolved] (SPARK-25274) Improve toPandas with Arrow by sending out-of-order record batches

2018-12-06 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-25274. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22275

[jira] [Assigned] (SPARK-25274) Improve toPandas with Arrow by sending out-of-order record batches

2018-12-06 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-25274: Assignee: Bryan Cutler > Improve toPandas with Arrow by sending out-of-order record

[jira] [Assigned] (SPARK-26297) improve the doc of Distribution/Partitioning

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26297: Assignee: Apache Spark (was: Wenchen Fan) > improve the doc of

[jira] [Commented] (SPARK-26297) improve the doc of Distribution/Partitioning

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711628#comment-16711628 ] Apache Spark commented on SPARK-26297: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26297) improve the doc of Distribution/Partitioning

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26297: Assignee: Wenchen Fan (was: Apache Spark) > improve the doc of

[jira] [Created] (SPARK-26297) improve the doc of Distribution/Partitioning

2018-12-06 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-26297: --- Summary: improve the doc of Distribution/Partitioning Key: SPARK-26297 URL: https://issues.apache.org/jira/browse/SPARK-26297 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-26244) Do not use case class as public API

2018-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26244. --- Resolution: Done > Do not use case class as public API > --- > >

[jira] [Commented] (SPARK-26254) Move delegation token providers into a separate project

2018-12-06 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711516#comment-16711516 ] Gabor Somogyi commented on SPARK-26254: --- I've reached a state where tradeoff has to be made, so

[jira] [Commented] (SPARK-26265) deadlock between TaskMemoryManager and BytesToBytesMap$MapIterator

2018-12-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711500#comment-16711500 ] Hyukjin Kwon commented on SPARK-26265: -- Thanks, [~qianhan], can you investigate and make a fix to

[jira] [Reopened] (SPARK-26265) deadlock between TaskMemoryManager and BytesToBytesMap$MapIterator

2018-12-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-26265: -- > deadlock between TaskMemoryManager and BytesToBytesMap$MapIterator >

[jira] [Updated] (SPARK-26288) add initRegisteredExecutorsDB in ExternalShuffleService

2018-12-06 Thread weixiuli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-26288: - Description: As we all know that spark on Yarn uses DB to record RegisteredExecutors information which

[jira] [Updated] (SPARK-26296) Base Spark over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread M. Le Bihan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M. Le Bihan updated SPARK-26296: Description: I am not using _Scala_ when I am programming _Spark_ but plain _Java_. I  believe

[jira] [Updated] (SPARK-26296) Spark build over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread M. Le Bihan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M. Le Bihan updated SPARK-26296: Description: I am not using _Scala_ when I am programming _Spark_ but plain _Java_. I  believe

[jira] [Updated] (SPARK-26296) Spark build over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread M. Le Bihan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M. Le Bihan updated SPARK-26296: Description: I am not using _Scala_ when I am programming _Spark_ but plain _Java_. I  believe

[jira] [Updated] (SPARK-26296) Spark build over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread M. Le Bihan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M. Le Bihan updated SPARK-26296: Description: I am not using _Scala_ when I am programming _Spark_ but plain _Java_. I  believe

[jira] [Updated] (SPARK-26296) Spark build over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread M. Le Bihan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M. Le Bihan updated SPARK-26296: Description: I am not using _Scala_ when I am programming _Spark_ but plain _Java_. I  believe

[jira] [Updated] (SPARK-26296) Base Spark over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread M. Le Bihan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M. Le Bihan updated SPARK-26296: Summary: Base Spark over Java and not over Scala, offering Scala as an option over Spark (was:

[jira] [Commented] (SPARK-26295) [K8S] serviceAccountName is not set in client mode

2018-12-06 Thread Adrian Tanase (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711464#comment-16711464 ] Adrian Tanase commented on SPARK-26295: --- ping [~vanzin],  > [K8S] serviceAccountName is not set

[jira] [Commented] (SPARK-26295) [K8S] serviceAccountName is not set in client mode

2018-12-06 Thread Adrian Tanase (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711462#comment-16711462 ] Adrian Tanase commented on SPARK-26295: --- I would be happy to give it a shot with a bit of

[jira] [Updated] (SPARK-26296) Spark build over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread M. Le Bihan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M. Le Bihan updated SPARK-26296: Description: I am not using _Scala_ when I am programming _Spark_ but plain _Java_. I  believe

[jira] [Created] (SPARK-26296) Spark build over Java and not over Scala, offering Scala as an option over Spark

2018-12-06 Thread M. Le Bihan (JIRA)
M. Le Bihan created SPARK-26296: --- Summary: Spark build over Java and not over Scala, offering Scala as an option over Spark Key: SPARK-26296 URL: https://issues.apache.org/jira/browse/SPARK-26296

[jira] [Comment Edited] (SPARK-24417) Build and Run Spark on JDK11

2018-12-06 Thread M. Le Bihan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710091#comment-16710091 ] M. Le Bihan edited comment on SPARK-24417 at 12/6/18 1:15 PM: -- Hello, 

[jira] [Updated] (SPARK-26295) [K8S] serviceAccountName is not set in client mode

2018-12-06 Thread Adrian Tanase (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Tanase updated SPARK-26295: -- Description: When deploying spark apps in client mode (in my case from inside the driver

[jira] [Updated] (SPARK-26295) [K8S] serviceAccountName is not set in client mode

2018-12-06 Thread Adrian Tanase (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Tanase updated SPARK-26295: -- Description: When deploying spark apps in client mode (in my case from inside the driver

[jira] [Created] (SPARK-26295) [K8S] serviceAccountName is not set in client mode

2018-12-06 Thread Adrian Tanase (JIRA)
Adrian Tanase created SPARK-26295: - Summary: [K8S] serviceAccountName is not set in client mode Key: SPARK-26295 URL: https://issues.apache.org/jira/browse/SPARK-26295 Project: Spark Issue

[jira] [Commented] (SPARK-26293) Cast exception when having python udf in subquery

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711360#comment-16711360 ] Apache Spark commented on SPARK-26293: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26293) Cast exception when having python udf in subquery

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26293: Assignee: Apache Spark (was: Wenchen Fan) > Cast exception when having python udf in

[jira] [Assigned] (SPARK-26293) Cast exception when having python udf in subquery

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26293: Assignee: Wenchen Fan (was: Apache Spark) > Cast exception when having python udf in

[jira] [Commented] (SPARK-26294) Delete Unnecessary If statement

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711341#comment-16711341 ] Apache Spark commented on SPARK-26294: -- User 'wangjiaochun' has created a pull request for this

[jira] [Assigned] (SPARK-26294) Delete Unnecessary If statement

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26294: Assignee: Apache Spark > Delete Unnecessary If statement >

[jira] [Assigned] (SPARK-26294) Delete Unnecessary If statement

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26294: Assignee: (was: Apache Spark) > Delete Unnecessary If statement >

[jira] [Created] (SPARK-26294) Delete Unnecessary If statement

2018-12-06 Thread wangjiaochun (JIRA)
wangjiaochun created SPARK-26294: Summary: Delete Unnecessary If statement Key: SPARK-26294 URL: https://issues.apache.org/jira/browse/SPARK-26294 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-26292) Assert statement of currentPage may be not in right place

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711309#comment-16711309 ] Apache Spark commented on SPARK-26292: -- User 'wangjiaochun' has created a pull request for this

[jira] [Assigned] (SPARK-26292) Assert statement of currentPage may be not in right place

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26292: Assignee: (was: Apache Spark) > Assert statement of currentPage may be not in right

[jira] [Commented] (SPARK-18165) Kinesis support in Structured Streaming

2018-12-06 Thread Vikram Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711319#comment-16711319 ] Vikram Agrawal commented on SPARK-18165: Hi [~danielil] - right now it is available at

[jira] [Commented] (SPARK-26292) Assert statement of currentPage may be not in right place

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711317#comment-16711317 ] Apache Spark commented on SPARK-26292: -- User 'wangjiaochun' has created a pull request for this

[jira] [Commented] (SPARK-18147) Broken Spark SQL Codegen

2018-12-06 Thread Eli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711316#comment-16711316 ] Eli commented on SPARK-18147: - [~mgaido]  it work well on 2.3.2 .    It should be a good idea to upgrade to

[jira] [Assigned] (SPARK-26292) Assert statement of currentPage may be not in right place

2018-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26292: Assignee: Apache Spark > Assert statement of currentPage may be not in right place >

[jira] [Created] (SPARK-26293) Cast exception when having python udf in subquery

2018-12-06 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-26293: --- Summary: Cast exception when having python udf in subquery Key: SPARK-26293 URL: https://issues.apache.org/jira/browse/SPARK-26293 Project: Spark Issue Type:

[jira] [Created] (SPARK-26292) Assert statement of currentPage may be not in right place

2018-12-06 Thread wangjiaochun (JIRA)
wangjiaochun created SPARK-26292: Summary: Assert statement of currentPage may be not in right place Key: SPARK-26292 URL: https://issues.apache.org/jira/browse/SPARK-26292 Project: Spark

  1   2   >