[jira] [Commented] (SPARK-27053) How about allowing engineers to use a different ExecutorBackend in StandAlone mode?

[ https://issues.apache.org/jira/browse/SPARK-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786520#comment-16786520 ] Hyukjin Kwon commented on SPARK-27053: -- Let's better make a discussion thread in the mailing list

[jira] [Commented] (SPARK-27069) Spark(2.3.2) LDA transfomation memory error(java.lang.OutOfMemoryError at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:1232

[ https://issues.apache.org/jira/browse/SPARK-27069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786519#comment-16786519 ] TAESUK KIM commented on SPARK-27069: I'm sorry for that. My mistake > Spark(2.3.2) LDA

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos commented on SPARK-26742: - This is one way to run things: ```

[jira] [Commented] (SPARK-27006) SPIP: .NET bindings for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786648#comment-16786648 ] Steve Loughran commented on SPARK-27006: I can see the appeal in having some extension API which

[jira] [Created] (SPARK-27084) Add function alias for Bitwise functions

Zhu, Lipeng created SPARK-27084: --- Summary: Add function alias for Bitwise functions Key: SPARK-27084 URL: https://issues.apache.org/jira/browse/SPARK-27084 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-27084) Add function alias for Bitwise functions

[ https://issues.apache.org/jira/browse/SPARK-27084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27084: Assignee: Apache Spark > Add function alias for Bitwise functions >

[jira] [Created] (SPARK-27085) Migrate CSV to File Data Source V2

Gengliang Wang created SPARK-27085: -- Summary: Migrate CSV to File Data Source V2 Key: SPARK-27085 URL: https://issues.apache.org/jira/browse/SPARK-27085 Project: Spark Issue Type: Task

[jira] [Assigned] (SPARK-27083) Add a config to control subqueryReuse

[ https://issues.apache.org/jira/browse/SPARK-27083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27083: Assignee: Apache Spark > Add a config to control subqueryReuse >

[jira] [Comment Edited] (SPARK-27017) Creating orc table with special symbols in column name via spark.sql

[ https://issues.apache.org/jira/browse/SPARK-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786797#comment-16786797 ] Henryk Cesnolovic edited comment on SPARK-27017 at 3/7/19 2:05 PM: ---

[jira] [Assigned] (SPARK-27085) Migrate CSV to File Data Source V2

[ https://issues.apache.org/jira/browse/SPARK-27085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27085: Assignee: Apache Spark > Migrate CSV to File Data Source V2 >

[jira] [Assigned] (SPARK-27085) Migrate CSV to File Data Source V2

[ https://issues.apache.org/jira/browse/SPARK-27085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27085: Assignee: (was: Apache Spark) > Migrate CSV to File Data Source V2 >

[jira] [Updated] (SPARK-27086) DataSourceV2 MicroBatchExecution commits last batch only if new batch is constructed

[ https://issues.apache.org/jira/browse/SPARK-27086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Herold updated SPARK-27086: - Description: I wanted to use the new {{DataSourceV2}} API to build a AWS SQS streaming

[jira] [Commented] (SPARK-27017) Creating orc table with special symbols in column name via spark.sql

[ https://issues.apache.org/jira/browse/SPARK-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786789#comment-16786789 ] Chakravarthi commented on SPARK-27017: -- [~uNxe] could you provide queries you have used,because

[jira] [Commented] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple ti

[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786823#comment-16786823 ] Apache Spark commented on SPARK-25250: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Updated] (SPARK-27084) Add function alias for Bitwise functions

[ https://issues.apache.org/jira/browse/SPARK-27084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu, Lipeng updated SPARK-27084: Description: Currently SparkSQL already have functions for bitwise related functions. Is that

[jira] [Commented] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786726#comment-16786726 ] Gabor Somogyi commented on SPARK-26998: --- I've tested the things through and see the issue. I think

[jira] [Assigned] (SPARK-27084) Add function alias for Bitwise functions

[ https://issues.apache.org/jira/browse/SPARK-27084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27084: Assignee: (was: Apache Spark) > Add function alias for Bitwise functions >

[jira] [Created] (SPARK-27087) Inability to access to column alias in pyspark

Vincent created SPARK-27087: --- Summary: Inability to access to column alias in pyspark Key: SPARK-27087 URL: https://issues.apache.org/jira/browse/SPARK-27087 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786781#comment-16786781 ] Chakravarthi commented on SPARK-27072: -- [~vanzin] Thanks for your answer. > Changing the parameter

[jira] [Commented] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple ti

[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786819#comment-16786819 ] Apache Spark commented on SPARK-25250: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Commented] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple ti

[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786817#comment-16786817 ] Apache Spark commented on SPARK-25250: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Commented] (SPARK-27084) Add function alias for Bitwise functions

[ https://issues.apache.org/jira/browse/SPARK-27084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786685#comment-16786685 ] Zhu, Lipeng commented on SPARK-27084: - https://github.com/apache/spark/pull/24004/ > Add function

[jira] [Commented] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple ti

[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786825#comment-16786825 ] Apache Spark commented on SPARK-25250: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Created] (SPARK-27086) DataSourceV2 MicroBatchExecution commits last batch only if new batch is constructed

Sebastian Herold created SPARK-27086: Summary: DataSourceV2 MicroBatchExecution commits last batch only if new batch is constructed Key: SPARK-27086 URL: https://issues.apache.org/jira/browse/SPARK-27086

[jira] [Commented] (SPARK-27017) Creating orc table with special symbols in column name via spark.sql

[ https://issues.apache.org/jira/browse/SPARK-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786797#comment-16786797 ] Henryk Cesnolovic commented on SPARK-27017: --- Hive allows, just use this quotes for column

[jira] [Commented] (SPARK-27076) Getting the timeout error while writing parquet/csv files to s3

[ https://issues.apache.org/jira/browse/SPARK-27076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786503#comment-16786503 ] Hyukjin Kwon commented on SPARK-27076: -- Does this happen in other file formats too? It doesn't

[jira] [Updated] (SPARK-27069) Spark(2.3.1) LDA transfomation memory error(java.lang.OutOfMemoryError at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:1232

[ https://issues.apache.org/jira/browse/SPARK-27069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27069: - Target Version/s: (was: 2.3.2) > Spark(2.3.1) LDA transfomation memory

[jira] [Resolved] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27072. -- Resolution: Not A Problem > Changing the parameter value of completedJob.sort to X prints

[jira] [Resolved] (SPARK-27053) How about allowing engineers to use a different ExecutorBackend in StandAlone mode?

[ https://issues.apache.org/jira/browse/SPARK-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27053. -- Resolution: Invalid > How about allowing engineers to use a different ExecutorBackend in

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:18 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:19 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:21 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:49 AM: -

[jira] [Commented] (SPARK-9860) Join: Determine the join strategy (broadcast join or shuffle join) at runtime

[ https://issues.apache.org/jira/browse/SPARK-9860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786627#comment-16786627 ] ant_nebula commented on SPARK-9860: --- Is it really "IN PROGRESS"? It looks like it's stalled > Join:

[jira] [Created] (SPARK-27083) Add a config to control subqueryReuse

liuxian created SPARK-27083: --- Summary: Add a config to control subqueryReuse Key: SPARK-27083 URL: https://issues.apache.org/jira/browse/SPARK-27083 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:20 AM: -

[jira] [Commented] (SPARK-24295) Purge Structured streaming FileStreamSinkLog metadata compact file data.

[ https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786569#comment-16786569 ] Jungtaek Lim commented on SPARK-24295: -- [~iqbal_khattra] [~alfredo-gimenez-bv] Would we be happy

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:47 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:48 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:46 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786573#comment-16786573 ] Jiaxin Shan edited comment on SPARK-26742 at 3/7/19 9:55 AM: - [~skonto]

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786573#comment-16786573 ] Jiaxin Shan commented on SPARK-26742: - [~skonto] Thanks! This is really helpful. I am running

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 10:12 AM:

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 10:11 AM:

[jira] [Resolved] (SPARK-27077) DataFrameReader and Number of Connection Limitation

[ https://issues.apache.org/jira/browse/SPARK-27077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27077. -- Resolution: Not A Problem > DataFrameReader and Number of Connection Limitation >

[jira] [Resolved] (SPARK-25863) java.lang.UnsupportedOperationException: empty.max at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.updateAndGetCompilationStats(CodeGenerator.scala:

[ https://issues.apache.org/jira/browse/SPARK-25863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-25863. -- Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 3.0.0

[jira] [Updated] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27060: - Description: Seems to be a compatibility issue compared to other components such as hive and

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:18 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:17 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:17 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:42 AM: -

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786581#comment-16786581 ] Stavros Kontopoulos commented on SPARK-26742: - You are probably trying to launch in the

[jira] [Updated] (SPARK-27082) Dynamic Allocation: we should consider the scenario that speculative task being killed and never resubmit

[ https://issues.apache.org/jira/browse/SPARK-27082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Fan updated SPARK-27082: - Description: Issue background: When we enable dynamic allocation, we expect that the executors can be

[jira] [Commented] (SPARK-27083) Add a config to control subqueryReuse

[ https://issues.apache.org/jira/browse/SPARK-27083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786653#comment-16786653 ] Apache Spark commented on SPARK-27083: -- User '10110346' has created a pull request for this issue:

[jira] [Updated] (SPARK-27069) Spark(2.3.1) LDA transfomation memory error(java.lang.OutOfMemoryError at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:1232

[ https://issues.apache.org/jira/browse/SPARK-27069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27069: - Description: I trained LDA(feature dimension : 100, iteration: 100 or 50, Distributed version

[jira] [Updated] (SPARK-25863) java.lang.UnsupportedOperationException: empty.max at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.updateAndGetCompilationStats(CodeGenerator.scala:1

[ https://issues.apache.org/jira/browse/SPARK-25863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-25863: - Fix Version/s: 2.4.2 2.3.4 >

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:44 AM: -

[jira] [Commented] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786619#comment-16786619 ] Piotr Kołaczkowski commented on SPARK-27018: The file does not exist on master branch. Where

[jira] [Created] (SPARK-27082) Dynamic Allocation: we should consider the scenario that speculative task being killed and never resubmit

Zhen Fan created SPARK-27082: Summary: Dynamic Allocation: we should consider the scenario that speculative task being killed and never resubmit Key: SPARK-27082 URL: https://issues.apache.org/jira/browse/SPARK-27082

[jira] [Assigned] (SPARK-27083) Add a config to control subqueryReuse

[ https://issues.apache.org/jira/browse/SPARK-27083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27083: Assignee: (was: Apache Spark) > Add a config to control subqueryReuse >

[jira] [Commented] (SPARK-27069) Spark(2.3.1) LDA transfomation memory error(java.lang.OutOfMemoryError at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:1232

[ https://issues.apache.org/jira/browse/SPARK-27069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786506#comment-16786506 ] Hyukjin Kwon commented on SPARK-27069: -- Please avoid to set the target version which is usually

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786532#comment-16786532 ] Sachin Ramachandra Setty commented on SPARK-27060: -- I verified in PostgreSQL as well

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:50 AM: -

[jira] [Commented] (SPARK-26868) Duplicate error message for implicit cartesian product in verbose explain

[ https://issues.apache.org/jira/browse/SPARK-26868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786596#comment-16786596 ] Sandeep Katta commented on SPARK-26868: --- [~maropu] how you wanted to print the error message in

[jira] [Updated] (SPARK-27069) Spark(2.3.2) LDA transfomation memory error(java.lang.OutOfMemoryError at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:1232

[ https://issues.apache.org/jira/browse/SPARK-27069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAESUK KIM updated SPARK-27069: --- Summary: Spark(2.3.2) LDA transfomation memory error(java.lang.OutOfMemoryError at

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786513#comment-16786513 ] Hyukjin Kwon commented on SPARK-27060: -- Looks Spark is more permissive, which apparently looks

[jira] [Comment Edited] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786532#comment-16786532 ] Sachin Ramachandra Setty edited comment on SPARK-27060 at 3/7/19 9:10 AM:

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:18 AM: -

[jira] [Comment Edited] (SPARK-26868) Duplicate error message for implicit cartesian product in verbose explain

[ https://issues.apache.org/jira/browse/SPARK-26868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786596#comment-16786596 ] Sandeep Katta edited comment on SPARK-26868 at 3/7/19 10:26 AM:

[jira] [Commented] (SPARK-25863) java.lang.UnsupportedOperationException: empty.max at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.updateAndGetCompilationStats(CodeGenerator.scala

[ https://issues.apache.org/jira/browse/SPARK-25863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786512#comment-16786512 ] Takeshi Yamamuro commented on SPARK-25863: -- [~vfeldsher] Can you check if your query works well

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:58 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 10:11 AM:

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:21 AM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786543#comment-16786543 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:41 AM: -

[jira] [Updated] (SPARK-27082) Dynamic Allocation: we should consider the scenario that speculative task being killed and never resubmit

[ https://issues.apache.org/jira/browse/SPARK-27082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Fan updated SPARK-27082: - Flags: Patch Labels: patch (was: ) Description: Issue background: When we enable

[jira] [Commented] (SPARK-27039) toPandas with Arrow swallows maxResultSize errors

[ https://issues.apache.org/jira/browse/SPARK-27039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786658#comment-16786658 ] peay commented on SPARK-27039: -- For reference, I've realized you can also get an incomplete but non-empty

[jira] [Comment Edited] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783478#comment-16783478 ] Gabor Somogyi edited comment on SPARK-27027 at 3/7/19 4:36 PM: --- What I've

[jira] [Resolved] (SPARK-23836) Support returning StructType to the level support in GroupedMap Arrow's "scalar" UDFS (or similar)

[ https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-23836. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23900

[jira] [Commented] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786984#comment-16786984 ] Marco Gaido commented on SPARK-27018: - The PeriodicCheckpointer is still there in master, you can

[jira] [Commented] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786924#comment-16786924 ] Anne Holler commented on SPARK-26247: - Hi, Sean, Thanks for your comments! >There are two issues

[jira] [Assigned] (SPARK-27071) Expose additional metrics in status.api.v1.StageData

[ https://issues.apache.org/jira/browse/SPARK-27071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27071: Assignee: (was: Apache Spark) > Expose additional metrics in status.api.v1.StageData

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786969#comment-16786969 ] Gabor Somogyi commented on SPARK-27027: --- Hmm, seems like the converted avro binary is exactly the

[jira] [Updated] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple time

[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25250: - Fix Version/s: 2.3.4 > Race condition with tasks running when new attempt for same stage is

[jira] [Created] (SPARK-27088) Apply conf "spark.sql.optimizer.planChangeLog.level" to batch plan change in RuleExecutor

Maryann Xue created SPARK-27088: --- Summary: Apply conf "spark.sql.optimizer.planChangeLog.level" to batch plan change in RuleExecutor Key: SPARK-27088 URL: https://issues.apache.org/jira/browse/SPARK-27088

[jira] [Comment Edited] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783478#comment-16783478 ] Gabor Somogyi edited comment on SPARK-27027 at 3/7/19 4:44 PM: --- What I've

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786978#comment-16786978 ] Gabor Somogyi commented on SPARK-27027: --- [~dongjoon] [~Gengliang.Wang] do you have any idea what

[jira] [Assigned] (SPARK-23836) Support returning StructType to the level support in GroupedMap Arrow's "scalar" UDFS (or similar)

[ https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-23836: Assignee: Bryan Cutler > Support returning StructType to the level support in GroupedMap

[jira] [Assigned] (SPARK-27071) Expose additional metrics in status.api.v1.StageData

[ https://issues.apache.org/jira/browse/SPARK-27071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27071: Assignee: Apache Spark > Expose additional metrics in status.api.v1.StageData >

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786991#comment-16786991 ] Dongjoon Hyun commented on SPARK-27027: --- Thank you for pinging me. I didn't have a chance to dig

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787049#comment-16787049 ] shane knapp commented on SPARK-26742: - two quick things: 1) i will document the minikube/k8s

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787097#comment-16787097 ] Jiaxin Shan commented on SPARK-26742: - Thanks [~shaneknapp] help on the testing stuff. It would be

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787152#comment-16787152 ] shane knapp commented on SPARK-26742: - [~seedjeffwan] just as an FYI, we're using the kvm2 driver on

[jira] [Assigned] (SPARK-26811) Add DataSourceV2 capabilities to check support for batch append, overwrite, truncate during analysis.

[ https://issues.apache.org/jira/browse/SPARK-26811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26811: Assignee: Apache Spark > Add DataSourceV2 capabilities to check support for batch

[jira] [Assigned] (SPARK-26811) Add DataSourceV2 capabilities to check support for batch append, overwrite, truncate during analysis.

[ https://issues.apache.org/jira/browse/SPARK-26811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26811: Assignee: (was: Apache Spark) > Add DataSourceV2 capabilities to check support for

[jira] [Created] (SPARK-27091) Spark 2.4 having issue with JDBC Hive connection

vaquar khan created SPARK-27091: --- Summary: Spark 2.4 having issue with JDBC Hive connection Key: SPARK-27091 URL: https://issues.apache.org/jira/browse/SPARK-27091 Project: Spark Issue Type:

[jira] [Updated] (SPARK-27091) Spark 2.4 having issue with JDBC Hive connection

[ https://issues.apache.org/jira/browse/SPARK-27091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vaquar khan updated SPARK-27091: Description: I am try to connect Apache Spark 2.4 using Java JDBC with Hive and getting  *Logs

[jira] [Created] (SPARK-27093) Honor ParseMode in AvroFileFormat

Tim Cerexhe created SPARK-27093: --- Summary: Honor ParseMode in AvroFileFormat Key: SPARK-27093 URL: https://issues.apache.org/jira/browse/SPARK-27093 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-27094) Thread interrupt being swallowed while launching executors in YarnAllocator

Marcelo Vanzin created SPARK-27094: -- Summary: Thread interrupt being swallowed while launching executors in YarnAllocator Key: SPARK-27094 URL: https://issues.apache.org/jira/browse/SPARK-27094

[jira] [Updated] (SPARK-27086) DataSourceV2 MicroBatchExecution commits last batch only if new batch is constructed

[ https://issues.apache.org/jira/browse/SPARK-27086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Herold updated SPARK-27086: - Description: I wanted to use the new {{DataSourceV2}} API to build a AWS SQS streaming

[jira] [Created] (SPARK-27095) We depend on silently accepting failures in setup-integration-test-env.sh

holdenk created SPARK-27095: --- Summary: We depend on silently accepting failures in setup-integration-test-env.sh Key: SPARK-27095 URL: https://issues.apache.org/jira/browse/SPARK-27095 Project: Spark

  1   2   >