[jira] [Commented] (SPARK-31622) Test-jar in the Spark distribution

2023-09-27 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769558#comment-17769558 ] Arseniy Tashoyan commented on SPARK-31622: -- In Spark 3.4.1 there are no test jars in the

[jira] [Commented] (SPARK-42425) spark-hadoop-cloud is not provided in the default Spark distribution

2023-03-11 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699248#comment-17699248 ] Arseniy Tashoyan commented on SPARK-42425: -- The doc says to declare this dependency as

[jira] [Created] (SPARK-42425) spark-hadoop-cloud is not provided in the default Spark distribution

2023-02-13 Thread Arseniy Tashoyan (Jira)
Arseniy Tashoyan created SPARK-42425: Summary: spark-hadoop-cloud is not provided in the default Spark distribution Key: SPARK-42425 URL: https://issues.apache.org/jira/browse/SPARK-42425

[jira] [Comment Edited] (SPARK-34345) Allow several properties files

2021-03-25 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307644#comment-17307644 ] Arseniy Tashoyan edited comment on SPARK-34345 at 3/25/21, 7:52 AM:

[jira] [Updated] (SPARK-34345) Allow several properties files

2021-03-24 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-34345: - Component/s: Kubernetes > Allow several properties files >

[jira] [Commented] (SPARK-34345) Allow several properties files

2021-03-24 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307644#comment-17307644 ] Arseniy Tashoyan commented on SPARK-34345: -- Typesafe Config seems attractive, as it provides

[jira] [Updated] (SPARK-34345) Allow several properties files

2021-03-24 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-34345: - Affects Version/s: 3.1.1 > Allow several properties files >

[jira] [Updated] (SPARK-34345) Allow several properties files

2021-03-24 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-34345: - Priority: Major (was: Minor) > Allow several properties files >

[jira] [Updated] (SPARK-31531) sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner() method not found during spark-submit

2021-03-24 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-31531: - Attachment: image.png > sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner() method not

[jira] [Updated] (SPARK-34345) Allow several properties files

2021-02-03 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-34345: - Description: Example: we have 2 applications A and B. These applications have some

[jira] [Updated] (SPARK-34345) Allow several properties files

2021-02-03 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-34345: - Description: Example: we have 2 applications A and B. These applications have some

[jira] [Created] (SPARK-34345) Allow several files property files

2021-02-03 Thread Arseniy Tashoyan (Jira)
Arseniy Tashoyan created SPARK-34345: Summary: Allow several files property files Key: SPARK-34345 URL: https://issues.apache.org/jira/browse/SPARK-34345 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34345) Allow several properties files

2021-02-03 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-34345: - Summary: Allow several properties files (was: Allow several files property files) >

[jira] [Updated] (SPARK-31805) Spark Structured Streaming with "assign" KafkaConsumer mode still uses group.id

2020-05-23 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-31805: - Description: [Spark Structured Streaming - Kafka

[jira] [Updated] (SPARK-31805) Spark Structured Streaming with "assign" KafkaConsumer mode still uses group.id

2020-05-23 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-31805: - Description: [Spark Structured Streaming - Kafka

[jira] [Created] (SPARK-31805) Spark Structured Streaming with "assign" KafkaConsumer mode still uses group.id

2020-05-23 Thread Arseniy Tashoyan (Jira)
Arseniy Tashoyan created SPARK-31805: Summary: Spark Structured Streaming with "assign" KafkaConsumer mode still uses group.id Key: SPARK-31805 URL: https://issues.apache.org/jira/browse/SPARK-31805

[jira] [Commented] (SPARK-30267) avro deserializer: ArrayList cannot be cast to GenericData$Array

2020-05-10 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103871#comment-17103871 ] Arseniy Tashoyan commented on SPARK-30267: -- Hi [~Gengliang.Wang], Surprisingly, my tests run

[jira] [Reopened] (SPARK-30267) avro deserializer: ArrayList cannot be cast to GenericData$Array

2020-05-08 Thread Arseniy Tashoyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan reopened SPARK-30267: -- With Spark 3.0.0 preview 2, I have the following failure here: {code:java}

[jira] [Created] (SPARK-31622) Test-jar in the Spark distribution

2020-05-01 Thread Arseniy Tashoyan (Jira)
Arseniy Tashoyan created SPARK-31622: Summary: Test-jar in the Spark distribution Key: SPARK-31622 URL: https://issues.apache.org/jira/browse/SPARK-31622 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23899) Built-in SQL Function Improvement

2018-12-02 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706156#comment-16706156 ] Arseniy Tashoyan commented on SPARK-23899: -- What do you think about this one: SPARK-23693? >

[jira] [Updated] (SPARK-20389) Upgrade kryo to fix NegativeArraySizeException

2018-05-21 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-20389: - Affects Version/s: 2.2.1 > Upgrade kryo to fix NegativeArraySizeException >

[jira] [Commented] (SPARK-20389) Upgrade kryo to fix NegativeArraySizeException

2018-05-21 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482408#comment-16482408 ] Arseniy Tashoyan commented on SPARK-20389: -- I confirm: the issue is in place for Spark 2.2.1. I

[jira] [Updated] (SPARK-23693) SQL function uuid()

2018-03-15 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-23693: - Description: Add function uuid() to org.apache.spark.sql.functions that returns

[jira] [Updated] (SPARK-23693) SQL function uuid()

2018-03-15 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-23693: - Description: Add function uuid() to org.apache.spark.sql.functions that returns

[jira] [Updated] (SPARK-23693) SQL function uuid()

2018-03-15 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-23693: - Description: Add function uuid() to org.apache.spark.sql.functions that returns

[jira] [Updated] (SPARK-23693) SQL function uuid()

2018-03-15 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-23693: - Description: Add function uuid() to org.apache.spark.sql.functions that returns

[jira] [Created] (SPARK-23693) SQL function uuid()

2018-03-15 Thread Arseniy Tashoyan (JIRA)
Arseniy Tashoyan created SPARK-23693: Summary: SQL function uuid() Key: SPARK-23693 URL: https://issues.apache.org/jira/browse/SPARK-23693 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23318) FP-growth: WARN FPGrowth: Input data is not cached

2018-02-08 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357482#comment-16357482 ] Arseniy Tashoyan commented on SPARK-23318: -- I want. But I'm short on time now. Will do. >

[jira] [Created] (SPARK-23318) FP-growth: WARN FPGrowth: Input data is not cached

2018-02-02 Thread Arseniy Tashoyan (JIRA)
Arseniy Tashoyan created SPARK-23318: Summary: FP-growth: WARN FPGrowth: Input data is not cached Key: SPARK-23318 URL: https://issues.apache.org/jira/browse/SPARK-23318 Project: Spark

[jira] [Commented] (SPARK-23269) FP-growth: Provide last transaction for each detected frequent pattern

2018-02-02 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350120#comment-16350120 ] Arseniy Tashoyan commented on SPARK-23269: -- Sure I can query, but this requires join with

[jira] [Updated] (SPARK-23269) FP-growth: Provide last transaction for each detected frequent pattern

2018-01-30 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-23269: - Description: FP-growth implementation gives patterns and their frequences:

[jira] [Created] (SPARK-23269) FP-growth: Provide last transaction for each detected frequent pattern

2018-01-30 Thread Arseniy Tashoyan (JIRA)
Arseniy Tashoyan created SPARK-23269: Summary: FP-growth: Provide last transaction for each detected frequent pattern Key: SPARK-23269 URL: https://issues.apache.org/jira/browse/SPARK-23269

[jira] [Commented] (SPARK-22471) SQLListener consumes much memory causing OutOfMemoryError

2017-11-08 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244836#comment-16244836 ] Arseniy Tashoyan commented on SPARK-22471: -- I am coming up with a simple fix, backportable to

[jira] [Updated] (SPARK-22471) SQLListener consumes much memory causing OutOfMemoryError

2017-11-08 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-22471: - Attachment: SQLListener_retained_size.png > SQLListener consumes much memory causing

[jira] [Updated] (SPARK-22471) SQLListener consumes much memory causing OutOfMemoryError

2017-11-08 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-22471: - Attachment: SQLListener_stageIdToStageMetrics_retained_size.png > SQLListener consumes

[jira] [Created] (SPARK-22471) SQLListener consumes much memory causing OutOfMemoryError

2017-11-08 Thread Arseniy Tashoyan (JIRA)
Arseniy Tashoyan created SPARK-22471: Summary: SQLListener consumes much memory causing OutOfMemoryError Key: SPARK-22471 URL: https://issues.apache.org/jira/browse/SPARK-22471 Project: Spark

[jira] [Reopened] (SPARK-21668) Ability to run driver programs within a container

2017-08-08 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan reopened SPARK-21668: -- > Ability to run driver programs within a container >

[jira] [Comment Edited] (SPARK-21668) Ability to run driver programs within a container

2017-08-08 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118468#comment-16118468 ] Arseniy Tashoyan edited comment on SPARK-21668 at 8/8/17 3:57 PM: -- I

[jira] [Comment Edited] (SPARK-21668) Ability to run driver programs within a container

2017-08-08 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118468#comment-16118468 ] Arseniy Tashoyan edited comment on SPARK-21668 at 8/8/17 3:55 PM: -- I

[jira] [Commented] (SPARK-21668) Ability to run driver programs within a container

2017-08-08 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118468#comment-16118468 ] Arseniy Tashoyan commented on SPARK-21668: -- I don't think it is a duplicate of [ SPARK-6680 ].

[jira] [Created] (SPARK-21668) Ability to run driver programs within a container

2017-08-08 Thread Arseniy Tashoyan (JIRA)
Arseniy Tashoyan created SPARK-21668: Summary: Ability to run driver programs within a container Key: SPARK-21668 URL: https://issues.apache.org/jira/browse/SPARK-21668 Project: Spark