[jira] [Assigned] (SPARK-23345) Flaky test: FileBasedDataSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-23345: --- Assignee: Liang-Chi Hsieh > Flaky test: FileBasedDataSourceSuite >

[jira] [Resolved] (SPARK-23345) Flaky test: FileBasedDataSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23345. - Resolution: Fixed Fix Version/s: 2.3.0 > Flaky test: FileBasedDataSourceSuite >

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355920#comment-16355920 ] Thomas Graves commented on SPARK-22683: --- If the config is set to 1 which keeps the current behavior

[jira] [Created] (SPARK-23354) spark jdbc does not maintain length of data type when I move data from MS sql server to Oracle using spark jdbc

Lav Patel created SPARK-23354: - Summary: spark jdbc does not maintain length of data type when I move data from MS sql server to Oracle using spark jdbc Key: SPARK-23354 URL:

[jira] [Commented] (SPARK-19870) Repeatable deadlock on BlockInfoManager and TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-19870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355982#comment-16355982 ] Imran Rashid commented on SPARK-19870: -- [~eyalfa] any chance you can share those exectuor logs?

[jira] [Updated] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamstra updated SPARK-21084: - Description: One important application of Spark is to support many notebook users with a single

[jira] [Commented] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355952#comment-16355952 ] Apache Spark commented on SPARK-22279: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355895#comment-16355895 ] Mark Hamstra commented on SPARK-22683: -- A concern that I have is that the discussion seems to be

[jira] [Commented] (SPARK-15176) Job Scheduling Within Application Suffers from Priority Inversion

[ https://issues.apache.org/jira/browse/SPARK-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355880#comment-16355880 ] Alex Duvall commented on SPARK-15176: - Another interested party here - I'd find being able to limit

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355994#comment-16355994 ] Edwina Lu commented on SPARK-23206: --- We ([~jerryshao], [~zhz] and I) are planning a conference call via

[jira] [Commented] (SPARK-23139) Read eventLog file with mixed encodings

[ https://issues.apache.org/jira/browse/SPARK-23139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355946#comment-16355946 ] Marcelo Vanzin commented on SPARK-23139: Even if you change {{file.encoding}}, Spark should be

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355943#comment-16355943 ] Mark Hamstra commented on SPARK-22683: -- I agree that setting the config to 1 should be sufficient to

[jira] [Commented] (SPARK-23105) Spark MLlib, GraphX 2.3 QA umbrella

[ https://issues.apache.org/jira/browse/SPARK-23105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356303#comment-16356303 ] Joseph K. Bradley commented on SPARK-23105: --- Sorry for being AWOL; I unfortunately had to be

[jira] [Commented] (SPARK-22446) Optimizer causing StringIndexerModel's indexer UDF to throw "Unseen label" exception incorrectly for filtered data.

[ https://issues.apache.org/jira/browse/SPARK-22446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356340#comment-16356340 ] Joseph K. Bradley commented on SPARK-22446: --- [~viirya] Did you confirm this is an issue in

[jira] [Created] (SPARK-23355) convertMetastore should not ignore table properties

Dongjoon Hyun created SPARK-23355: - Summary: convertMetastore should not ignore table properties Key: SPARK-23355 URL: https://issues.apache.org/jira/browse/SPARK-23355 Project: Spark Issue

[jira] [Commented] (SPARK-23308) ignoreCorruptFiles should not ignore retryable IOException

[ https://issues.apache.org/jira/browse/SPARK-23308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356271#comment-16356271 ] Márcio Furlani Carmona commented on SPARK-23308: That's true Steve. I totally agree that

[jira] [Commented] (SPARK-23319) Skip PySpark tests for old Pandas and old PyArrow

[ https://issues.apache.org/jira/browse/SPARK-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356359#comment-16356359 ] Apache Spark commented on SPARK-23319: -- User 'ueshin' has created a pull request for this issue:

[jira] [Updated] (SPARK-23300) Print out if Pandas and PyArrow are installed or not in tests

[ https://issues.apache.org/jira/browse/SPARK-23300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23300: - Fix Version/s: (was: 2.4.0) 2.3.0 > Print out if Pandas and PyArrow are

[jira] [Updated] (SPARK-22289) Cannot save LogisticRegressionModel with bounds on coefficients

[ https://issues.apache.org/jira/browse/SPARK-22289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22289: -- Summary: Cannot save LogisticRegressionModel with bounds on coefficients (was: Cannot

[jira] [Updated] (SPARK-22446) Optimizer causing StringIndexerModel's indexer UDF to throw "Unseen label" exception incorrectly for filtered data.

[ https://issues.apache.org/jira/browse/SPARK-22446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22446: -- Affects Version/s: (was: 2.2.0) (was: 2.0.0)

[jira] [Commented] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356354#comment-16356354 ] Weichen Xu commented on SPARK-22700: [~podongfeng] Have you checked other transformers with

[jira] [Reopened] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-22279: --- This will be reverted. > Turn on spark.sql.hive.convertMetastoreOrc by default >

[jira] [Updated] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22279: -- Fix Version/s: (was: 2.3.0) > Turn on spark.sql.hive.convertMetastoreOrc by default >

[jira] [Assigned] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22279: Assignee: Apache Spark > Turn on spark.sql.hive.convertMetastoreOrc by default >

[jira] [Assigned] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22279: Assignee: (was: Apache Spark) > Turn on spark.sql.hive.convertMetastoreOrc by default

[jira] [Assigned] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23314: Assignee: (was: Apache Spark) > Pandas grouped udf on dataset with timestamp column

[jira] [Commented] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356189#comment-16356189 ] Apache Spark commented on SPARK-23314: -- User 'icexelloss' has created a pull request for this issue:

[jira] [Updated] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23348: -- Description:   {code:java} Seq(1 -> "a").toDF("i", "j").write.saveAsTable("t") Seq("c" ->

[jira] [Commented] (SPARK-23318) FP-growth: WARN FPGrowth: Input data is not cached

[ https://issues.apache.org/jira/browse/SPARK-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356111#comment-16356111 ] Sean Owen commented on SPARK-23318: --- [~tashoyan] did you want to submit a PR for this? > FP-growth:

[jira] [Commented] (SPARK-23354) spark jdbc does not maintain length of data type when I move data from MS sql server to Oracle using spark jdbc

[ https://issues.apache.org/jira/browse/SPARK-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356132#comment-16356132 ] Sean Owen commented on SPARK-23354: --- I'm not clear what about this involves Spark. What length do you

[jira] [Updated] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23348: -- Affects Version/s: 2.0.2 > append data using saveAsTable should adjust the data types >

[jira] [Updated] (SPARK-23045) Have RFormula use OneHotEncoderEstimator

[ https://issues.apache.org/jira/browse/SPARK-23045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23045: -- Summary: Have RFormula use OneHotEncoderEstimator (was: Have RFormula use

[jira] [Updated] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23348: -- Affects Version/s: 2.1.2 > append data using saveAsTable should adjust the data types >

[jira] [Resolved] (SPARK-23092) Migrate MemoryStream to DataSource V2

[ https://issues.apache.org/jira/browse/SPARK-23092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23092. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 20445

[jira] [Assigned] (SPARK-23092) Migrate MemoryStream to DataSource V2

[ https://issues.apache.org/jira/browse/SPARK-23092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-23092: - Assignee: Tathagata Das > Migrate MemoryStream to DataSource V2 >

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356168#comment-16356168 ] Thomas Graves commented on SPARK-22683: --- I agree, I think default behavior stays 1.  I ran a few

[jira] [Comment Edited] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356168#comment-16356168 ] Thomas Graves edited comment on SPARK-22683 at 2/7/18 10:24 PM: I agree,

[jira] [Commented] (SPARK-23355) convertMetastore should not ignore table properties

[ https://issues.apache.org/jira/browse/SPARK-23355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356076#comment-16356076 ] Apache Spark commented on SPARK-23355: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-23355) convertMetastore should not ignore table properties

[ https://issues.apache.org/jira/browse/SPARK-23355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23355: Assignee: Apache Spark > convertMetastore should not ignore table properties >

[jira] [Assigned] (SPARK-23355) convertMetastore should not ignore table properties

[ https://issues.apache.org/jira/browse/SPARK-23355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23355: Assignee: (was: Apache Spark) > convertMetastore should not ignore table properties >

[jira] [Commented] (SPARK-23329) Update the function descriptions with the arguments and returned values of the trigonometric functions

[ https://issues.apache.org/jira/browse/SPARK-23329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356119#comment-16356119 ] Sean Owen commented on SPARK-23329: --- Yeah, none of the other docs talk about a Column. It's implied

[jira] [Updated] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23348: -- Affects Version/s: 2.2.1 > append data using saveAsTable should adjust the data types >

[jira] [Updated] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23348: -- Description:   {code:java} Seq(1 -> "a").toDF("i", "j").write.saveAsTable("t") Seq("c" ->

[jira] [Commented] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356176#comment-16356176 ] Sameer Agarwal commented on SPARK-23348: yes, +1 > append data using saveAsTable should adjust

[jira] [Assigned] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23314: Assignee: Apache Spark > Pandas grouped udf on dataset with timestamp column error >

[jira] [Resolved] (SPARK-23349) Duplicate and redundant type determination for ShuffleManager Object

[ https://issues.apache.org/jira/browse/SPARK-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23349. --- Resolution: Won't Fix Fix Version/s: (was: 2.2.1) > Duplicate and redundant type

[jira] [Commented] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356174#comment-16356174 ] Dongjoon Hyun commented on SPARK-23348: --- [~cloud_fan], [~smilegator], [~sameerag]. Although this is

[jira] [Comment Edited] (SPARK-23139) Read eventLog file with mixed encodings

[ https://issues.apache.org/jira/browse/SPARK-23139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356507#comment-16356507 ] Jiang Xingbo edited comment on SPARK-23139 at 2/8/18 5:25 AM: --

[jira] [Commented] (SPARK-20090) Add StructType.fieldNames to Python API

[ https://issues.apache.org/jira/browse/SPARK-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356536#comment-16356536 ] Reynold Xin commented on SPARK-20090: - Do you mind doing it? Thanks. > Add StructType.fieldNames

[jira] [Commented] (SPARK-23139) Read eventLog file with mixed encodings

[ https://issues.apache.org/jira/browse/SPARK-23139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356507#comment-16356507 ] Jiang Xingbo commented on SPARK-23139: -- ``` EventLog may contain mixed encodings such as custom

[jira] [Commented] (SPARK-23139) Read eventLog file with mixed encodings

[ https://issues.apache.org/jira/browse/SPARK-23139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356428#comment-16356428 ] DENG FEI commented on SPARK-23139: -- _ASSII_ is enough to spark event log. And if forcing writing with

[jira] [Commented] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356431#comment-16356431 ] zhengruifeng commented on SPARK-22700: -- [~WeichenXu123] I have checked others, and them seems ok >

[jira] [Commented] (SPARK-22446) Optimizer causing StringIndexerModel's indexer UDF to throw "Unseen label" exception incorrectly for filtered data.

[ https://issues.apache.org/jira/browse/SPARK-22446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356506#comment-16356506 ] Liang-Chi Hsieh commented on SPARK-22446: - Yes, this is an issue in Spark 2.2. For earlier

[jira] [Commented] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356441#comment-16356441 ] Apache Spark commented on SPARK-22700: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-23139) Read eventLog file with mixed encodings

[ https://issues.apache.org/jira/browse/SPARK-23139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356440#comment-16356440 ] Marcelo Vanzin commented on SPARK-23139: bq. ASSII is enough to spark event log. No it's not.

[jira] [Commented] (SPARK-22446) Optimizer causing StringIndexerModel's indexer UDF to throw "Unseen label" exception incorrectly for filtered data.

[ https://issues.apache.org/jira/browse/SPARK-22446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356525#comment-16356525 ] Liang-Chi Hsieh commented on SPARK-22446: - 2.0 and 2.1 also have this issue. > Optimizer causing

[jira] [Updated] (SPARK-23319) Skip PySpark tests for old Pandas and old PyArrow

[ https://issues.apache.org/jira/browse/SPARK-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23319: - Fix Version/s: (was: 2.4.0) 2.3.0 > Skip PySpark tests for old Pandas and

[jira] [Commented] (SPARK-20090) Add StructType.fieldNames to Python API

[ https://issues.apache.org/jira/browse/SPARK-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356558#comment-16356558 ] Hyukjin Kwon commented on SPARK-20090: -- Sure, will open up a PR soon. > Add StructType.fieldNames

[jira] [Created] (SPARK-23356) Pushes Project to both sides of Union when expression is non-deterministic

caoxuewen created SPARK-23356: - Summary: Pushes Project to both sides of Union when expression is non-deterministic Key: SPARK-23356 URL: https://issues.apache.org/jira/browse/SPARK-23356 Project: Spark

[jira] [Created] (SPARK-23348) append data using saveAsTable should adjust the data types

Wenchen Fan created SPARK-23348: --- Summary: append data using saveAsTable should adjust the data types Key: SPARK-23348 URL: https://issues.apache.org/jira/browse/SPARK-23348 Project: Spark

[jira] [Commented] (SPARK-23349) Duplicate and redundant type determination for ShuffleManager Object

[ https://issues.apache.org/jira/browse/SPARK-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355137#comment-16355137 ] Apache Spark commented on SPARK-23349: -- User 'wujianping10043419' has created a pull request for

[jira] [Assigned] (SPARK-23349) Duplicate and redundant type determination for ShuffleManager Object

[ https://issues.apache.org/jira/browse/SPARK-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23349: Assignee: (was: Apache Spark) > Duplicate and redundant type determination for

[jira] [Assigned] (SPARK-23349) Duplicate and redundant type determination for ShuffleManager Object

[ https://issues.apache.org/jira/browse/SPARK-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23349: Assignee: Apache Spark > Duplicate and redundant type determination for ShuffleManager

[jira] [Commented] (SPARK-23329) Update the function descriptions with the arguments and returned values of the trigonometric functions

[ https://issues.apache.org/jira/browse/SPARK-23329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355155#comment-16355155 ] Mihaly Toth commented on SPARK-23329: - Nice. I like that the redundant description part is simply

[jira] [Assigned] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23348: Assignee: (was: Apache Spark) > append data using saveAsTable should adjust the data

[jira] [Commented] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355160#comment-16355160 ] Apache Spark commented on SPARK-23348: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23348: Assignee: Apache Spark > append data using saveAsTable should adjust the data types >

[jira] [Commented] (SPARK-19870) Repeatable deadlock on BlockInfoManager and TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-19870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355174#comment-16355174 ] Eyal Farago commented on SPARK-19870: - [~irashid], wenth through executors' logs and found no errors.

[jira] [Created] (SPARK-23349) Duplicate and redundant type determination for ShuffleManager Object

Phoenix_Daddy created SPARK-23349: - Summary: Duplicate and redundant type determination for ShuffleManager Object Key: SPARK-23349 URL: https://issues.apache.org/jira/browse/SPARK-23349 Project:

[jira] [Updated] (SPARK-23348) append data using saveAsTable should adjust the data types

[ https://issues.apache.org/jira/browse/SPARK-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-23348: Description:   {code:java} Seq(1 -> "a").toDF("i", "j").write.saveAsTable("t") Seq("c" ->

[jira] [Commented] (SPARK-14047) GBT improvement umbrella

[ https://issues.apache.org/jira/browse/SPARK-14047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355216#comment-16355216 ] Nick Pentreath commented on SPARK-14047: SPARK-12375 should fix that? Can you check it against

[jira] [Created] (SPARK-23350) [SS]Exception when stopping continuous processing application

Wang Yanlin created SPARK-23350: --- Summary: [SS]Exception when stopping continuous processing application Key: SPARK-23350 URL: https://issues.apache.org/jira/browse/SPARK-23350 Project: Spark

[jira] [Created] (SPARK-23351) checkpoint corruption in long running application

David Ahern created SPARK-23351: --- Summary: checkpoint corruption in long running application Key: SPARK-23351 URL: https://issues.apache.org/jira/browse/SPARK-23351 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23350) [SS]Exception when stopping continuous processing application

[ https://issues.apache.org/jira/browse/SPARK-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355311#comment-16355311 ] Apache Spark commented on SPARK-23350: -- User 'yanlin-Lynn' has created a pull request for this

[jira] [Updated] (SPARK-23351) checkpoint corruption in long running application

[ https://issues.apache.org/jira/browse/SPARK-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Ahern updated SPARK-23351: Description: hi, after leaving my (somewhat high volume) Structured Streaming application running

[jira] [Assigned] (SPARK-23350) [SS]Exception when stopping continuous processing application

[ https://issues.apache.org/jira/browse/SPARK-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23350: Assignee: Apache Spark > [SS]Exception when stopping continuous processing application >

[jira] [Assigned] (SPARK-23350) [SS]Exception when stopping continuous processing application

[ https://issues.apache.org/jira/browse/SPARK-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23350: Assignee: (was: Apache Spark) > [SS]Exception when stopping continuous processing

[jira] [Commented] (SPARK-23350) [SS]Exception when stopping continuous processing application

[ https://issues.apache.org/jira/browse/SPARK-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355326#comment-16355326 ] Apache Spark commented on SPARK-23350: -- User 'yanlin-Lynn' has created a pull request for this

[jira] [Commented] (SPARK-23349) Duplicate and redundant type determination for ShuffleManager Object

[ https://issues.apache.org/jira/browse/SPARK-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355340#comment-16355340 ] Apache Spark commented on SPARK-23349: -- User 'wujianping10043419' has created a pull request for

[jira] [Created] (SPARK-23352) Explicitly specify supported types in Pandas UDFs

Hyukjin Kwon created SPARK-23352: Summary: Explicitly specify supported types in Pandas UDFs Key: SPARK-23352 URL: https://issues.apache.org/jira/browse/SPARK-23352 Project: Spark Issue

[jira] [Assigned] (SPARK-23352) Explicitly specify supported types in Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23352: Assignee: (was: Apache Spark) > Explicitly specify supported types in Pandas UDFs >

[jira] [Commented] (SPARK-23352) Explicitly specify supported types in Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355442#comment-16355442 ] Apache Spark commented on SPARK-23352: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-23352) Explicitly specify supported types in Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23352: Assignee: Apache Spark > Explicitly specify supported types in Pandas UDFs >

[jira] [Commented] (SPARK-23353) Allow ExecutorMetricsUpdate events to be logged to the event log with sampling

[ https://issues.apache.org/jira/browse/SPARK-23353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355454#comment-16355454 ] Apache Spark commented on SPARK-23353: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23353) Allow ExecutorMetricsUpdate events to be logged to the event log with sampling

[ https://issues.apache.org/jira/browse/SPARK-23353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23353: Assignee: (was: Apache Spark) > Allow ExecutorMetricsUpdate events to be logged to

[jira] [Assigned] (SPARK-23353) Allow ExecutorMetricsUpdate events to be logged to the event log with sampling

[ https://issues.apache.org/jira/browse/SPARK-23353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23353: Assignee: Apache Spark > Allow ExecutorMetricsUpdate events to be logged to the event log

[jira] [Commented] (SPARK-23300) Print out if Pandas and PyArrow are installed or not in tests

[ https://issues.apache.org/jira/browse/SPARK-23300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355483#comment-16355483 ] Apache Spark commented on SPARK-23300: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-23319) Skip PySpark tests for old Pandas and old PyArrow

[ https://issues.apache.org/jira/browse/SPARK-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355531#comment-16355531 ] Apache Spark commented on SPARK-23319: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-23319) Skip PySpark tests for old Pandas and old PyArrow

[ https://issues.apache.org/jira/browse/SPARK-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23319: - Fix Version/s: 2.4.0 > Skip PySpark tests for old Pandas and old PyArrow >

[jira] [Resolved] (SPARK-23319) Skip PySpark tests for old Pandas and old PyArrow

[ https://issues.apache.org/jira/browse/SPARK-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23319. -- Resolution: Fixed Assignee: Hyukjin Kwon Target Version/s: 2.4.0 Fixed

[jira] [Updated] (SPARK-23319) Skip PySpark tests for old Pandas and old PyArrow

[ https://issues.apache.org/jira/browse/SPARK-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23319: - Target Version/s: (was: 2.4.0) > Skip PySpark tests for old Pandas and old PyArrow >

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1637#comment-1637 ] Thomas Graves commented on SPARK-22683: --- ok thanks,  I would like to try this out myself on a few

[jira] [Assigned] (SPARK-23341) DataSourceOptions should handle path and table names to avoid confusion.

[ https://issues.apache.org/jira/browse/SPARK-23341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23341: Assignee: Apache Spark > DataSourceOptions should handle path and table names to avoid

[jira] [Commented] (SPARK-23341) DataSourceOptions should handle path and table names to avoid confusion.

[ https://issues.apache.org/jira/browse/SPARK-23341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355563#comment-16355563 ] Apache Spark commented on SPARK-23341: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23341) DataSourceOptions should handle path and table names to avoid confusion.

[ https://issues.apache.org/jira/browse/SPARK-23341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23341: Assignee: (was: Apache Spark) > DataSourceOptions should handle path and table names

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355634#comment-16355634 ] Xuefu Zhang commented on SPARK-22683: - +1 on the idea of including this. Also, +1 on renaming the

[jira] [Updated] (SPARK-23351) checkpoint corruption in long running application

[ https://issues.apache.org/jira/browse/SPARK-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Ahern updated SPARK-23351: Description: hi, after leaving my (somewhat high volume) Structured Streaming application running

[jira] [Updated] (SPARK-23352) Explicitly specify supported types in Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23352: - Description: Currently, we don't support {{BinaryType}} in Pandas UDFs: {code} >>> from

[jira] [Created] (SPARK-23353) Allow ExecutorMetricsUpdate events to be logged to the event log with sampling

Lantao Jin created SPARK-23353: -- Summary: Allow ExecutorMetricsUpdate events to be logged to the event log with sampling Key: SPARK-23353 URL: https://issues.apache.org/jira/browse/SPARK-23353 Project: