[jira] [Updated] (SPARK-24216) Spark TypedAggregateExpression uses getSimpleName that is not safe in scala

[ https://issues.apache.org/jira/browse/SPARK-24216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangshi Li updated SPARK-24216: --- Description: When user create a aggregator object in scala and pass the aggregator to Spark

[jira] [Commented] (SPARK-18620) Spark Streaming + Kinesis : Receiver MaxRate is violated

[ https://issues.apache.org/jira/browse/SPARK-18620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470069#comment-16470069 ] bruce_zhao commented on SPARK-18620: This PR makes the input rate flat, but the limit of get-records

[jira] [Updated] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-24241: - Summary: Do not fail fast when dynamic resource allocation enabled with 0 executor (was: Do not fail

[jira] [Assigned] (SPARK-23907) Support regr_* functions

[ https://issues.apache.org/jira/browse/SPARK-23907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-23907: - Assignee: Marco Gaido > Support regr_* functions > > >

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470067#comment-16470067 ] Li Yuanjian commented on SPARK-24036: - I agree with the division about the kinds of tasks, that's

[jira] [Updated] (SPARK-24235) create the top-of-task RDD sending rows to the remote buffer

[ https://issues.apache.org/jira/browse/SPARK-24235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-24235: Description:

[jira] [Updated] (SPARK-24235) create the top-of-task RDD sending rows to the remote buffer

[ https://issues.apache.org/jira/browse/SPARK-24235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-24235: Description:

[jira] [Updated] (SPARK-24235) create the top-of-task RDD sending rows to the remote buffer

[ https://issues.apache.org/jira/browse/SPARK-24235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-24235: Description:

[jira] [Assigned] (SPARK-24242) RangeExec should have correct outputOrdering

[ https://issues.apache.org/jira/browse/SPARK-24242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24242: Assignee: Apache Spark > RangeExec should have correct outputOrdering >

[jira] [Assigned] (SPARK-24242) RangeExec should have correct outputOrdering

[ https://issues.apache.org/jira/browse/SPARK-24242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24242: Assignee: (was: Apache Spark) > RangeExec should have correct outputOrdering >

[jira] [Resolved] (SPARK-23907) Support regr_* functions

[ https://issues.apache.org/jira/browse/SPARK-23907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23907. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21054

[jira] [Created] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled

Kent Yao created SPARK-24241: Summary: Do not fail fast when dynamic resource allocation enabled Key: SPARK-24241 URL: https://issues.apache.org/jira/browse/SPARK-24241 Project: Spark Issue

[jira] [Assigned] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24241: Assignee: (was: Apache Spark) > Do not fail fast when dynamic resource allocation

[jira] [Assigned] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24241: Assignee: Apache Spark > Do not fail fast when dynamic resource allocation enabled with 0

[jira] [Commented] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470115#comment-16470115 ] Apache Spark commented on SPARK-24241: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Created] (SPARK-24242) RangeExec should have correct outputOrdering

Liang-Chi Hsieh created SPARK-24242: --- Summary: RangeExec should have correct outputOrdering Key: SPARK-24242 URL: https://issues.apache.org/jira/browse/SPARK-24242 Project: Spark Issue

[jira] [Commented] (SPARK-24242) RangeExec should have correct outputOrdering

[ https://issues.apache.org/jira/browse/SPARK-24242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470201#comment-16470201 ] Apache Spark commented on SPARK-24242: -- User 'viirya' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469858#comment-16469858 ] spark_user edited comment on SPARK-24217 at 5/10/18 12:22 PM: --      Hi

[jira] [Comment Edited] (SPARK-24232) Allow referring to kubernetes secrets as env variable

[ https://issues.apache.org/jira/browse/SPARK-24232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470333#comment-16470333 ] Stavros Kontopoulos edited comment on SPARK-24232 at 5/10/18 12:43 PM:

[jira] [Created] (SPARK-24243) Expose exceptions from InProcessAppHandle

Sahil Takiar created SPARK-24243: Summary: Expose exceptions from InProcessAppHandle Key: SPARK-24243 URL: https://issues.apache.org/jira/browse/SPARK-24243 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-11150) Dynamic partition pruning

[ https://issues.apache.org/jira/browse/SPARK-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469551#comment-16469551 ] tim geary edited comment on SPARK-11150 at 5/10/18 12:42 PM: - I have a

[jira] [Commented] (SPARK-24243) Expose exceptions from InProcessAppHandle

[ https://issues.apache.org/jira/browse/SPARK-24243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470339#comment-16470339 ] Sahil Takiar commented on SPARK-24243: -- [~vanzin] would adding something like this be possible?

[jira] [Commented] (SPARK-24232) Allow referring to kubernetes secrets as env variable

[ https://issues.apache.org/jira/browse/SPARK-24232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470333#comment-16470333 ] Stavros Kontopoulos commented on SPARK-24232: - Check also what we use on mesos for naming

[jira] [Commented] (SPARK-24068) CSV schema inferring doesn't work for compressed files

[ https://issues.apache.org/jira/browse/SPARK-24068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470361#comment-16470361 ] Apache Spark commented on SPARK-24068: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469984#comment-16469984 ] Takeshi Yamamuro commented on SPARK-23519: -- I think typical databases can't use duplicate column

[jira] [Created] (SPARK-24240) Add a config to control whether InMemoryFileIndex should update cache when refresh.

jin xing created SPARK-24240: Summary: Add a config to control whether InMemoryFileIndex should update cache when refresh. Key: SPARK-24240 URL: https://issues.apache.org/jira/browse/SPARK-24240 Project:

[jira] [Commented] (SPARK-24240) Add a config to control whether InMemoryFileIndex should update cache when refresh.

[ https://issues.apache.org/jira/browse/SPARK-24240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470039#comment-16470039 ] Apache Spark commented on SPARK-24240: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24240) Add a config to control whether InMemoryFileIndex should update cache when refresh.

[ https://issues.apache.org/jira/browse/SPARK-24240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24240: Assignee: (was: Apache Spark) > Add a config to control whether InMemoryFileIndex

[jira] [Assigned] (SPARK-24240) Add a config to control whether InMemoryFileIndex should update cache when refresh.

[ https://issues.apache.org/jira/browse/SPARK-24240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24240: Assignee: Apache Spark > Add a config to control whether InMemoryFileIndex should update

[jira] [Commented] (SPARK-6235) Address various 2G limits

[ https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469991#comment-16469991 ] Cyanny commented on SPARK-6235: --- Hi, when will this Jira feature included in a spark releas? [~rxin] >

[jira] [Commented] (SPARK-24233) union operation on read of dataframe does nor produce correct result

[ https://issues.apache.org/jira/browse/SPARK-24233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469975#comment-16469975 ] Takeshi Yamamuro commented on SPARK-24233: -- Can you simplify your query? Also, can you put the

[jira] [Assigned] (SPARK-24197) add array_sort function

[ https://issues.apache.org/jira/browse/SPARK-24197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24197: Assignee: Apache Spark > add array_sort function > --- > >

[jira] [Assigned] (SPARK-24197) add array_sort function

[ https://issues.apache.org/jira/browse/SPARK-24197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24197: Assignee: (was: Apache Spark) > add array_sort function > --- > >

[jira] [Commented] (SPARK-24197) add array_sort function

[ https://issues.apache.org/jira/browse/SPARK-24197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470616#comment-16470616 ] Apache Spark commented on SPARK-24197: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Commented] (SPARK-24237) continuous shuffle dependency

[ https://issues.apache.org/jira/browse/SPARK-24237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470546#comment-16470546 ] Apache Spark commented on SPARK-24237: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-24237) continuous shuffle dependency

[ https://issues.apache.org/jira/browse/SPARK-24237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24237: Assignee: Apache Spark > continuous shuffle dependency > - >

[jira] [Assigned] (SPARK-24237) continuous shuffle dependency

[ https://issues.apache.org/jira/browse/SPARK-24237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24237: Assignee: (was: Apache Spark) > continuous shuffle dependency >

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470552#comment-16470552 ] Jose Torres commented on SPARK-24036: - My concern isn't that we'll have to write more code, but that

[jira] [Commented] (SPARK-24227) Not able to submit spark job to kubernetes on 2.3

[ https://issues.apache.org/jira/browse/SPARK-24227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470493#comment-16470493 ] Felipe Cavalcanti commented on SPARK-24227: --- solved it, spark was using the basic auth from the

[jira] [Commented] (SPARK-24204) Verify a write schema in OrcFileFormat

[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470634#comment-16470634 ] Dongjoon Hyun commented on SPARK-24204: --- Thank you for pinging me, [~maropu]. Could you make a PR

[jira] [Updated] (SPARK-24204) Verify a write schema in Json/Orc/ParquetFileFormat

[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24204: -- Summary: Verify a write schema in Json/Orc/ParquetFileFormat (was: Verify a write schema in

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469562#comment-16469562 ] Joseph K. Bradley edited comment on SPARK-24217 at 5/10/18 4:37 PM:

[jira] [Commented] (SPARK-15784) Add Power Iteration Clustering to spark.ml

[ https://issues.apache.org/jira/browse/SPARK-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470701#comment-16470701 ] Joseph K. Bradley commented on SPARK-15784: --- So... we originally agreed to make this a

[jira] [Commented] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470735#comment-16470735 ] Dongjoon Hyun commented on SPARK-23458: --- Oh, I missed your ping here, [~smilegator]. According to

[jira] [Comment Edited] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470735#comment-16470735 ] Dongjoon Hyun edited comment on SPARK-23458 at 5/10/18 5:10 PM: Oh, I

[jira] [Updated] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franck Tago updated SPARK-23519: Attachment: image-2018-05-10-10-48-57-259.png > Create View Commands Fails with The view output

[jira] [Commented] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470843#comment-16470843 ] Franck Tago commented on SPARK-23519: - I do not agree with the 'typical database' claim .  mysql ,

[jira] [Updated] (SPARK-24225) Support closing AutoClosable objects in MemoryStore so Broadcast Variables can be released properly

[ https://issues.apache.org/jira/browse/SPARK-24225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-24225: Shepherd: DB Tsai Priority: Major (was: Minor) > Support closing AutoClosable objects in MemoryStore

[jira] [Updated] (SPARK-24225) Support closing AutoClosable objects in MemoryStore so Broadcast Variables can be released properly

[ https://issues.apache.org/jira/browse/SPARK-24225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-24225: Issue Type: New Feature (was: Improvement) > Support closing AutoClosable objects in MemoryStore so

[jira] [Updated] (SPARK-24204) Verify a write schema in Orc/ParquetFileFormat

[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24204: -- Summary: Verify a write schema in Orc/ParquetFileFormat (was: Verify a write schema in

[jira] [Updated] (SPARK-24068) CSV schema inferring doesn't work for compressed files

[ https://issues.apache.org/jira/browse/SPARK-24068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24068: - Fix Version/s: 2.3.1 > CSV schema inferring doesn't work for compressed files >

[jira] [Comment Edited] (SPARK-15784) Add Power Iteration Clustering to spark.ml

[ https://issues.apache.org/jira/browse/SPARK-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470701#comment-16470701 ] Joseph K. Bradley edited comment on SPARK-15784 at 5/10/18 4:45 PM:

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470777#comment-16470777 ] Felix Cheung commented on SPARK-23206: -- yes, for use network and disk IO stats. We have been

[jira] [Comment Edited] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470777#comment-16470777 ] Felix Cheung edited comment on SPARK-23206 at 5/10/18 5:20 PM: --- yes, for us

[jira] [Commented] (SPARK-24243) Expose exceptions from InProcessAppHandle

[ https://issues.apache.org/jira/browse/SPARK-24243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470797#comment-16470797 ] Marcelo Vanzin commented on SPARK-24243: Sure. As long as the child process handle returns

[jira] [Updated] (SPARK-24204) Verify a write schema in Json/Orc/ParquetFileFormat

[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24204: -- Description: *SUMMARY* - CSV: Raising analysis exception. - JSON: dropping columns with null

[jira] [Comment Edited] (SPARK-24204) Verify a write schema in Json/Orc/ParquetFileFormat

[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470634#comment-16470634 ] Dongjoon Hyun edited comment on SPARK-24204 at 5/10/18 4:29 PM: Thank you

[jira] [Commented] (SPARK-24213) Power Iteration Clustering in the SparkML throws exception, when the ID is IntType

[ https://issues.apache.org/jira/browse/SPARK-24213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470705#comment-16470705 ] Joseph K. Bradley commented on SPARK-24213: --- On the topic of eating my words, please check out

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470789#comment-16470789 ] Edwina Lu commented on SPARK-23206: --- [~irashid], I do not have the rest of the changes for 2.3/master

[jira] [Assigned] (SPARK-24225) Support closing AutoClosable objects in MemoryStore so Broadcast Variables can be released properly

[ https://issues.apache.org/jira/browse/SPARK-24225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-24225: --- Assignee: Doug Rohrer > Support closing AutoClosable objects in MemoryStore so Broadcast Variables

[jira] [Updated] (SPARK-24233) union operation on read of dataframe does nor produce correct result

[ https://issues.apache.org/jira/browse/SPARK-24233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] smohr003 updated SPARK-24233: - Description: I know that I can use wild card * to read all subfolders. But, I am trying to use .par and

[jira] [Comment Edited] (SPARK-24204) Verify a write schema in Json/Orc/ParquetFileFormat

[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470634#comment-16470634 ] Dongjoon Hyun edited comment on SPARK-24204 at 5/10/18 4:28 PM: Thank you

[jira] [Commented] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470704#comment-16470704 ] Joseph K. Bradley commented on SPARK-24217: --- On the topic of eating my words, please check out

[jira] [Updated] (SPARK-24228) Fix the lint error

[ https://issues.apache.org/jira/browse/SPARK-24228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24228: -- Priority: Minor (was: Major) > Fix the lint error > -- > >

[jira] [Created] (SPARK-24244) Parse only required columns of CSV file

Maxim Gekk created SPARK-24244: -- Summary: Parse only required columns of CSV file Key: SPARK-24244 URL: https://issues.apache.org/jira/browse/SPARK-24244 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-24230) With Parquet 1.10 upgrade has errors in the vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-24230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470948#comment-16470948 ] Apache Spark commented on SPARK-24230: -- User 'rdblue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24230) With Parquet 1.10 upgrade has errors in the vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-24230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24230: Assignee: Apache Spark > With Parquet 1.10 upgrade has errors in the vectorized reader >

[jira] [Commented] (SPARK-24244) Parse only required columns of CSV file

[ https://issues.apache.org/jira/browse/SPARK-24244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470947#comment-16470947 ] Apache Spark commented on SPARK-24244: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24230) With Parquet 1.10 upgrade has errors in the vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-24230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24230: Assignee: (was: Apache Spark) > With Parquet 1.10 upgrade has errors in the

[jira] [Assigned] (SPARK-24244) Parse only required columns of CSV file

[ https://issues.apache.org/jira/browse/SPARK-24244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24244: Assignee: Apache Spark > Parse only required columns of CSV file >

[jira] [Assigned] (SPARK-24244) Parse only required columns of CSV file

[ https://issues.apache.org/jira/browse/SPARK-24244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24244: Assignee: (was: Apache Spark) > Parse only required columns of CSV file >

[jira] [Updated] (SPARK-24137) [K8s] Mount temporary directories in emptydir volumes

[ https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li updated SPARK-24137: - Fix Version/s: (was: 2.3.1) > [K8s] Mount temporary directories in emptydir volumes >

[jira] [Assigned] (SPARK-24137) [K8s] Mount temporary directories in emptydir volumes

[ https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan reassigned SPARK-24137: -- Assignee: Matt Cheah > [K8s] Mount temporary directories in emptydir volumes

[jira] [Resolved] (SPARK-24137) [K8s] Mount temporary directories in emptydir volumes

[ https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan resolved SPARK-24137. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21238

[jira] [Updated] (SPARK-24137) [K8s] Mount temporary directories in emptydir volumes

[ https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li updated SPARK-24137: - Fix Version/s: 2.3.1 > [K8s] Mount temporary directories in emptydir volumes >

[jira] [Resolved] (SPARK-10878) Race condition when resolving Maven coordinates via Ivy

[ https://issues.apache.org/jira/browse/SPARK-10878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10878. Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.4.0

[jira] [Created] (SPARK-24247) [K8S] currentNodeToLocalTaskCount is unused in KubernetesClusterSchedulerBackend

Matt Cheah created SPARK-24247: -- Summary: [K8S] currentNodeToLocalTaskCount is unused in KubernetesClusterSchedulerBackend Key: SPARK-24247 URL: https://issues.apache.org/jira/browse/SPARK-24247

[jira] [Created] (SPARK-24248) [K8S] Use the Kubernetes cluster as the backing store for the state of pods

Matt Cheah created SPARK-24248: -- Summary: [K8S] Use the Kubernetes cluster as the backing store for the state of pods Key: SPARK-24248 URL: https://issues.apache.org/jira/browse/SPARK-24248 Project:

[jira] [Commented] (SPARK-24248) [K8S] Use the Kubernetes cluster as the backing store for the state of pods

[ https://issues.apache.org/jira/browse/SPARK-24248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471206#comment-16471206 ] Matt Cheah commented on SPARK-24248: [~foxish] [~liyinan926] curious as to what you think about this

[jira] [Assigned] (SPARK-24198) add slice function

[ https://issues.apache.org/jira/browse/SPARK-24198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24198: Assignee: (was: Apache Spark) > add slice function > -- > >

[jira] [Commented] (SPARK-24198) add slice function

[ https://issues.apache.org/jira/browse/SPARK-24198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471215#comment-16471215 ] Apache Spark commented on SPARK-24198: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24198) add slice function

[ https://issues.apache.org/jira/browse/SPARK-24198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24198: Assignee: Apache Spark > add slice function > -- > > Key:

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471040#comment-16471040 ] shahid edited comment on SPARK-24217 at 5/10/18 9:11 PM: - Thanks for the

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471040#comment-16471040 ] shahid edited comment on SPARK-24217 at 5/10/18 9:15 PM: - Thanks for the

[jira] [Created] (SPARK-24246) Improve AnalysisException by setting the cause when it's available

Shixiong Zhu created SPARK-24246: Summary: Improve AnalysisException by setting the cause when it's available Key: SPARK-24246 URL: https://issues.apache.org/jira/browse/SPARK-24246 Project: Spark

[jira] [Commented] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471128#comment-16471128 ] shahid commented on SPARK-24217: Thanks for the clarification Joseph K. Bradley Is it really required

[jira] [Assigned] (SPARK-24246) Improve AnalysisException by setting the cause when it's available

[ https://issues.apache.org/jira/browse/SPARK-24246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24246: Assignee: Shixiong Zhu (was: Apache Spark) > Improve AnalysisException by setting the

[jira] [Assigned] (SPARK-24246) Improve AnalysisException by setting the cause when it's available

[ https://issues.apache.org/jira/browse/SPARK-24246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24246: Assignee: Apache Spark (was: Shixiong Zhu) > Improve AnalysisException by setting the

[jira] [Issue Comment Deleted] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-24217: --- Comment: was deleted (was: Thanks for the clarification.  Is it really required to append the result with

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471128#comment-16471128 ] shahid edited comment on SPARK-24217 at 5/10/18 9:19 PM: - Thanks for the

[jira] [Commented] (SPARK-24246) Improve AnalysisException by setting the cause when it's available

[ https://issues.apache.org/jira/browse/SPARK-24246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471131#comment-16471131 ] Apache Spark commented on SPARK-24246: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471128#comment-16471128 ] shahid edited comment on SPARK-24217 at 5/10/18 9:23 PM: - Thanks for the

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471128#comment-16471128 ] shahid edited comment on SPARK-24217 at 5/10/18 9:26 PM: - Thanks for the

[jira] [Assigned] (SPARK-19181) SparkListenerSuite.local metrics fails when average executorDeserializeTime is too short.

[ https://issues.apache.org/jira/browse/SPARK-19181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-19181: -- Assignee: Attila Zsolt Piros > SparkListenerSuite.local metrics fails when average

[jira] [Resolved] (SPARK-19181) SparkListenerSuite.local metrics fails when average executorDeserializeTime is too short.

[ https://issues.apache.org/jira/browse/SPARK-19181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19181. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471128#comment-16471128 ] shahid edited comment on SPARK-24217 at 5/10/18 9:28 PM: - Thanks for the

[jira] [Issue Comment Deleted] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-24217: --- Comment: was deleted (was: Thanks for the clarification Joseph K. Bradley Is it really required to append

[jira] [Commented] (SPARK-24204) Verify a write schema in Json/Orc/ParquetFileFormat

[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471283#comment-16471283 ] Takeshi Yamamuro commented on SPARK-24204: -- ok, I'll do it later. Thanks for the description

[jira] [Commented] (SPARK-24248) [K8S] Use the Kubernetes cluster as the backing store for the state of pods

[ https://issues.apache.org/jira/browse/SPARK-24248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471288#comment-16471288 ] Yinan Li commented on SPARK-24248: -- Just realized one thing: solely replying on the watcher poses risks

[jira] [Updated] (SPARK-24233) union operation on read of dataframe does nor produce correct result

[ https://issues.apache.org/jira/browse/SPARK-24233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] smohr003 updated SPARK-24233: - Description: I know that I can use wild card * to read all subfolders. But, I am trying to use .par and

  1   2   >