[jira] [Commented] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334061#comment-16334061 ] Igor Babalich commented on SPARK-12216: --- Same issue under Windows10, Java 1.8  Spark 2.1.1 Hadoop

[jira] [Commented] (SPARK-19217) Offer easy cast from vector to array

[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333987#comment-16333987 ] Takeshi Yamamuro commented on SPARK-19217: -- ok, I'll reconsider this. > Offer easy cast from

[jira] [Created] (SPARK-23177) PySpark parameter-less UDFs raise exception if applied after distinct

Jakub created SPARK-23177: - Summary: PySpark parameter-less UDFs raise exception if applied after distinct Key: SPARK-23177 URL: https://issues.apache.org/jira/browse/SPARK-23177 Project: Spark

[jira] [Comment Edited] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

[ https://issues.apache.org/jira/browse/SPARK-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334206#comment-16334206 ] KIryl Sultanau edited comment on SPARK-23178 at 1/22/18 12:38 PM: -- With

[jira] [Updated] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

[ https://issues.apache.org/jira/browse/SPARK-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KIryl Sultanau updated SPARK-23178: --- Attachment: Unsafe-off.png > Kryo Unsafe problems with count distinct from cache >

[jira] [Comment Edited] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

[ https://issues.apache.org/jira/browse/SPARK-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334206#comment-16334206 ] KIryl Sultanau edited comment on SPARK-23178 at 1/22/18 12:48 PM: -- With

[jira] [Updated] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

[ https://issues.apache.org/jira/browse/SPARK-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KIryl Sultanau updated SPARK-23178: --- Attachment: Unsafe-issue.png > Kryo Unsafe problems with count distinct from cache >

[jira] [Commented] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

[ https://issues.apache.org/jira/browse/SPARK-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334206#comment-16334206 ] KIryl Sultanau commented on SPARK-23178: With unsafe switch off this works fine:  

[jira] [Updated] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

[ https://issues.apache.org/jira/browse/SPARK-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KIryl Sultanau updated SPARK-23178: --- Priority: Minor (was: Major) > Kryo Unsafe problems with count distinct from cache >

[jira] [Comment Edited] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

[ https://issues.apache.org/jira/browse/SPARK-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334206#comment-16334206 ] KIryl Sultanau edited comment on SPARK-23178 at 1/22/18 12:35 PM: -- With

[jira] [Resolved] (SPARK-23170) Dump the statistics of effective runs of analyzer and optimizer rules

[ https://issues.apache.org/jira/browse/SPARK-23170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23170. - Resolution: Fixed Fix Version/s: 2.3.0 > Dump the statistics of effective runs of analyzer and

[jira] [Created] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

KIryl Sultanau created SPARK-23178: -- Summary: Kryo Unsafe problems with count distinct from cache Key: SPARK-23178 URL: https://issues.apache.org/jira/browse/SPARK-23178 Project: Spark

[jira] [Updated] (SPARK-23178) Kryo Unsafe problems with count distinct from cache

[ https://issues.apache.org/jira/browse/SPARK-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KIryl Sultanau updated SPARK-23178: --- Description: Spark incorrectly process cached data with Kryo & Unsafe options. Distinct

[jira] [Commented] (SPARK-23154) Document backwards compatibility guarantees for ML persistence

[ https://issues.apache.org/jira/browse/SPARK-23154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334757#comment-16334757 ] Yanbo Liang commented on SPARK-23154: - Sounds good! It should be helpful to document backwards

[jira] [Assigned] (SPARK-23121) When the Spark Streaming app is running for a period of time, the page is incorrectly reported when accessing '/ jobs /' or '/ jobs / job /? Id = 13' and ui can not be

[ https://issues.apache.org/jira/browse/SPARK-23121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23121: -- Assignee: Sandor Murakozi > When the Spark Streaming app is running for a period of

[jira] [Resolved] (SPARK-23121) When the Spark Streaming app is running for a period of time, the page is incorrectly reported when accessing '/ jobs /' or '/ jobs / job /? Id = 13' and ui can not be

[ https://issues.apache.org/jira/browse/SPARK-23121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23121. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20330

[jira] [Assigned] (SPARK-23014) Migrate MemorySink fully to v2

[ https://issues.apache.org/jira/browse/SPARK-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23014: Assignee: Apache Spark > Migrate MemorySink fully to v2 > --

[jira] [Commented] (SPARK-23014) Migrate MemorySink fully to v2

[ https://issues.apache.org/jira/browse/SPARK-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334747#comment-16334747 ] Apache Spark commented on SPARK-23014: -- User 'jose-torres' has created a pull request for this

[jira] [Assigned] (SPARK-23014) Migrate MemorySink fully to v2

[ https://issues.apache.org/jira/browse/SPARK-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23014: Assignee: (was: Apache Spark) > Migrate MemorySink fully to v2 >

[jira] [Commented] (SPARK-23186) Initialize DriverManager first before loading Drivers

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335382#comment-16335382 ] Apache Spark commented on SPARK-23186: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-23186) Initialize DriverManager first before loading Drivers

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23186: -- Description: Since some JDBC Drivers have class initialization code to call `DriverManager`,

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335428#comment-16335428 ] Saisai Shao commented on SPARK-23187: - I will try to investigate on it. > Accumulator object can not

[jira] [Commented] (SPARK-23186) Initialize DriverManager first before loading Drivers

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335451#comment-16335451 ] Apache Spark commented on SPARK-23186: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-23186) Initialize DriverManager first before loading Drivers

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23186: -- Summary: Initialize DriverManager first before loading Drivers (was: Loading JDBC Drivers

[jira] [Updated] (SPARK-23186) Initialize DriverManager first before loading Drivers

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23186: -- Description: Since some JDBC Drivers have class initialization code to call `DriverManager`,

[jira] [Resolved] (SPARK-22274) User-defined aggregation functions with pandas udf

[ https://issues.apache.org/jira/browse/SPARK-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-22274. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 19872

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335424#comment-16335424 ] Lantao Jin commented on SPARK-23187: Hi [~jerryshao], do you have time to look at it? > Accumulator

[jira] [Assigned] (SPARK-23177) PySpark parameter-less UDFs raise exception if applied after distinct

[ https://issues.apache.org/jira/browse/SPARK-23177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23177: Assignee: (was: Apache Spark) > PySpark parameter-less UDFs raise exception if

[jira] [Assigned] (SPARK-23177) PySpark parameter-less UDFs raise exception if applied after distinct

[ https://issues.apache.org/jira/browse/SPARK-23177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23177: Assignee: Apache Spark > PySpark parameter-less UDFs raise exception if applied after

[jira] [Commented] (SPARK-23177) PySpark parameter-less UDFs raise exception if applied after distinct

[ https://issues.apache.org/jira/browse/SPARK-23177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335433#comment-16335433 ] Apache Spark commented on SPARK-23177: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

Lantao Jin created SPARK-23187: -- Summary: Accumulator object can not be sent from Executor to Driver Key: SPARK-23187 URL: https://issues.apache.org/jira/browse/SPARK-23187 Project: Spark Issue

[jira] [Updated] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-23187: --- Affects Version/s: 2.3.1 > Accumulator object can not be sent from Executor to Driver >

[jira] [Updated] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23187: Affects Version/s: (was: 2.3.1) 2.3.0 > Accumulator object can not be

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335431#comment-16335431 ] Lantao Jin commented on SPARK-23187: The simplest way to verify is just set a fixed value in

[jira] [Assigned] (SPARK-22274) User-defined aggregation functions with pandas udf

[ https://issues.apache.org/jira/browse/SPARK-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-22274: - Assignee: Li Jin > User-defined aggregation functions with pandas udf >

[jira] [Updated] (SPARK-23186) Initialize DriverManager first before loading Drivers

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23186: -- Description: Since some JDBC Drivers have class initialization code to call `DriverManager`,

[jira] [Updated] (SPARK-23186) Initialize DriverManager first before loading Drivers

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23186: -- Description: Since some JDBC Drivers have class initialization code to call `DriverManager`,

[jira] [Commented] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335463#comment-16335463 ] seemab commented on SPARK-22711: kindly share..how did you updated thet code > _pickle.PicklingError:

[jira] [Commented] (SPARK-23173) from_json can produce nulls for fields which are marked as non-nullable

[ https://issues.apache.org/jira/browse/SPARK-23173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334226#comment-16334226 ] Burak Yavuz commented on SPARK-23173: - In terms of usability, I prefer 1. In terms of the viewpoint

[jira] [Commented] (SPARK-23179) Support option to throw exception if overflow occurs

[ https://issues.apache.org/jira/browse/SPARK-23179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334321#comment-16334321 ] Apache Spark commented on SPARK-23179: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23179) Support option to throw exception if overflow occurs

[ https://issues.apache.org/jira/browse/SPARK-23179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23179: Assignee: Apache Spark > Support option to throw exception if overflow occurs >

[jira] [Assigned] (SPARK-23179) Support option to throw exception if overflow occurs

[ https://issues.apache.org/jira/browse/SPARK-23179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23179: Assignee: (was: Apache Spark) > Support option to throw exception if overflow occurs

[jira] [Commented] (SPARK-13964) Feature hashing improvements

[ https://issues.apache.org/jira/browse/SPARK-13964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334350#comment-16334350 ] Artem Kalchenko commented on SPARK-13964: - How about adding hashing for quadratic features like

[jira] [Commented] (SPARK-23173) from_json can produce nulls for fields which are marked as non-nullable

[ https://issues.apache.org/jira/browse/SPARK-23173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334388#comment-16334388 ] Michał Świtakowski commented on SPARK-23173: I think starting with option 1 is a good idea

[jira] [Updated] (SPARK-23183) Failure caused by TaskContext is missing in the thread spawned by Custom RDD

[ https://issues.apache.org/jira/browse/SPARK-23183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23183: -- Shepherd: (was: Jiang Xingbo) Flags: (was: Important) Target Version/s:

[jira] [Commented] (SPARK-23183) Failure caused by TaskContext is missing in the thread spawned by Custom RDD

[ https://issues.apache.org/jira/browse/SPARK-23183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335168#comment-16335168 ] Sean Owen commented on SPARK-23183: --- OK; fix the title? but same point stands I think. Spawning your

[jira] [Updated] (SPARK-23183) Failure caused by TaskContext is missing in the thread spawned by user code

[ https://issues.apache.org/jira/browse/SPARK-23183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongqin Xiao updated SPARK-23183: - Summary: Failure caused by TaskContext is missing in the thread spawned by user code (was:

[jira] [Created] (SPARK-23181) Add compatibility tests for SHS serialized data / disk format

Marcelo Vanzin created SPARK-23181: -- Summary: Add compatibility tests for SHS serialized data / disk format Key: SPARK-23181 URL: https://issues.apache.org/jira/browse/SPARK-23181 Project: Spark

[jira] [Assigned] (SPARK-11315) Add YARN extension service to publish Spark events to YARN timeline service

[ https://issues.apache.org/jira/browse/SPARK-11315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-11315: -- Assignee: Marcelo Vanzin > Add YARN extension service to publish Spark events to YARN

[jira] [Updated] (SPARK-23052) Migrate Microbatch ConsoleSink to v2

[ https://issues.apache.org/jira/browse/SPARK-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-23052: -- Summary: Migrate Microbatch ConsoleSink to v2 (was: Migrate MicrConsoleSink to v2) > Migrate

[jira] [Commented] (SPARK-23103) LevelDB store not iterating correctly when indexed value has negative value

[ https://issues.apache.org/jira/browse/SPARK-23103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334972#comment-16334972 ] Apache Spark commented on SPARK-23103: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335155#comment-16335155 ] Shixiong Zhu commented on SPARK-23184: -- Not yet. But I see it's duplicated after reading the patch.

[jira] [Resolved] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23184. -- Resolution: Duplicate Fix Version/s: 2.3.0 > All jobs page is broken when some stage is

[jira] [Created] (SPARK-23183) Failure caused by TaskContext is missing in the thread spawned by Custom RDD

Yongqin Xiao created SPARK-23183: Summary: Failure caused by TaskContext is missing in the thread spawned by Custom RDD Key: SPARK-23183 URL: https://issues.apache.org/jira/browse/SPARK-23183

[jira] [Commented] (SPARK-23183) Failure caused by TaskContext is missing in the thread spawned by Custom RDD

[ https://issues.apache.org/jira/browse/SPARK-23183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335059#comment-16335059 ] Yongqin Xiao commented on SPARK-23183: -- Exact same issue reported by other user: 

[jira] [Created] (SPARK-23184) All jobs page is broken when some stage is missing

Shixiong Zhu created SPARK-23184: Summary: All jobs page is broken when some stage is missing Key: SPARK-23184 URL: https://issues.apache.org/jira/browse/SPARK-23184 Project: Spark Issue

[jira] [Commented] (SPARK-23183) Failure caused by TaskContext is missing in the thread spawned by Custom RDD

[ https://issues.apache.org/jira/browse/SPARK-23183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335153#comment-16335153 ] Sean Owen commented on SPARK-23183: --- I don't think a custom RDD that spawns threads is guaranteed to

[jira] [Updated] (SPARK-23183) Failure caused by TaskContext is missing in the thread spawned by Custom RDD

[ https://issues.apache.org/jira/browse/SPARK-23183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongqin Xiao updated SPARK-23183: - Description: This is related to the already resolved issue

[jira] [Assigned] (SPARK-23163) Sync Python ML API docs with Scala

[ https://issues.apache.org/jira/browse/SPARK-23163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23163: Assignee: (was: Apache Spark) > Sync Python ML API docs with Scala >

[jira] [Commented] (SPARK-23163) Sync Python ML API docs with Scala

[ https://issues.apache.org/jira/browse/SPARK-23163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335040#comment-16335040 ] Apache Spark commented on SPARK-23163: -- User 'BryanCutler' has created a pull request for this

[jira] [Assigned] (SPARK-23163) Sync Python ML API docs with Scala

[ https://issues.apache.org/jira/browse/SPARK-23163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23163: Assignee: Apache Spark > Sync Python ML API docs with Scala >

[jira] [Commented] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335128#comment-16335128 ] Shixiong Zhu commented on SPARK-23184: -- This seems caused by the fix for SPARK-23051. > All jobs

[jira] [Commented] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335131#comment-16335131 ] Shixiong Zhu commented on SPARK-23184: -- cc [~vanzin] [~smurakozi] [~cloud_fan] > All jobs page is

[jira] [Commented] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335146#comment-16335146 ] Marcelo Vanzin commented on SPARK-23184: Did you test with the fix for SPARK-23121? > All jobs

[jira] [Resolved] (SPARK-22389) partitioning reporting

[ https://issues.apache.org/jira/browse/SPARK-22389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22389. - Resolution: Fixed Fix Version/s: 2.3.0 > partitioning reporting > -- > >

[jira] [Commented] (SPARK-23183) Failure caused by TaskContext is missing in the thread spawned by Custom RDD

[ https://issues.apache.org/jira/browse/SPARK-23183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335160#comment-16335160 ] Yongqin Xiao commented on SPARK-23183: -- Please look at reproduction mentioned in this site: 

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334892#comment-16334892 ] Apache Spark commented on SPARK-21727: -- User 'neilalex' has created a pull request for this issue:

[jira] [Resolved] (SPARK-11930) StreamInterceptor causes channel to be closed if user code throws exceptions

[ https://issues.apache.org/jira/browse/SPARK-11930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11930. Resolution: Won't Fix No point in looking at this until there's a use case. Current code

[jira] [Updated] (SPARK-23052) Migrate MicrConsoleSink to v2

[ https://issues.apache.org/jira/browse/SPARK-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-23052: -- Summary: Migrate MicrConsoleSink to v2 (was: Migrate ConsoleSink to v2) > Migrate

[jira] [Assigned] (SPARK-23052) Migrate ConsoleSink to v2

[ https://issues.apache.org/jira/browse/SPARK-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-23052: - Assignee: Jose Torres > Migrate ConsoleSink to v2 > - > >

[jira] [Assigned] (SPARK-11315) Add YARN extension service to publish Spark events to YARN timeline service

[ https://issues.apache.org/jira/browse/SPARK-11315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-11315: -- Assignee: (was: Marcelo Vanzin) > Add YARN extension service to publish Spark

[jira] [Resolved] (SPARK-10439) Catalyst should check for overflow / underflow of date and timestamp values

[ https://issues.apache.org/jira/browse/SPARK-10439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10439. Resolution: Won't Fix Doesn't look like there's any interest in fixing this. > Catalyst

[jira] [Commented] (SPARK-20664) Remove stale applications from SHS listing

[ https://issues.apache.org/jira/browse/SPARK-20664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334971#comment-16334971 ] Apache Spark commented on SPARK-20664: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23114) Spark R 2.3 QA umbrella

[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334885#comment-16334885 ] Hossein Falaki commented on SPARK-23114: [~felixcheung] I don't have any datasets to share. I

[jira] [Assigned] (SPARK-12977) Factoring out StreamingListener and UI to support history UI

[ https://issues.apache.org/jira/browse/SPARK-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-12977: -- Assignee: (was: Marcelo Vanzin) > Factoring out StreamingListener and UI to

[jira] [Assigned] (SPARK-12977) Factoring out StreamingListener and UI to support history UI

[ https://issues.apache.org/jira/browse/SPARK-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-12977: -- Assignee: Marcelo Vanzin > Factoring out StreamingListener and UI to support history

[jira] [Resolved] (SPARK-20841) Support table column aliases in FROM clause

[ https://issues.apache.org/jira/browse/SPARK-20841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20841. - Resolution: Fixed Fix Version/s: 2.3.0 > Support table column aliases in FROM clause >

[jira] [Created] (SPARK-23182) Allow enabling of TCP keep alive for master RPC connections

Petar Petrov created SPARK-23182: Summary: Allow enabling of TCP keep alive for master RPC connections Key: SPARK-23182 URL: https://issues.apache.org/jira/browse/SPARK-23182 Project: Spark

[jira] [Updated] (SPARK-23172) Expand the ReorderJoin rule to handle Project nodes

[ https://issues.apache.org/jira/browse/SPARK-23172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-23172: - Summary: Expand the ReorderJoin rule to handle Project nodes (was: Respect Project

[jira] [Issue Comment Deleted] (SPARK-21646) Add new type coercion rules to compatible with Hive

[ https://issues.apache.org/jira/browse/SPARK-21646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal updated SPARK-21646: --- Comment: was deleted (was: Marking it as a duplicate of SPARK-22722) > Add new type

[jira] [Assigned] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

[ https://issues.apache.org/jira/browse/SPARK-23148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23148: Assignee: (was: Apache Spark) > spark.read.csv with multiline=true gives

[jira] [Commented] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

[ https://issues.apache.org/jira/browse/SPARK-23148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335260#comment-16335260 ] Apache Spark commented on SPARK-23148: -- User 'henryr' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

[ https://issues.apache.org/jira/browse/SPARK-23148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23148: Assignee: Apache Spark > spark.read.csv with multiline=true gives FileNotFoundException

[jira] [Created] (SPARK-23185) Make the configuration "spark.default.parallelism" can be changed on each SQL session to decrease empty files

LvDongrong created SPARK-23185: -- Summary: Make the configuration "spark.default.parallelism" can be changed on each SQL session to decrease empty files Key: SPARK-23185 URL:

[jira] [Assigned] (SPARK-23185) Make the configuration "spark.default.parallelism" can be changed on each SQL session to decrease empty files

[ https://issues.apache.org/jira/browse/SPARK-23185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23185: Assignee: Apache Spark > Make the configuration "spark.default.parallelism" can be

[jira] [Assigned] (SPARK-23185) Make the configuration "spark.default.parallelism" can be changed on each SQL session to decrease empty files

[ https://issues.apache.org/jira/browse/SPARK-23185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23185: Assignee: (was: Apache Spark) > Make the configuration "spark.default.parallelism"

[jira] [Commented] (SPARK-23185) Make the configuration "spark.default.parallelism" can be changed on each SQL session to decrease empty files

[ https://issues.apache.org/jira/browse/SPARK-23185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335346#comment-16335346 ] Apache Spark commented on SPARK-23185: -- User 'lvdongr' has created a pull request for this issue:

[jira] [Created] (SPARK-23186) Loading JDBC Drivers should be syncronized

Dongjoon Hyun created SPARK-23186: - Summary: Loading JDBC Drivers should be syncronized Key: SPARK-23186 URL: https://issues.apache.org/jira/browse/SPARK-23186 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-23186) Loading JDBC Drivers should be syncronized

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23186: Assignee: Apache Spark > Loading JDBC Drivers should be syncronized >

[jira] [Commented] (SPARK-23186) Loading JDBC Drivers should be syncronized

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335369#comment-16335369 ] Apache Spark commented on SPARK-23186: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-23186) Loading JDBC Drivers should be syncronized

[ https://issues.apache.org/jira/browse/SPARK-23186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23186: Assignee: (was: Apache Spark) > Loading JDBC Drivers should be syncronized >

[jira] [Commented] (SPARK-20749) Built-in SQL Function Support - all variants of LEN[GTH]

[ https://issues.apache.org/jira/browse/SPARK-20749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335375#comment-16335375 ] Apache Spark commented on SPARK-20749: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-20927) Add cache operator to Unsupported Operations in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-20927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334363#comment-16334363 ] Mathieu DESPRIEE edited comment on SPARK-20927 at 1/22/18 4:19 PM: ---

[jira] [Created] (SPARK-23180) RFormulaModel should have labels member

Kevin Kuo created SPARK-23180: - Summary: RFormulaModel should have labels member Key: SPARK-23180 URL: https://issues.apache.org/jira/browse/SPARK-23180 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23173) from_json can produce nulls for fields which are marked as non-nullable

[ https://issues.apache.org/jira/browse/SPARK-23173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334005#comment-16334005 ] Liang-Chi Hsieh commented on SPARK-23173: - +1 for 1 too. > from_json can produce nulls for

[jira] [Resolved] (SPARK-23176) REPL project build failing in Spark v2.2.0

[ https://issues.apache.org/jira/browse/SPARK-23176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23176. --- Resolution: Not A Problem The 2.2.0 release built and builds correctly. You don't show a problem

[jira] [Comment Edited] (SPARK-20927) Add cache operator to Unsupported Operations in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-20927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334363#comment-16334363 ] Mathieu DESPRIEE edited comment on SPARK-20927 at 1/22/18 2:59 PM: ---

[jira] [Commented] (SPARK-20927) Add cache operator to Unsupported Operations in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-20927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334363#comment-16334363 ] Mathieu DESPRIEE commented on SPARK-20927: -- @[~zsxwing] actually it does make sens to cache

[jira] [Created] (SPARK-23179) Support option to throw exception if overflow occurs

Marco Gaido created SPARK-23179: --- Summary: Support option to throw exception if overflow occurs Key: SPARK-23179 URL: https://issues.apache.org/jira/browse/SPARK-23179 Project: Spark Issue

[jira] [Resolved] (SPARK-11630) ClosureCleaner incorrectly warns for class based closures

[ https://issues.apache.org/jira/browse/SPARK-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11630. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20337

  1   2   >