[jira] [Commented] (SPARK-23102) Migrate kafka sink

[ https://issues.apache.org/jira/browse/SPARK-23102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524628#comment-16524628 ] Richard Yu commented on SPARK-23102: Hi [~joseph.torres] Mind if I take this JIRA?

[jira] [Issue Comment Deleted] (SPARK-23102) Migrate kafka sink

[ https://issues.apache.org/jira/browse/SPARK-23102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Yu updated SPARK-23102: --- Comment: was deleted (was: [~joseph.torres] Just a question: I have noted that {{KafkaStreamWriter}}

[jira] [Comment Edited] (SPARK-23102) Migrate kafka sink

[ https://issues.apache.org/jira/browse/SPARK-23102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524615#comment-16524615 ] Richard Yu edited comment on SPARK-23102 at 6/27/18 6:04 AM: -

[jira] [Comment Edited] (SPARK-23102) Migrate kafka sink

[ https://issues.apache.org/jira/browse/SPARK-23102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524615#comment-16524615 ] Richard Yu edited comment on SPARK-23102 at 6/27/18 6:03 AM: -

[jira] [Commented] (SPARK-23102) Migrate kafka sink

[ https://issues.apache.org/jira/browse/SPARK-23102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524615#comment-16524615 ] Richard Yu commented on SPARK-23102: Just a question: I have noted that ```KafkaStre

[jira] [Commented] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

[ https://issues.apache.org/jira/browse/SPARK-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524603#comment-16524603 ] Apache Spark commented on SPARK-24665: -- User 'xuanyuanking' has created a pull requ

[jira] [Assigned] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

[ https://issues.apache.org/jira/browse/SPARK-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24665: Assignee: Apache Spark > Add SQLConf in PySpark to manage all sql configs > -

[jira] [Assigned] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

[ https://issues.apache.org/jira/browse/SPARK-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24665: Assignee: (was: Apache Spark) > Add SQLConf in PySpark to manage all sql configs > --

[jira] [Created] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

Li Yuanjian created SPARK-24665: --- Summary: Add SQLConf in PySpark to manage all sql configs Key: SPARK-24665 URL: https://issues.apache.org/jira/browse/SPARK-24665 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-24642) Add a function which infers schema from a JSON column

[ https://issues.apache.org/jira/browse/SPARK-24642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524590#comment-16524590 ] Reynold Xin commented on SPARK-24642: - Do we want this as an aggregate function? I'm

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524554#comment-16524554 ] Xiao Li commented on SPARK-24530: - [~hyukjin.kwon]  Thanks for helping this! > Sphinx d

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524544#comment-16524544 ] Hyukjin Kwon commented on SPARK-24530: -- Will take a look on the weekends. Please go

[jira] [Comment Edited] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524544#comment-16524544 ] Hyukjin Kwon edited comment on SPARK-24530 at 6/27/18 4:01 AM: ---

[jira] [Commented] (SPARK-21335) support un-aliased subquery

[ https://issues.apache.org/jira/browse/SPARK-21335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524527#comment-16524527 ] Apache Spark commented on SPARK-21335: -- User 'cnZach' has created a pull request fo

[jira] [Commented] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524519#comment-16524519 ] Apache Spark commented on SPARK-23927: -- User 'ueshin' has created a pull request fo

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24530: -- Summary: Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pysp

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (using Python 2?)

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24530: -- Summary: Sphinx doesn't render autodoc_docstring_signature correctly (using Python 2?) (was:

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524510#comment-16524510 ] Xiangrui Meng commented on SPARK-24530: --- Confirmed that macOS, python 3, and Sphin

[jira] [Resolved] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23927. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21155 [https://

[jira] [Assigned] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-23927: - Assignee: Alex Vayda > High-order function: sequence > - >

[jira] [Resolved] (SPARK-24605) size(null) should return null

[ https://issues.apache.org/jira/browse/SPARK-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24605. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21598 [https://gith

[jira] [Assigned] (SPARK-24605) size(null) should return null

[ https://issues.apache.org/jira/browse/SPARK-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24605: --- Assignee: Maxim Gekk > size(null) should return null > - > >

[jira] [Created] (SPARK-24664) Column support name getter

zhengruifeng created SPARK-24664: Summary: Column support name getter Key: SPARK-24664 URL: https://issues.apache.org/jira/browse/SPARK-24664 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23014) Migrate MemorySink fully to v2

[ https://issues.apache.org/jira/browse/SPARK-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524481#comment-16524481 ] Richard Yu commented on SPARK-23014: Hi [~joseph.torres] Are you still working on th

[jira] [Resolved] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24659. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21643 [https://gith

[jira] [Assigned] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24659: --- Assignee: Kris Mok > GenericArrayData.equals should respect element type differences >

[jira] [Updated] (SPARK-24447) Pyspark RowMatrix.columnSimilarities() loses spark context

[ https://issues.apache.org/jira/browse/SPARK-24447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Perry Chu updated SPARK-24447: -- Description: The RDD behind the CoordinateMatrix returned by RowMatrix.columnSimilarities() appears t

[jira] [Updated] (SPARK-24447) Pyspark RowMatrix.columnSimilarities() loses spark context

[ https://issues.apache.org/jira/browse/SPARK-24447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Perry Chu updated SPARK-24447: -- Description: The RDD behind the CoordinateMatrix returned by RowMatrix.columnSimilarities() appears t

[jira] [Updated] (SPARK-24447) Pyspark RowMatrix.columnSimilarities() loses spark context

[ https://issues.apache.org/jira/browse/SPARK-24447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Perry Chu updated SPARK-24447: -- Priority: Minor (was: Major) > Pyspark RowMatrix.columnSimilarities() loses spark context > -

[jira] [Created] (SPARK-24663) Flaky test: StreamingContextSuite "stop slow receiver gracefully"

Marcelo Vanzin created SPARK-24663: -- Summary: Flaky test: StreamingContextSuite "stop slow receiver gracefully" Key: SPARK-24663 URL: https://issues.apache.org/jira/browse/SPARK-24663 Project: Spark

[jira] [Commented] (SPARK-24208) Cannot resolve column in self join after applying Pandas UDF

[ https://issues.apache.org/jira/browse/SPARK-24208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524341#comment-16524341 ] Stu (Michael Stewart) commented on SPARK-24208: --- [~hyukjin.kwon] I can con

[jira] [Assigned] (SPARK-6237) Support uploading blocks > 2GB as a stream

[ https://issues.apache.org/jira/browse/SPARK-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-6237: - Assignee: Imran Rashid > Support uploading blocks > 2GB as a stream > --

[jira] [Resolved] (SPARK-6237) Support uploading blocks > 2GB as a stream

[ https://issues.apache.org/jira/browse/SPARK-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6237. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21346 [https://g

[jira] [Created] (SPARK-24662) Structured Streaming should support LIMIT

Mukul Murthy created SPARK-24662: Summary: Structured Streaming should support LIMIT Key: SPARK-24662 URL: https://issues.apache.org/jira/browse/SPARK-24662 Project: Spark Issue Type: New Fea

[jira] [Resolved] (SPARK-24423) Add a new option `query` for JDBC sources

[ https://issues.apache.org/jira/browse/SPARK-24423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24423. - Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 2.4.0 > Add a new option `query`

[jira] [Comment Edited] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524233#comment-16524233 ] Dongjoon Hyun edited comment on SPARK-24530 at 6/26/18 9:49 PM: --

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524233#comment-16524233 ] Dongjoon Hyun commented on SPARK-24530: --- [~mengxr] and [~hyukjin.kwon]. My environ

[jira] [Resolved] (SPARK-24658) Remove workaround for ANTLR bug

[ https://issues.apache.org/jira/browse/SPARK-24658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24658. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 2.4.0 > Remove workaround for ANT

[jira] [Assigned] (SPARK-24537) Add array_remove / array_zip / map_from_arrays / array_distinct

[ https://issues.apache.org/jira/browse/SPARK-24537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24537: Assignee: Apache Spark > Add array_remove / array_zip / map_from_arrays / array_distinct

[jira] [Commented] (SPARK-24537) Add array_remove / array_zip / map_from_arrays / array_distinct

[ https://issues.apache.org/jira/browse/SPARK-24537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524114#comment-16524114 ] Apache Spark commented on SPARK-24537: -- User 'huaxingao' has created a pull request

[jira] [Assigned] (SPARK-24537) Add array_remove / array_zip / map_from_arrays / array_distinct

[ https://issues.apache.org/jira/browse/SPARK-24537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24537: Assignee: (was: Apache Spark) > Add array_remove / array_zip / map_from_arrays / arra

[jira] [Issue Comment Deleted] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-24631: --- Comment: was deleted (was: User 'vanzin' has created a pull request for this issue: https://

[jira] [Commented] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523918#comment-16523918 ] Marcelo Vanzin commented on SPARK-24631: Sorry for the noise, pasted the wrong b

[jira] [Assigned] (SPARK-24653) Flaky test "JoinSuite.test SortMergeJoin (with spill)"

[ https://issues.apache.org/jira/browse/SPARK-24653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24653: Assignee: Apache Spark > Flaky test "JoinSuite.test SortMergeJoin (with spill)" > ---

[jira] [Assigned] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24631: -- Assignee: Marcelo Vanzin > Cannot up cast column from bigint to smallint as it may tr

[jira] [Commented] (SPARK-24653) Flaky test "JoinSuite.test SortMergeJoin (with spill)"

[ https://issues.apache.org/jira/browse/SPARK-24653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523916#comment-16523916 ] Apache Spark commented on SPARK-24653: -- User 'vanzin' has created a pull request fo

[jira] [Assigned] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24631: -- Assignee: (was: Marcelo Vanzin) > Cannot up cast column from bigint to smallint a

[jira] [Assigned] (SPARK-24653) Flaky test "JoinSuite.test SortMergeJoin (with spill)"

[ https://issues.apache.org/jira/browse/SPARK-24653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24653: Assignee: (was: Apache Spark) > Flaky test "JoinSuite.test SortMergeJoin (with spill)

[jira] [Comment Edited] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523888#comment-16523888 ] Hari Sekhon edited comment on SPARK-6305 at 6/26/18 3:47 PM: -

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523888#comment-16523888 ] Hari Sekhon commented on SPARK-6305: Log4j 2.x would really help with Spark logging i

[jira] [Updated] (SPARK-24661) Window API - using multiple fields for partitioning with WindowSpec API and dataset that is cached causes org.apache.spark.sql.catalyst.errors.package$TreeNodeException

[ https://issues.apache.org/jira/browse/SPARK-24661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mavashev updated SPARK-24661: --- Description: Steps to reproduce: Creating a data set:   {code:java} List simpleWindowColum

[jira] [Created] (SPARK-24661) Window API - using multiple fields for partitioning with WindowSpec API and dataset that is cached causes org.apache.spark.sql.catalyst.errors.package$TreeNodeException

David Mavashev created SPARK-24661: -- Summary: Window API - using multiple fields for partitioning with WindowSpec API and dataset that is cached causes org.apache.spark.sql.catalyst.errors.package$TreeNodeException Key: SPARK-24

[jira] [Assigned] (SPARK-24660) SHS is not showing properly errors when downloading logs

[ https://issues.apache.org/jira/browse/SPARK-24660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24660: Assignee: Apache Spark > SHS is not showing properly errors when downloading logs > -

[jira] [Commented] (SPARK-24660) SHS is not showing properly errors when downloading logs

[ https://issues.apache.org/jira/browse/SPARK-24660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523788#comment-16523788 ] Apache Spark commented on SPARK-24660: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-24660) SHS is not showing properly errors when downloading logs

[ https://issues.apache.org/jira/browse/SPARK-24660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24660: Assignee: (was: Apache Spark) > SHS is not showing properly errors when downloading l

[jira] [Created] (SPARK-24660) SHS is not showing properly errors when downloading logs

Marco Gaido created SPARK-24660: --- Summary: SHS is not showing properly errors when downloading logs Key: SPARK-24660 URL: https://issues.apache.org/jira/browse/SPARK-24660 Project: Spark Issue

[jira] [Assigned] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24659: Assignee: Apache Spark > GenericArrayData.equals should respect element type differences

[jira] [Assigned] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24659: Assignee: (was: Apache Spark) > GenericArrayData.equals should respect element type d

[jira] [Commented] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523578#comment-16523578 ] Apache Spark commented on SPARK-24659: -- User 'rednaxelafx' has created a pull reque

[jira] [Created] (SPARK-24659) GenericArrayData.equals should respect element type differences

Kris Mok created SPARK-24659: Summary: GenericArrayData.equals should respect element type differences Key: SPARK-24659 URL: https://issues.apache.org/jira/browse/SPARK-24659 Project: Spark Issu

[jira] [Commented] (SPARK-18649) sc.textFile(my_file).collect() raises socket.timeout on large files

[ https://issues.apache.org/jira/browse/SPARK-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523479#comment-16523479 ] Andrei Gorlanov commented on SPARK-18649: - Hello, I am going to take care of it.

[jira] [Commented] (SPARK-24347) df.alias() in python API should not clear metadata by default

[ https://issues.apache.org/jira/browse/SPARK-24347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523476#comment-16523476 ] Ruben Berenguel commented on SPARK-24347: - Pinging [~hyukjin.kwon], too :) > df

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523475#comment-16523475 ] Ruben Berenguel commented on SPARK-24458: - Oh, big facepalm, thanks [~hyukjin.kw

[jira] [Assigned] (SPARK-22425) add output files information to EventLogger

[ https://issues.apache.org/jira/browse/SPARK-22425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22425: Assignee: (was: Apache Spark) > add output files information to EventLogger > ---

[jira] [Commented] (SPARK-22425) add output files information to EventLogger

[ https://issues.apache.org/jira/browse/SPARK-22425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523459#comment-16523459 ] Apache Spark commented on SPARK-22425: -- User 'voidfunction' has created a pull requ

[jira] [Assigned] (SPARK-22425) add output files information to EventLogger

[ https://issues.apache.org/jira/browse/SPARK-22425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22425: Assignee: Apache Spark > add output files information to EventLogger > --

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523443#comment-16523443 ] Hyukjin Kwon commented on SPARK-24458: -- I usually just checkout on the tag, for exa

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523433#comment-16523433 ] Hyukjin Kwon commented on SPARK-24530: -- I have another computer: macOS, Python 2.7.

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523431#comment-16523431 ] Hyukjin Kwon commented on SPARK-24530: -- macOS, Python 2.7.14, Sphinx 1.4.1 shows:

[jira] [Commented] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523417#comment-16523417 ] Hyukjin Kwon commented on SPARK-24570: -- So you are saying {code} == SQL == SHOW TA

[jira] [Commented] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523405#comment-16523405 ] Hyukjin Kwon commented on SPARK-24647: -- (please avoid to set a fix version which is

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24647: - Fix Version/s: (was: 2.4.0) > Sink Should Return OffsetSeqs For ProgressReporting >

[jira] [Commented] (SPARK-24643) from_json should accept an aggregate function as schema

[ https://issues.apache.org/jira/browse/SPARK-24643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523403#comment-16523403 ] Hyukjin Kwon commented on SPARK-24643: -- SPARK-24642 is not added yet though ... >

[jira] [Commented] (SPARK-24644) Pyarrow exception while running pandas_udf on pyspark 2.3.1

[ https://issues.apache.org/jira/browse/SPARK-24644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523399#comment-16523399 ] Hyukjin Kwon commented on SPARK-24644: -- Can you clarify the environment, in particu

[jira] [Resolved] (SPARK-24649) SparkUDF.unapply is not backwards compatable

[ https://issues.apache.org/jira/browse/SPARK-24649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24649. -- Resolution: Invalid catalysis is considered as an internal API, and subject to change between

[jira] [Commented] (SPARK-24651) Add ability to write null values while writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523383#comment-16523383 ] Hyukjin Kwon commented on SPARK-24651: -- I think it's basically a duplicate of SPARK

[jira] [Commented] (SPARK-24650) GroupingSet

[ https://issues.apache.org/jira/browse/SPARK-24650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523351#comment-16523351 ] Hyukjin Kwon commented on SPARK-24650: -- Please avoid to set a blocker which is usua

[jira] [Updated] (SPARK-24650) GroupingSet

[ https://issues.apache.org/jira/browse/SPARK-24650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24650: - Priority: Major (was: Blocker) > GroupingSet > --- > > Key: SPARK-24650

[jira] [Commented] (SPARK-24528) Missing optimization for Aggregations/Windowing on a bucketed table

[ https://issues.apache.org/jira/browse/SPARK-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523344#comment-16523344 ] Ohad Raviv commented on SPARK-24528: Hi, well it took me some time to get to it, bu