[jira] [Commented] (SPARK-37741) Remove Jenkins badge in README.md

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465303#comment-17465303 ] Apache Spark commented on SPARK-37741: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37741) Remove Jenkins badge in README.md

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37741: Assignee: (was: Apache Spark) > Remove Jenkins badge in README.md >

[jira] [Commented] (SPARK-37741) Remove Jenkins badge in README.md

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465302#comment-17465302 ] Apache Spark commented on SPARK-37741: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37741) Remove Jenkins badge in README.md

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37741: Assignee: Apache Spark > Remove Jenkins badge in README.md >

[jira] [Updated] (SPARK-37744) Remove/update obsolete codes in PySpark, SparkR and other docs

2021-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37744: - Description: {code} R/check-cran.sh:# Jenkins installs arrow. See SPARK-29339.

[jira] [Updated] (SPARK-37743) Remove/update obsolete codes in Scala tests

2021-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37743: - Description: {code} core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala: //

[jira] [Updated] (SPARK-37743) Remove/update obsolete codes in Scala tests

2021-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37743: - Description: {code} core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala: //

[jira] [Created] (SPARK-37744) Remove/update obsolete codes in PySpark, SparkR and other docs

2021-12-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37744: Summary: Remove/update obsolete codes in PySpark, SparkR and other docs Key: SPARK-37744 URL: https://issues.apache.org/jira/browse/SPARK-37744 Project: Spark

[jira] [Created] (SPARK-37743) Remove/update obsolete codes in Scala tests

2021-12-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37743: Summary: Remove/update obsolete codes in Scala tests Key: SPARK-37743 URL: https://issues.apache.org/jira/browse/SPARK-37743 Project: Spark Issue Type:

[jira] [Created] (SPARK-37742) Remove obsolate codes in testing scripts for Jenkins

2021-12-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37742: Summary: Remove obsolate codes in testing scripts for Jenkins Key: SPARK-37742 URL: https://issues.apache.org/jira/browse/SPARK-37742 Project: Spark Issue

[jira] [Created] (SPARK-37741) Remove Jenkins badge in README.md

2021-12-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37741: Summary: Remove Jenkins badge in README.md Key: SPARK-37741 URL: https://issues.apache.org/jira/browse/SPARK-37741 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-37740) Removal of obsolete codes per Jenkins' retirement

2021-12-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37740: Summary: Removal of obsolete codes per Jenkins' retirement Key: SPARK-37740 URL: https://issues.apache.org/jira/browse/SPARK-37740 Project: Spark Issue

[jira] [Resolved] (SPARK-37739) Upgrade Arrow to 6.0.1

2021-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37739. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35024

[jira] [Assigned] (SPARK-37739) Upgrade Arrow to 6.0.1

2021-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37739: - Assignee: William Hyun > Upgrade Arrow to 6.0.1 > -- > >

[jira] [Commented] (SPARK-37708) pyspark adding third-party Dependencies on k8s

2021-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465293#comment-17465293 ] Hyukjin Kwon commented on SPARK-37708: -- [~zhongjingxiong] If it works, we should try. I think it

[jira] [Commented] (SPARK-37738) PySpark date_add only accepts an integer as it's second parameter

2021-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465292#comment-17465292 ] Hyukjin Kwon commented on SPARK-37738: -- we should make it supported for columns too if it only

[jira] [Updated] (SPARK-37738) PySpark date_add only accepts an integer as it's second parameter

2021-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37738: - Component/s: SQL (was: Spark Core) > PySpark date_add only accepts an

[jira] [Resolved] (SPARK-37737) Update Black to 21.12.b0

2021-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37737. -- Fix Version/s: 3.3.0 Assignee: Maciej Szymkiewicz Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-37739) Upgrade Arrow to 6.0.1

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465290#comment-17465290 ] Apache Spark commented on SPARK-37739: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-37739) Upgrade Arrow to 6.0.1

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37739: Assignee: Apache Spark > Upgrade Arrow to 6.0.1 > -- > >

[jira] [Commented] (SPARK-37739) Upgrade Arrow to 6.0.1

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465289#comment-17465289 ] Apache Spark commented on SPARK-37739: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-37739) Upgrade Arrow to 6.0.1

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37739: Assignee: (was: Apache Spark) > Upgrade Arrow to 6.0.1 > -- > >

[jira] [Created] (SPARK-37739) Upgrade Arrow to 6.0.1

2021-12-25 Thread William Hyun (Jira)
William Hyun created SPARK-37739: Summary: Upgrade Arrow to 6.0.1 Key: SPARK-37739 URL: https://issues.apache.org/jira/browse/SPARK-37739 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37659) Fix FsHistoryProvider race condition between list and delet log info

2021-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37659: -- Fix Version/s: 3.2.1 > Fix FsHistoryProvider race condition between list and delet log info >

[jira] [Updated] (SPARK-37659) Fix FsHistoryProvider race condition between list and delet log info

2021-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37659: -- Target Version/s: (was: 3.3.0) > Fix FsHistoryProvider race condition between list and

[jira] [Commented] (SPARK-37709) Add AVERAGE_DURATION executor roll policy

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465283#comment-17465283 ] Apache Spark commented on SPARK-37709: -- User 'williamhyun' has created a pull request for this

[jira] [Commented] (SPARK-37709) Add AVERAGE_DURATION executor roll policy

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465282#comment-17465282 ] Apache Spark commented on SPARK-37709: -- User 'williamhyun' has created a pull request for this

[jira] [Created] (SPARK-37738) PySpark date_add only accepts an integer as it's second parameter

2021-12-25 Thread Daniel Davies (Jira)
Daniel Davies created SPARK-37738: - Summary: PySpark date_add only accepts an integer as it's second parameter Key: SPARK-37738 URL: https://issues.apache.org/jira/browse/SPARK-37738 Project: Spark

[jira] [Commented] (SPARK-37407) Inline type hints for python/pyspark/ml/functions.py

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465246#comment-17465246 ] Apache Spark commented on SPARK-37407: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-37737) Update Black to 21.12.b0

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465208#comment-17465208 ] Apache Spark commented on SPARK-37737: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37737) Update Black to 21.12.b0

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37737: Assignee: (was: Apache Spark) > Update Black to 21.12.b0 >

[jira] [Assigned] (SPARK-37737) Update Black to 21.12.b0

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37737: Assignee: Apache Spark > Update Black to 21.12.b0 > > >

[jira] [Created] (SPARK-37737) Update Black to 21.12.b0

2021-12-25 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-37737: -- Summary: Update Black to 21.12.b0 Key: SPARK-37737 URL: https://issues.apache.org/jira/browse/SPARK-37737 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-37407) Inline type hints for python/pyspark/ml/functions.py

2021-12-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37407. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37407) Inline type hints for python/pyspark/ml/functions.py

2021-12-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37407: -- Assignee: dch nguyen > Inline type hints for python/pyspark/ml/functions.py

[jira] [Commented] (SPARK-37429) Inline type hints for python/pyspark/mllib/linalg/__init__.py

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465204#comment-17465204 ] Apache Spark commented on SPARK-37429: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37429) Inline type hints for python/pyspark/mllib/linalg/__init__.py

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37429: Assignee: Apache Spark > Inline type hints for python/pyspark/mllib/linalg/__init__.py >

[jira] [Commented] (SPARK-37429) Inline type hints for python/pyspark/mllib/linalg/__init__.py

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465203#comment-17465203 ] Apache Spark commented on SPARK-37429: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37429) Inline type hints for python/pyspark/mllib/linalg/__init__.py

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37429: Assignee: (was: Apache Spark) > Inline type hints for

[jira] [Commented] (SPARK-34652) Support SchemaRegistry in from_avro method

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465202#comment-17465202 ] Apache Spark commented on SPARK-34652: -- User 'blcksrx' has created a pull request for this issue:

[jira] [Commented] (SPARK-34652) Support SchemaRegistry in from_avro method

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465201#comment-17465201 ] Apache Spark commented on SPARK-34652: -- User 'blcksrx' has created a pull request for this issue:

[jira] [Commented] (SPARK-37736) Comparation between string(great than Int.MaxValue, such as '2147483648') and int return null

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465155#comment-17465155 ] Apache Spark commented on SPARK-37736: -- User 'monkeyboy123' has created a pull request for this

[jira] [Assigned] (SPARK-37736) Comparation between string(great than Int.MaxValue, such as '2147483648') and int return null

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37736: Assignee: (was: Apache Spark) > Comparation between string(great than Int.MaxValue,

[jira] [Assigned] (SPARK-37736) Comparation between string(great than Int.MaxValue, such as '2147483648') and int return null

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37736: Assignee: Apache Spark > Comparation between string(great than Int.MaxValue, such as

[jira] [Commented] (SPARK-37736) Comparation between string(great than Int.MaxValue, such as '2147483648') and int return null

2021-12-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465154#comment-17465154 ] Apache Spark commented on SPARK-37736: -- User 'monkeyboy123' has created a pull request for this

[jira] [Updated] (SPARK-37736) Comparation between string(great than Int.MaxValue, such as '2147483648') and int return null

2021-12-25 Thread jiahong.li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li updated SPARK-37736: --- Priority: Major (was: Minor) > Comparation between string(great than Int.MaxValue, such as

[jira] [Created] (SPARK-37736) Comparation between string(great than Int.MaxValue, such as '2147483648') and int return null

2021-12-25 Thread jiahong.li (Jira)
jiahong.li created SPARK-37736: -- Summary: Comparation between string(great than Int.MaxValue, such as '2147483648') and int return null Key: SPARK-37736 URL: https://issues.apache.org/jira/browse/SPARK-37736