[jira] [Commented] (SPARK-37740) Migrate to M1 machines in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-37740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465560#comment-17465560 ] Dongjoon Hyun commented on SPARK-37740: --- Thank you, [~hyukjin.kwon]! > Migrate to M1 machines in

[jira] [Created] (SPARK-37749) Built-in ORC reader cannot read data file in sub-directories created by Hive Tez

Ye Li created SPARK-37749: - Summary: Built-in ORC reader cannot read data file in sub-directories created by Hive Tez Key: SPARK-37749 URL: https://issues.apache.org/jira/browse/SPARK-37749 Project: Spark

[jira] [Commented] (SPARK-37747) Upgrade zstd-jni to 1.5.1-1

[ https://issues.apache.org/jira/browse/SPARK-37747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465550#comment-17465550 ] Apache Spark commented on SPARK-37747: -- User 'williamhyun' has created a pull request for this

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465507#comment-17465507 ] Apache Spark commented on SPARK-34544: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465506#comment-17465506 ] Apache Spark commented on SPARK-34544: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-37741) Remove Jenkins badge in README.md

[ https://issues.apache.org/jira/browse/SPARK-37741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37741. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35025

[jira] [Commented] (SPARK-37740) Migrate to M1 machines in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-37740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465499#comment-17465499 ] Hyukjin Kwon commented on SPARK-37740: -- cc [~dongjoon] [~viirya] [~dbtsai] FYI > Migrate to M1

[jira] [Updated] (SPARK-37743) Revisit codes in Scala tests

[ https://issues.apache.org/jira/browse/SPARK-37743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37743: - Summary: Revisit codes in Scala tests (was: Remove/update obsolete codes in Scala tests) >

[jira] [Assigned] (SPARK-37741) Remove Jenkins badge in README.md

[ https://issues.apache.org/jira/browse/SPARK-37741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37741: Assignee: Hyukjin Kwon > Remove Jenkins badge in README.md >

[jira] [Updated] (SPARK-37742) Revisit codes in testing scripts for Jenkins

[ https://issues.apache.org/jira/browse/SPARK-37742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37742: - Summary: Revisit codes in testing scripts for Jenkins (was: Remove obsolete code in testing

[jira] [Updated] (SPARK-37744) Revisit codes in PySpark, SparkR and other docs

[ https://issues.apache.org/jira/browse/SPARK-37744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37744: - Summary: Revisit codes in PySpark, SparkR and other docs (was: Remove/update obsolete codes in

[jira] [Updated] (SPARK-37740) Migrate to M1 machines in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-37740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37740: - Description: See

[jira] [Created] (SPARK-37748) Add Jenkins badge back when M1 machines are ready

Hyukjin Kwon created SPARK-37748: Summary: Add Jenkins badge back when M1 machines are ready Key: SPARK-37748 URL: https://issues.apache.org/jira/browse/SPARK-37748 Project: Spark Issue

[jira] [Updated] (SPARK-37740) Migrate to M1 machines in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-37740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37740: - Summary: Migrate to M1 machines in Jenkins (was: Removal of obsolete codes per Jenkins'

[jira] [Updated] (SPARK-37740) Migrate to M1 machines in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-37740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37740: - Description: See

[jira] [Assigned] (SPARK-37578) DSV2 is not updating Output Metrics

[ https://issues.apache.org/jira/browse/SPARK-37578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37578: Assignee: Apache Spark > DSV2 is not updating Output Metrics >

[jira] [Commented] (SPARK-37578) DSV2 is not updating Output Metrics

[ https://issues.apache.org/jira/browse/SPARK-37578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465498#comment-17465498 ] Apache Spark commented on SPARK-37578: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37578) DSV2 is not updating Output Metrics

[ https://issues.apache.org/jira/browse/SPARK-37578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37578: Assignee: (was: Apache Spark) > DSV2 is not updating Output Metrics >

[jira] [Resolved] (SPARK-37747) Upgrade zstd-jni to 1.5.1-1

[ https://issues.apache.org/jira/browse/SPARK-37747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37747. --- Fix Version/s: 3.3.0 Assignee: William Hyun Resolution: Fixed This is

[jira] [Commented] (SPARK-37747) Upgrade zstd-jni to 1.5.1-1

[ https://issues.apache.org/jira/browse/SPARK-37747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465452#comment-17465452 ] Apache Spark commented on SPARK-37747: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-37747) Upgrade zstd-jni to 1.5.1-1

[ https://issues.apache.org/jira/browse/SPARK-37747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37747: Assignee: Apache Spark > Upgrade zstd-jni to 1.5.1-1 > --- > >

[jira] [Commented] (SPARK-37747) Upgrade zstd-jni to 1.5.1-1

[ https://issues.apache.org/jira/browse/SPARK-37747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465451#comment-17465451 ] Apache Spark commented on SPARK-37747: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-37747) Upgrade zstd-jni to 1.5.1-1

[ https://issues.apache.org/jira/browse/SPARK-37747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37747: Assignee: (was: Apache Spark) > Upgrade zstd-jni to 1.5.1-1 >

[jira] [Created] (SPARK-37747) Upgrade zstd-jni to 1.5.1-1

William Hyun created SPARK-37747: Summary: Upgrade zstd-jni to 1.5.1-1 Key: SPARK-37747 URL: https://issues.apache.org/jira/browse/SPARK-37747 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-37697) Make it easier to convert numpy arrays to Spark Dataframes

[ https://issues.apache.org/jira/browse/SPARK-37697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465443#comment-17465443 ] Daniel Davies edited comment on SPARK-37697 at 12/26/21, 8:29 PM: -- Hey

[jira] [Comment Edited] (SPARK-37697) Make it easier to convert numpy arrays to Spark Dataframes

[ https://issues.apache.org/jira/browse/SPARK-37697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465443#comment-17465443 ] Daniel Davies edited comment on SPARK-37697 at 12/26/21, 8:24 PM: -- Hey

[jira] [Comment Edited] (SPARK-37697) Make it easier to convert numpy arrays to Spark Dataframes

[ https://issues.apache.org/jira/browse/SPARK-37697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465443#comment-17465443 ] Daniel Davies edited comment on SPARK-37697 at 12/26/21, 8:23 PM: -- Hey

[jira] [Commented] (SPARK-37697) Make it easier to convert numpy arrays to Spark Dataframes

[ https://issues.apache.org/jira/browse/SPARK-37697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465443#comment-17465443 ] Daniel Davies commented on SPARK-37697: --- Hey Douglas,   I've definitely been caught by numpy types

[jira] [Comment Edited] (SPARK-37697) Make it easier to convert numpy arrays to Spark Dataframes

[ https://issues.apache.org/jira/browse/SPARK-37697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465443#comment-17465443 ] Daniel Davies edited comment on SPARK-37697 at 12/26/21, 8:22 PM: -- Hey

[jira] [Commented] (SPARK-37738) PySpark date_add only accepts an integer as it's second parameter

[ https://issues.apache.org/jira/browse/SPARK-37738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465417#comment-17465417 ] Daniel Davies commented on SPARK-37738: --- Thanks for the quick reply [~hyukjin.kwon] - I'll take a

[jira] [Commented] (SPARK-37746) log4j2-defaults.properties is not working since log4j 2 is always initialized by default

[ https://issues.apache.org/jira/browse/SPARK-37746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465405#comment-17465405 ] Apache Spark commented on SPARK-37746: -- User 'chia7712' has created a pull request for this issue:

[jira] [Commented] (SPARK-37746) log4j2-defaults.properties is not working since log4j 2 is always initialized by default

[ https://issues.apache.org/jira/browse/SPARK-37746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465406#comment-17465406 ] Apache Spark commented on SPARK-37746: -- User 'chia7712' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37746) log4j2-defaults.properties is not working since log4j 2 is always initialized by default

[ https://issues.apache.org/jira/browse/SPARK-37746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37746: Assignee: Apache Spark > log4j2-defaults.properties is not working since log4j 2 is

[jira] [Assigned] (SPARK-37746) log4j2-defaults.properties is not working since log4j 2 is always initialized by default

[ https://issues.apache.org/jira/browse/SPARK-37746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37746: Assignee: (was: Apache Spark) > log4j2-defaults.properties is not working since

[jira] [Updated] (SPARK-37746) log4j2-defaults.properties is not working since log4j 2 is always initialized by default

[ https://issues.apache.org/jira/browse/SPARK-37746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated SPARK-37746: --- Description: the code used to check initialization is shown below.       val

[jira] [Updated] (SPARK-37746) log4j2-defaults.properties is not working since log4j 2 is always initialized by default

[ https://issues.apache.org/jira/browse/SPARK-37746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated SPARK-37746: --- Description: the code used to check initialization is shown below.       val

[jira] [Created] (SPARK-37746) log4j2-defaults.properties is not working since log4j 2 is always initialized by default

Chia-Ping Tsai created SPARK-37746: -- Summary: log4j2-defaults.properties is not working since log4j 2 is always initialized by default Key: SPARK-37746 URL: https://issues.apache.org/jira/browse/SPARK-37746

[jira] [Updated] (SPARK-37742) Remove obsolete code in testing scripts for Jenkins

[ https://issues.apache.org/jira/browse/SPARK-37742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-37742: - Summary: Remove obsolete code in testing scripts for Jenkins (was: Remove obsolate codes in

[jira] [Created] (SPARK-37745) Add new option to CSVOption ignoreEmptyLines

Izek Greenfield created SPARK-37745: --- Summary: Add new option to CSVOption ignoreEmptyLines Key: SPARK-37745 URL: https://issues.apache.org/jira/browse/SPARK-37745 Project: Spark Issue