[jira] [Created] (SPARK-26610) Fix inconsistency between toJSON Method in Python and Scala

2019-01-13 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26610: - Summary: Fix inconsistency between toJSON Method in Python and Scala Key: SPARK-26610 URL: https://issues.apache.org/jira/browse/SPARK-26610 Project: Spark

[jira] [Updated] (SPARK-26482) Use ConfigEntry for hardcoded configs for ui categories.

2018-12-29 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-26482: -- Description: Make the following hardcoded configs to use ConfigEntry. {code} spark.ui

[jira] [Resolved] (SPARK-26484) Use ConfigEntry for hardcoded configs for authenticate category.

2018-12-29 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-26484. --- Resolution: Duplicate > Use ConfigEntry for hardcoded configs for authenticate category. >

[jira] [Resolved] (SPARK-26480) Use ConfigEntry for hardcoded configs for broadcast category.

2018-12-29 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-26480. --- Resolution: Duplicate > Use ConfigEntry for hardcoded configs for broadcast category. >

[jira] [Updated] (SPARK-26482) Use ConfigEntry for hardcoded configs for ui categories.

2018-12-29 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-26482: -- Summary: Use ConfigEntry for hardcoded configs for ui categories. (was: Use ConfigEntry for

[jira] [Resolved] (SPARK-26487) Use ConfigEntry for hardcoded configs for admin category.

2018-12-29 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-26487. --- Resolution: Duplicate > Use ConfigEntry for hardcoded configs for admin category. >

[jira] [Resolved] (SPARK-26485) Use ConfigEntry for hardcoded configs for master.rest/ui categories.

2018-12-29 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-26485. --- Resolution: Duplicate > Use ConfigEntry for hardcoded configs for master.rest/ui

[jira] [Resolved] (SPARK-26483) Use ConfigEntry for hardcoded configs for ssl category.

2018-12-29 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-26483. --- Resolution: Duplicate > Use ConfigEntry for hardcoded configs for ssl category. >

[jira] [Resolved] (SPARK-26481) Use ConfigEntry for hardcoded configs for reducer category.

2018-12-29 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-26481. --- Resolution: Duplicate > Use ConfigEntry for hardcoded configs for reducer category. >

[jira] [Commented] (SPARK-26442) Use ConfigEntry for hardcoded configs.

2018-12-28 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730334#comment-16730334 ] Takuya Ueshin commented on SPARK-26442: --- I'm sorry for bothering you. Since there are so many

[jira] [Commented] (SPARK-26442) Use ConfigEntry for hardcoded configs.

2018-12-28 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730528#comment-16730528 ] Takuya Ueshin commented on SPARK-26442: --- I see. I'll combine them into several subtasks. I'll

[jira] [Created] (SPARK-25418) The metadata of DataSource table should not include Hive-generated storage properties.

2018-09-12 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-25418: - Summary: The metadata of DataSource table should not include Hive-generated storage properties. Key: SPARK-25418 URL: https://issues.apache.org/jira/browse/SPARK-25418

[jira] [Updated] (SPARK-25431) Fix function examples and unify the format of the example results.

2018-09-14 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-25431: -- Summary: Fix function examples and unify the format of the example results. (was: Fix

[jira] [Created] (SPARK-25431) Fix function examples and unify the format of the functions results.

2018-09-14 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-25431: - Summary: Fix function examples and unify the format of the functions results. Key: SPARK-25431 URL: https://issues.apache.org/jira/browse/SPARK-25431 Project:

[jira] [Commented] (SPARK-21914) Running examples as tests in SQL builtin function documentation

2018-09-17 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617123#comment-16617123 ] Takuya Ueshin commented on SPARK-21914: --- Actually I was just thinking of the tests like that. I

[jira] [Commented] (SPARK-25431) Fix function examples and unify the format of the example results.

2018-09-17 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617169#comment-16617169 ] Takuya Ueshin commented on SPARK-25431: --- Seems like the automatic link didn't work again. The new

[jira] [Resolved] (SPARK-25444) Refactor GenArrayData.genCodeToCreateArrayData() method

2018-09-17 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-25444. --- Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.5.0 Issue

[jira] [Created] (SPARK-27314) Deduplicate exprIds for Union.

2019-03-28 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-27314: - Summary: Deduplicate exprIds for Union. Key: SPARK-27314 URL: https://issues.apache.org/jira/browse/SPARK-27314 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-27240) Use pandas DataFrame for struct type argument in Scalar Pandas UDF.

2019-03-22 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-27240: - Summary: Use pandas DataFrame for struct type argument in Scalar Pandas UDF. Key: SPARK-27240 URL: https://issues.apache.org/jira/browse/SPARK-27240 Project: Spark

[jira] [Created] (SPARK-26887) Create datetime.date directly instead of creating datetime64[ns] as intermediate data.

2019-02-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-26887: - Summary: Create datetime.date directly instead of creating datetime64[ns] as intermediate data. Key: SPARK-26887 URL: https://issues.apache.org/jira/browse/SPARK-26887

[jira] [Created] (SPARK-28052) ArrayExists should follow the three-valued boolean logic.

2019-06-14 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-28052: - Summary: ArrayExists should follow the three-valued boolean logic. Key: SPARK-28052 URL: https://issues.apache.org/jira/browse/SPARK-28052 Project: Spark

[jira] [Updated] (SPARK-28052) ArrayExists should follow the three-valued boolean logic.

2019-06-14 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-28052: -- Description: Currently {{ArrayExists}} always returns boolean values (if the arguments are

[jira] [Resolved] (SPARK-28077) ANSI SQL: OVERLAY function(T312)

2019-06-28 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-28077. --- Resolution: Fixed Assignee: jiaan.geng Fix Version/s: 3.0.0 Issue resolved

[jira] [Resolved] (SPARK-27905) Add higher order function`forall`

2019-08-06 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-27905. --- Resolution: Fixed Assignee: Nikolas Vanderhoof Fix Version/s: 3.0.0 Issue

[jira] [Resolved] (SPARK-27297) Add higher order functions to Scala API

2019-10-02 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-27297. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 24232

[jira] [Assigned] (SPARK-27297) Add higher order functions to Scala API

2019-10-02 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-27297: - Assignee: Nikolas Vanderhoof > Add higher order functions to Scala API >

[jira] [Resolved] (SPARK-28962) High-order function: filter(array, function) → array

2019-10-02 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-28962. --- Fix Version/s: 3.0.0 Assignee: Henry Davidge Resolution: Fixed Issue

[jira] [Updated] (SPARK-28052) ArrayExists should follow the three-valued boolean logic.

2019-06-14 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-28052: -- Description: Currently {{ArrayExists}} always returns boolean values (if the arguments are

[jira] [Created] (SPARK-31441) Support duplicated column names for toPandas with arrow execution.

2020-04-13 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-31441: - Summary: Support duplicated column names for toPandas with arrow execution. Key: SPARK-31441 URL: https://issues.apache.org/jira/browse/SPARK-31441 Project: Spark

[jira] [Updated] (SPARK-31441) Support duplicated column names for toPandas with Arrow execution.

2020-04-13 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31441: -- Summary: Support duplicated column names for toPandas with Arrow execution. (was: Support

[jira] [Resolved] (SPARK-32897) SparkSession.builder.getOrCreate should not show deprecation warning of SQLContext

2020-09-16 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-32897. --- Fix Version/s: 3.1.0 3.0.2 Assignee: Hyukjin Kwon

[jira] [Updated] (SPARK-32897) SparkSession.builder.getOrCreate should not show deprecation warning of SQLContext

2020-09-16 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-32897: -- Affects Version/s: (was: 2.4.7) > SparkSession.builder.getOrCreate should not show

[jira] [Created] (SPARK-32160) Executors should not be able to create SparkContext.

2020-07-02 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-32160: - Summary: Executors should not be able to create SparkContext. Key: SPARK-32160 URL: https://issues.apache.org/jira/browse/SPARK-32160 Project: Spark Issue

[jira] [Created] (SPARK-31945) Make more cache enable for Python UDFs.

2020-06-09 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-31945: - Summary: Make more cache enable for Python UDFs. Key: SPARK-31945 URL: https://issues.apache.org/jira/browse/SPARK-31945 Project: Spark Issue Type:

[jira] [Updated] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-04 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31903: -- Component/s: R > toPandas with Arrow enabled doesn't show metrics in Query UI. >

[jira] [Created] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-03 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-31903: - Summary: toPandas with Arrow enabled doesn't show metrics in Query UI. Key: SPARK-31903 URL: https://issues.apache.org/jira/browse/SPARK-31903 Project: Spark

[jira] [Updated] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31903: -- Attachment: Screen Shot 2020-06-03 at 4.47.27 PM.png > toPandas with Arrow enabled doesn't

[jira] [Updated] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31903: -- Attachment: Screen Shot 2020-06-03 at 4.47.07 PM.png > toPandas with Arrow enabled doesn't

[jira] [Updated] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31903: -- Description: When calling {{toPandas}}, usually Query UI shows each plan node's metric and

[jira] [Updated] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31903: -- Description: When calling {{toPandas}}, usually Query UI shows each plan node's metric and

[jira] [Updated] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31903: -- Description: When calling {{toPandas}}, usually Query UI shows each plan node's metric and

[jira] [Updated] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31903: -- Description: When calling {{toPandas}}, usually Query UI shows each plan node's metric and

[jira] [Updated] (SPARK-31903) toPandas with Arrow enabled doesn't show metrics in Query UI.

2020-06-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-31903: -- Description: When calling {{toPandas}}, usually Query UI shows each plan node's metric and

[jira] [Resolved] (SPARK-32338) Add overload for slice that accepts Columns or Int

2020-07-20 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-32338. --- Fix Version/s: 3.1.0 Assignee: Nikolas Vanderhoof Resolution: Fixed Issue

[jira] [Created] (SPARK-33277) Python/Pandas UDF right after off-heap vectorized reader could cause executor crash.

2020-10-28 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-33277: - Summary: Python/Pandas UDF right after off-heap vectorized reader could cause executor crash. Key: SPARK-33277 URL: https://issues.apache.org/jira/browse/SPARK-33277

[jira] [Resolved] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

2021-06-11 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35475. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-35759) Remove the upperbound for numpy for pandas-on-Spark

2021-06-14 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35759: - Summary: Remove the upperbound for numpy for pandas-on-Spark Key: SPARK-35759 URL: https://issues.apache.org/jira/browse/SPARK-35759 Project: Spark Issue

[jira] [Resolved] (SPARK-35616) Make astype data-type-based

2021-06-14 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35616. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Created] (SPARK-35761) Use type-annotation based pandas_udf or avoid specifying udf types to suppress warnings.

2021-06-14 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35761: - Summary: Use type-annotation based pandas_udf or avoid specifying udf types to suppress warnings. Key: SPARK-35761 URL: https://issues.apache.org/jira/browse/SPARK-35761

[jira] [Resolved] (SPARK-35343) Make the conversion from/to pandas data-type-based for non-ExtensionDtypes

2021-06-07 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35343. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Resolved] (SPARK-35341) Introduce BooleanExtensionOps

2021-06-07 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35341. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Resolved] (SPARK-35474) Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-09 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35474. --- Fix Version/s: 3.2.0 Assignee: Kevin Su Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-35708) Add BaseTest for DataTypeOps

2021-06-18 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35708. --- Fix Version/s: 3.2.0 Assignee: Yikun Jiang Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-35847) Manage InternalField in DataTypeOps.isnull

2021-06-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35847. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Resolved] (SPARK-35849) Make astype data-type-based for DecimalOps

2021-06-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35849. --- Fix Version/s: 3.2.0 Assignee: Yikun Jiang Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-35859) Cleanup type hints.

2021-06-22 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35859: - Summary: Cleanup type hints. Key: SPARK-35859 URL: https://issues.apache.org/jira/browse/SPARK-35859 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

2021-06-23 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368476#comment-17368476 ] Takuya Ueshin commented on SPARK-35476: --- I'm working on this. > Enable disallow_untyped_defs mypy

[jira] [Commented] (SPARK-35473) Enable disallow_untyped_defs mypy check for pyspark.pandas.groupby.

2021-06-21 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17366949#comment-17366949 ] Takuya Ueshin commented on SPARK-35473: --- I'm working on this. > Enable disallow_untyped_defs mypy

[jira] [Resolved] (SPARK-35614) Make the conversion to pandas data-type-based for ExtensionDtypes

2021-06-21 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35614. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Created] (SPARK-35847) Manage InternalField in DataTypeOps.isnull

2021-06-21 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35847: - Summary: Manage InternalField in DataTypeOps.isnull Key: SPARK-35847 URL: https://issues.apache.org/jira/browse/SPARK-35847 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-06-24 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368970#comment-17368970 ] Takuya Ueshin commented on SPARK-35471: --- I'm working on this. > Enable disallow_untyped_defs mypy

[jira] [Resolved] (SPARK-35342) Introduce DecimalOps

2021-06-18 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35342. --- Fix Version/s: 3.2.0 Assignee: Yikun Jiang Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-35478) Enable disallow_untyped_defs mypy check for pyspark.pandas.window.

2021-06-18 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35478. --- Fix Version/s: 3.2.0 Assignee: Kevin Su Resolution: Fixed Issue resolved by

[jira] [Commented] (SPARK-35808) Always enable the `pandas_metadata` in DataFrame.parquet

2021-06-18 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17365816#comment-17365816 ] Takuya Ueshin commented on SPARK-35808: --- I think we shouldn't do this. The reason we introduced

[jira] [Resolved] (SPARK-35808) Always enable the `pandas_metadata` in DataFrame.parquet

2021-06-18 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35808. --- Resolution: Won't Do I'd close this. > Always enable the `pandas_metadata` in

[jira] [Commented] (SPARK-35810) Remove ps.broadcast API

2021-06-18 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17365823#comment-17365823 ] Takuya Ueshin commented on SPARK-35810: --- I think we should deprecate it first and remove later

[jira] [Resolved] (SPARK-35601) Complete arithmetic operators involving bool literals, Series, and Index

2021-06-09 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35601. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Commented] (SPARK-34941) Enable mypy for pandas-on-Spark

2021-05-05 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339868#comment-17339868 ] Takuya Ueshin commented on SPARK-34941: --- I'm going to work on this. > Enable mypy for

[jira] [Resolved] (SPARK-34941) Enable mypy for pandas-on-Spark

2021-05-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-34941. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-35388) Allow the PR source branch to include slashes.

2021-05-12 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35388: - Summary: Allow the PR source branch to include slashes. Key: SPARK-35388 URL: https://issues.apache.org/jira/browse/SPARK-35388 Project: Spark Issue

[jira] [Created] (SPARK-35473) Enable disallow_untyped_defs mypy check for pyspark.pandas.groupby.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35473: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.groupby. Key: SPARK-35473 URL: https://issues.apache.org/jira/browse/SPARK-35473 Project: Spark

[jira] [Created] (SPARK-35472) Enable disallow_untyped_defs mypy check for pyspark.pandas.generic.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35472: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.generic. Key: SPARK-35472 URL: https://issues.apache.org/jira/browse/SPARK-35472 Project: Spark

[jira] [Created] (SPARK-35474) Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35474: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing. Key: SPARK-35474 URL: https://issues.apache.org/jira/browse/SPARK-35474 Project:

[jira] [Created] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35475: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace. Key: SPARK-35475 URL: https://issues.apache.org/jira/browse/SPARK-35475 Project:

[jira] [Created] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35476: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.series. Key: SPARK-35476 URL: https://issues.apache.org/jira/browse/SPARK-35476 Project: Spark

[jira] [Created] (SPARK-35464) pandas APIs on Spark: Enable mypy check "disallow_untyped_defs" for main codes.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35464: - Summary: pandas APIs on Spark: Enable mypy check "disallow_untyped_defs" for main codes. Key: SPARK-35464 URL: https://issues.apache.org/jira/browse/SPARK-35464

[jira] [Created] (SPARK-35467) Enable disallow_untyped_defs mypy check for pyspark.pandas.spark.accessors.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35467: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.spark.accessors. Key: SPARK-35467 URL: https://issues.apache.org/jira/browse/SPARK-35467

[jira] [Created] (SPARK-35469) Enable disallow_untyped_defs mypy check for pyspark.pandas.accessors.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35469: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.accessors. Key: SPARK-35469 URL: https://issues.apache.org/jira/browse/SPARK-35469 Project:

[jira] [Created] (SPARK-35468) Enable disallow_untyped_defs mypy check for pyspark.pandas.typedef.typehints.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35468: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.typedef.typehints. Key: SPARK-35468 URL: https://issues.apache.org/jira/browse/SPARK-35468

[jira] [Created] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35471: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.frame. Key: SPARK-35471 URL: https://issues.apache.org/jira/browse/SPARK-35471 Project: Spark

[jira] [Created] (SPARK-35470) Enable disallow_untyped_defs mypy check for pyspark.pandas.base.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35470: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.base. Key: SPARK-35470 URL: https://issues.apache.org/jira/browse/SPARK-35470 Project: Spark

[jira] [Updated] (SPARK-35466) Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops.

2021-05-20 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-35466: -- Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops. (was:

[jira] [Created] (SPARK-35477) Enable disallow_untyped_defs mypy check for pyspark.pandas.utils.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35477: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.utils. Key: SPARK-35477 URL: https://issues.apache.org/jira/browse/SPARK-35477 Project: Spark

[jira] [Created] (SPARK-35478) Enable disallow_untyped_defs mypy check for pyspark.pandas.window.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35478: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.window. Key: SPARK-35478 URL: https://issues.apache.org/jira/browse/SPARK-35478 Project: Spark

[jira] [Resolved] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-20 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35364. --- Fix Version/s: 3.2.0 Assignee: Haejoon Lee Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-35465) Enable disallow_untyped_defs mypy check except for major modules.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35465: - Summary: Enable disallow_untyped_defs mypy check except for major modules. Key: SPARK-35465 URL: https://issues.apache.org/jira/browse/SPARK-35465 Project: Spark

[jira] [Created] (SPARK-35466) Enable disallow_untyped_defs for pyspark.pandas.data_type_ops.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35466: - Summary: Enable disallow_untyped_defs for pyspark.pandas.data_type_ops. Key: SPARK-35466 URL: https://issues.apache.org/jira/browse/SPARK-35466 Project: Spark

[jira] [Resolved] (SPARK-35338) Separate arithmetic operations into data type based structures

2021-05-19 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35338. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32596

[jira] [Resolved] (SPARK-35338) Separate arithmetic operations into data type based structures

2021-05-19 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35338. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Reopened] (SPARK-35338) Separate arithmetic operations into data type based structures

2021-05-19 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reopened SPARK-35338: --- The PR was reverted at

[jira] [Updated] (SPARK-35338) Separate arithmetic operations into data type based structures

2021-05-19 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-35338: -- Fix Version/s: (was: 3.2.0) > Separate arithmetic operations into data type based

[jira] [Created] (SPARK-35450) Follow checkout-merge way to use the latest commit for linter, or other workflows.

2021-05-19 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35450: - Summary: Follow checkout-merge way to use the latest commit for linter, or other workflows. Key: SPARK-35450 URL: https://issues.apache.org/jira/browse/SPARK-35450

[jira] [Created] (SPARK-35505) Remove APIs that have been deprecated in Koalas.

2021-05-24 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35505: - Summary: Remove APIs that have been deprecated in Koalas. Key: SPARK-35505 URL: https://issues.apache.org/jira/browse/SPARK-35505 Project: Spark Issue

[jira] [Commented] (SPARK-35505) Remove APIs that have been deprecated in Koalas.

2021-05-24 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350692#comment-17350692 ] Takuya Ueshin commented on SPARK-35505: --- I'm working on this. > Remove APIs that have been

[jira] [Resolved] (SPARK-35452) Introduce ArrayOps, MapOps and StructOps

2021-05-26 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35452. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Created] (SPARK-35537) Introduce a util function to check whether the underlying expressions of the columns are the same.

2021-05-26 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35537: - Summary: Introduce a util function to check whether the underlying expressions of the columns are the same. Key: SPARK-35537 URL:

[jira] [Resolved] (SPARK-35465) Enable disallow_untyped_defs mypy check except for major modules.

2021-05-21 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35465. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Updated] (SPARK-35638) Introduce InternalField to manage dtypes and StructFields.

2021-06-04 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-35638: -- Summary: Introduce InternalField to manage dtypes and StructFields. (was: Introduce Field to

[jira] [Resolved] (SPARK-35505) Remove APIs that have been deprecated in Koalas.

2021-05-25 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35505. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Resolved] (SPARK-35522) Introduce BinaryOps for BinaryType

2021-05-26 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35522. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

<    1   2   3   4   5   6   7   8   9   >