[jira] [Updated] (SPARK-26577) Add input optimizer when reading Hive table by SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-26577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deegue updated SPARK-26577: --- Description: When using SparkSQL, for example the ThriftServer, if we set

[jira] [Commented] (SPARK-26565) modify dev/create-release/release-build.sh to let jenkins build packages w/o publishing

[ https://issues.apache.org/jira/browse/SPARK-26565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742845#comment-16742845 ] Felix Cheung commented on SPARK-26565: -- ok AFAIK the first `cannot stat` is from this line  

[jira] [Assigned] (SPARK-26620) DataFrameReader.json and csv in Python should accept DataFrame.

[ https://issues.apache.org/jira/browse/SPARK-26620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26620: Assignee: (was: Apache Spark) > DataFrameReader.json and csv in Python should accept

[jira] [Assigned] (SPARK-26620) DataFrameReader.json and csv in Python should accept DataFrame.

[ https://issues.apache.org/jira/browse/SPARK-26620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26620: Assignee: Apache Spark > DataFrameReader.json and csv in Python should accept DataFrame.

[jira] [Created] (SPARK-26620) DataFrameReader.json and csv in Python should accept DataFrame.

Takuya Ueshin created SPARK-26620: - Summary: DataFrameReader.json and csv in Python should accept DataFrame. Key: SPARK-26620 URL: https://issues.apache.org/jira/browse/SPARK-26620 Project: Spark

[jira] [Commented] (SPARK-25935) Prevent null rows from JSON parser

[ https://issues.apache.org/jira/browse/SPARK-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742741#comment-16742741 ] Wenchen Fan commented on SPARK-25935: - It has been reverted in

[jira] [Resolved] (SPARK-25935) Prevent null rows from JSON parser

[ https://issues.apache.org/jira/browse/SPARK-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25935. - Resolution: Won't Fix > Prevent null rows from JSON parser > --

[jira] [Reopened] (SPARK-25935) Prevent null rows from JSON parser

[ https://issues.apache.org/jira/browse/SPARK-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-25935: - Assignee: (was: Maxim Gekk) > Prevent null rows from JSON parser >

[jira] [Commented] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742735#comment-16742735 ] Hyukjin Kwon commented on SPARK-26611: -- Which OS do you use? can you try to set an environment

[jira] [Assigned] (SPARK-23153) Support application dependencies in submission client's local file system

[ https://issues.apache.org/jira/browse/SPARK-23153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23153: Assignee: Apache Spark > Support application dependencies in submission client's local

[jira] [Assigned] (SPARK-23153) Support application dependencies in submission client's local file system

[ https://issues.apache.org/jira/browse/SPARK-23153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23153: Assignee: (was: Apache Spark) > Support application dependencies in submission

[jira] [Updated] (SPARK-26530) Validate heartheat arguments in HeartbeatReceiver

[ https://issues.apache.org/jira/browse/SPARK-26530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26530: - Summary: Validate heartheat arguments in HeartbeatReceiver (was: Validate heartheat arguments

[jira] [Updated] (SPARK-26619) Prune the unused serializers from `SerializeFromObject`

[ https://issues.apache.org/jira/browse/SPARK-26619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26619: Affects Version/s: (was: 2.4.0) 3.0.0 > Prune the unused serializers from

[jira] [Updated] (SPARK-25603) `Projection` expression pushdown through `coalesce` and `limit`

[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25603: Affects Version/s: (was: 2.4.1) 3.0.0 > `Projection` expression pushdown

[jira] [Created] (SPARK-26619) Prune the unused serializers from `SerializeFromObject`

DB Tsai created SPARK-26619: --- Summary: Prune the unused serializers from `SerializeFromObject` Key: SPARK-26619 URL: https://issues.apache.org/jira/browse/SPARK-26619 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-25603) `Projection` expression pushdown through `coalesce` and `limit`

[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25603: Assignee: Apache Spark (was: DB Tsai) > `Projection` expression pushdown through

[jira] [Assigned] (SPARK-25603) `Projection` expression pushdown through `coalesce` and `limit`

[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25603: Assignee: DB Tsai (was: Apache Spark) > `Projection` expression pushdown through

[jira] [Commented] (SPARK-26257) SPIP: Interop Support for Spark Language Extensions

[ https://issues.apache.org/jira/browse/SPARK-26257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742519#comment-16742519 ] Alessandro Bellina commented on SPARK-26257: [~tcondie] I think you should share the design

[jira] [Commented] (SPARK-26591) Scalar Pandas UDF fails with 'illegal hardware instruction' in a certain environment

[ https://issues.apache.org/jira/browse/SPARK-26591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742517#comment-16742517 ] Bryan Cutler commented on SPARK-26591: -- I created the same virtual environment and could not

[jira] [Assigned] (SPARK-26618) Make typed Timestamp/Date literals consistent to casting

[ https://issues.apache.org/jira/browse/SPARK-26618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26618: Assignee: Apache Spark > Make typed Timestamp/Date literals consistent to casting >

[jira] [Resolved] (SPARK-26350) Allow the user to override the group id of the Kafka's consumer

[ https://issues.apache.org/jira/browse/SPARK-26350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-26350. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 3.0.0 > Allow the

[jira] [Assigned] (SPARK-26618) Make typed Timestamp/Date literals consistent to casting

[ https://issues.apache.org/jira/browse/SPARK-26618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26618: Assignee: (was: Apache Spark) > Make typed Timestamp/Date literals consistent to

[jira] [Created] (SPARK-26618) Make typed Timestamp/Date literals consistent to casting

Maxim Gekk created SPARK-26618: -- Summary: Make typed Timestamp/Date literals consistent to casting Key: SPARK-26618 URL: https://issues.apache.org/jira/browse/SPARK-26618 Project: Spark Issue

[jira] [Commented] (SPARK-26608) Remove Jenkins jobs for `branch-2.2`

[ https://issues.apache.org/jira/browse/SPARK-26608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742447#comment-16742447 ] shane knapp commented on SPARK-26608: - the four jobs you listed are currently disabled: !Screen

[jira] [Updated] (SPARK-26608) Remove Jenkins jobs for `branch-2.2`

[ https://issues.apache.org/jira/browse/SPARK-26608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp updated SPARK-26608: Attachment: Screen Shot 2019-01-14 at 11.42.46 AM.png > Remove Jenkins jobs for `branch-2.2` >

[jira] [Commented] (SPARK-26608) Remove Jenkins jobs for `branch-2.2`

[ https://issues.apache.org/jira/browse/SPARK-26608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742443#comment-16742443 ] shane knapp commented on SPARK-26608: - do we want to nix ALL of the 2.2 jobs, or just these?

[jira] [Updated] (SPARK-26608) Remove Jenkins jobs for `branch-2.2`

[ https://issues.apache.org/jira/browse/SPARK-26608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp updated SPARK-26608: Attachment: Screen Shot 2019-01-14 at 11.39.05 AM.png > Remove Jenkins jobs for `branch-2.2` >

[jira] [Assigned] (SPARK-26615) Fixing transport server/client resource leaks in the core unittests

[ https://issues.apache.org/jira/browse/SPARK-26615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26615: Assignee: (was: Apache Spark) > Fixing transport server/client resource leaks in the

[jira] [Assigned] (SPARK-26615) Fixing transport server/client resource leaks in the core unittests

[ https://issues.apache.org/jira/browse/SPARK-26615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26615: Assignee: Apache Spark > Fixing transport server/client resource leaks in the core

[jira] [Assigned] (SPARK-26617) CacheManager blocks during requery

[ https://issues.apache.org/jira/browse/SPARK-26617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26617: Assignee: (was: Apache Spark) > CacheManager blocks during requery >

[jira] [Assigned] (SPARK-26617) CacheManager blocks during requery

[ https://issues.apache.org/jira/browse/SPARK-26617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26617: Assignee: Apache Spark > CacheManager blocks during requery >

[jira] [Commented] (SPARK-26565) modify dev/create-release/release-build.sh to let jenkins build packages w/o publishing

[ https://issues.apache.org/jira/browse/SPARK-26565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742361#comment-16742361 ] shane knapp commented on SPARK-26565: - [~felixcheung] sounds great. my goal here is to re-create

[jira] [Resolved] (SPARK-26613) Add another rename table grammar for spark sql

[ https://issues.apache.org/jira/browse/SPARK-26613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26613. --- Resolution: Won't Fix > Add another rename table grammar for spark sql >

[jira] [Created] (SPARK-26617) CacheManager blocks during requery

Dave DeCaprio created SPARK-26617: - Summary: CacheManager blocks during requery Key: SPARK-26617 URL: https://issues.apache.org/jira/browse/SPARK-26617 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-26569) Fixed point for batch Operator Optimizations never reached when optimize logicalPlan

[ https://issues.apache.org/jira/browse/SPARK-26569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742305#comment-16742305 ] Marco Gaido commented on SPARK-26569: - [~chenfan] may you please try a more recent version of Spark?

[jira] [Commented] (SPARK-26615) Fixing transport server/client resource leaks in the core unittests

[ https://issues.apache.org/jira/browse/SPARK-26615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742243#comment-16742243 ] Attila Zsolt Piros commented on SPARK-26615: I am working on it (soon a PR will be created)

[jira] [Updated] (SPARK-26615) Fixing transport server/client resource leaks in the core unittests

[ https://issues.apache.org/jira/browse/SPARK-26615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-26615: --- Affects Version/s: 3.0.0 > Fixing transport server/client resource leaks in the

[jira] [Created] (SPARK-26616) Expose document frequency in IDFModel

Jatin Puri created SPARK-26616: -- Summary: Expose document frequency in IDFModel Key: SPARK-26616 URL: https://issues.apache.org/jira/browse/SPARK-26616 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-26615) Fixing transport server/client resource leaks in the core unittests

Attila Zsolt Piros created SPARK-26615: -- Summary: Fixing transport server/client resource leaks in the core unittests Key: SPARK-26615 URL: https://issues.apache.org/jira/browse/SPARK-26615

[jira] [Commented] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742140#comment-16742140 ] Alberto commented on SPARK-26611: - Updating pyarrow to version 0.9 seem to be soving the problem >

[jira] [Updated] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto updated SPARK-26611: Description: The following snippet crashes with error: org.apache.spark.SparkException: Python worker

[jira] [Commented] (SPARK-23153) Support application dependencies in submission client's local file system

[ https://issues.apache.org/jira/browse/SPARK-23153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742096#comment-16742096 ] Stavros Kontopoulos commented on SPARK-23153: - I will have a PR shortly regarding hadoop

[jira] [Resolved] (SPARK-26456) Cast date/timestamp by Date/TimestampFormatter

[ https://issues.apache.org/jira/browse/SPARK-26456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26456. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23391

[jira] [Assigned] (SPARK-26456) Cast date/timestamp by Date/TimestampFormatter

[ https://issues.apache.org/jira/browse/SPARK-26456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26456: --- Assignee: Maxim Gekk > Cast date/timestamp by Date/TimestampFormatter >

[jira] [Updated] (SPARK-26591) Scalar Pandas UDF fails with 'illegal hardware instruction' in a certain environment

[ https://issues.apache.org/jira/browse/SPARK-26591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elchin updated SPARK-26591: --- Description: When I try to use pandas_udf from examples in

[jira] [Assigned] (SPARK-26614) Speculation kill might cause job failure

[ https://issues.apache.org/jira/browse/SPARK-26614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26614: Assignee: (was: Apache Spark) > Speculation kill might cause job failure >

[jira] [Assigned] (SPARK-26614) Speculation kill might cause job failure

[ https://issues.apache.org/jira/browse/SPARK-26614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26614: Assignee: Apache Spark > Speculation kill might cause job failure >

[jira] [Assigned] (SPARK-26612) Speculation kill causing finished stage recomputed

[ https://issues.apache.org/jira/browse/SPARK-26612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26612: Assignee: Apache Spark > Speculation kill causing finished stage recomputed >

[jira] [Assigned] (SPARK-26612) Speculation kill causing finished stage recomputed

[ https://issues.apache.org/jira/browse/SPARK-26612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26612: Assignee: (was: Apache Spark) > Speculation kill causing finished stage recomputed >

[jira] [Assigned] (SPARK-26613) Add another rename table grammar for spark sql

[ https://issues.apache.org/jira/browse/SPARK-26613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26613: Assignee: Apache Spark > Add another rename table grammar for spark sql >

[jira] [Created] (SPARK-26614) Speculation kill might cause job failure

liupengcheng created SPARK-26614: Summary: Speculation kill might cause job failure Key: SPARK-26614 URL: https://issues.apache.org/jira/browse/SPARK-26614 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-26613) Add another rename table grammar for spark sql

[ https://issues.apache.org/jira/browse/SPARK-26613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26613: Assignee: (was: Apache Spark) > Add another rename table grammar for spark sql >

[jira] [Created] (SPARK-26612) Speculation kill causing finished stage recomputed

liupengcheng created SPARK-26612: Summary: Speculation kill causing finished stage recomputed Key: SPARK-26612 URL: https://issues.apache.org/jira/browse/SPARK-26612 Project: Spark Issue

[jira] [Updated] (SPARK-26614) Speculation kill might cause job failure

[ https://issues.apache.org/jira/browse/SPARK-26614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26614: - Description: This issue is similar to SPARK-26612 Some odd exceptions might be thrown in

[jira] [Created] (SPARK-26613) Add another rename table grammar for spark sql

jiaan.geng created SPARK-26613: -- Summary: Add another rename table grammar for spark sql Key: SPARK-26613 URL: https://issues.apache.org/jira/browse/SPARK-26613 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto updated SPARK-26611: Description: The following snippet crashes with error: org.apache.spark.SparkException: Python worker

[jira] [Updated] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto updated SPARK-26611: Description: The following snippet crashes with error: org.apache.spark.SparkException: Python worker

[jira] [Updated] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto updated SPARK-26611: Component/s: (was: SQL) > GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly" >

[jira] [Updated] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto updated SPARK-26611: Description: The following snippet crashes with error: org.apache.spark.SparkException: Python worker

[jira] [Updated] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto updated SPARK-26611: Description: The following snippet crashes with error: org.apache.spark.SparkException: Python worker

[jira] [Updated] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

[ https://issues.apache.org/jira/browse/SPARK-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto updated SPARK-26611: Description: The following snippet crashes with error: org.apache.spark.SparkException: Python worker

[jira] [Created] (SPARK-26611) GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly"

Alberto created SPARK-26611: --- Summary: GROUPED_MAP pandas_udf crashing "Python worker exited unexpectedly" Key: SPARK-26611 URL: https://issues.apache.org/jira/browse/SPARK-26611 Project: Spark