[jira] [Created] (SPARK-23722) Add support for inserting empty map or array to table

Sital Kedia created SPARK-23722: --- Summary: Add support for inserting empty map or array to table Key: SPARK-23722 URL: https://issues.apache.org/jira/browse/SPARK-23722 Project: Spark Issue

[jira] [Comment Edited] (SPARK-23713) Clean-up UnsafeWriter classes

[ https://issues.apache.org/jira/browse/SPARK-23713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403246#comment-16403246 ] Kazuaki Ishizaki edited comment on SPARK-23713 at 3/17/18 3:42 AM: --- It

[jira] [Commented] (SPARK-23713) Clean-up UnsafeWriter classes

[ https://issues.apache.org/jira/browse/SPARK-23713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403246#comment-16403246 ] Kazuaki Ishizaki commented on SPARK-23713: -- It will make sense. I will work for this. >

[jira] [Updated] (SPARK-23721) Enhance BlockManagerId to include container's underlying host machine hostname

[ https://issues.apache.org/jira/browse/SPARK-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-23721: Summary: Enhance BlockManagerId to include container's underlying host machine

[jira] [Updated] (SPARK-23721) Enhance BlockManagerId to include container's underlying host machie hostname

[ https://issues.apache.org/jira/browse/SPARK-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-23721: Summary: Enhance BlockManagerId to include container's underlying host machie

[jira] [Updated] (SPARK-23721) Enhance BlockManagerId to include container's underlying host name

[ https://issues.apache.org/jira/browse/SPARK-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-23721: Summary: Enhance BlockManagerId to include container's underlying host name (was:

[jira] [Created] (SPARK-23721) Use actual node's hostname for host and rack locality computation

Mridul Muralidharan created SPARK-23721: --- Summary: Use actual node's hostname for host and rack locality computation Key: SPARK-23721 URL: https://issues.apache.org/jira/browse/SPARK-23721

[jira] [Created] (SPARK-23720) Leverage shuffle service when running in non-host networking mode in hadoop 3 docker support

Mridul Muralidharan created SPARK-23720: --- Summary: Leverage shuffle service when running in non-host networking mode in hadoop 3 docker support Key: SPARK-23720 URL:

[jira] [Updated] (SPARK-23718) Document using docker in host networking mode in hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-23718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-23718: Issue Type: Documentation (was: Task) > Document using docker in host networking

[jira] [Created] (SPARK-23719) Use correct hostname in non-host networking mode in hadoop 3 docker support

Mridul Muralidharan created SPARK-23719: --- Summary: Use correct hostname in non-host networking mode in hadoop 3 docker support Key: SPARK-23719 URL: https://issues.apache.org/jira/browse/SPARK-23719

[jira] [Updated] (SPARK-23718) Document using docker in host networking mode in hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-23718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-23718: Issue Type: Improvement (was: Documentation) > Document using docker in host

[jira] [Updated] (SPARK-23718) Document using docker in host networking mode in hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-23718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-23718: Issue Type: Task (was: Improvement) > Document using docker in host networking

[jira] [Created] (SPARK-23718) Document using docker in host networking mode in hadoop 3

Mridul Muralidharan created SPARK-23718: --- Summary: Document using docker in host networking mode in hadoop 3 Key: SPARK-23718 URL: https://issues.apache.org/jira/browse/SPARK-23718 Project:

[jira] [Created] (SPARK-23717) Leverage docker support in Hadoop 3

Mridul Muralidharan created SPARK-23717: --- Summary: Leverage docker support in Hadoop 3 Key: SPARK-23717 URL: https://issues.apache.org/jira/browse/SPARK-23717 Project: Spark Issue

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403186#comment-16403186 ] Dongjoon Hyun commented on SPARK-23710: --- [~q79969786]. As I commented on your PR, you cannot remove

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403183#comment-16403183 ] Bruce Robbins commented on SPARK-23715: --- It almost seems like FromUTCTimestamp needs its own Cast

[jira] [Updated] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-5498: - Fix Version/s: (was: 3.0.0) > [SPARK-SQL]when the partition schema does not match table

[jira] [Closed] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-5498. > [SPARK-SQL]when the partition schema does not match table schema,it throws >

[jira] [Resolved] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-5498. -- Resolution: Fixed > [SPARK-SQL]when the partition schema does not match table schema,it throws

[jira] [Updated] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-5498: - Affects Version/s: (was: 2.2.0) > [SPARK-SQL]when the partition schema does not match table

[jira] [Commented] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403176#comment-16403176 ] Dongjoon Hyun commented on SPARK-5498: -- And, SPARK-5498 has resolved 3 years ago. Please open a new

[jira] [Commented] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403175#comment-16403175 ] Dongjoon Hyun commented on SPARK-5498: -- [~liutang123]. Spark should not do this kind of risky thing.

[jira] [Comment Edited] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402117#comment-16402117 ] Yuming Wang edited comment on SPARK-23710 at 3/17/18 12:37 AM: --- Hive

[jira] [Created] (SPARK-23716) Change SHA512 style in release artifacts to play nicely with shasum utility

Nicholas Chammas created SPARK-23716: Summary: Change SHA512 style in release artifacts to play nicely with shasum utility Key: SPARK-23716 URL: https://issues.apache.org/jira/browse/SPARK-23716

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Created] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

Bruce Robbins created SPARK-23715: - Summary: from_utc_timestamp returns incorrect results for some UTC date/time values Key: SPARK-23715 URL: https://issues.apache.org/jira/browse/SPARK-23715

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Commented] (SPARK-23714) Add metrics for cached KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-23714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403112#comment-16403112 ] Ted Yu commented on SPARK-23714: [~tdas] made the following suggestion: apache commons pool does expose

[jira] [Commented] (SPARK-23623) Avoid concurrent use of cached KafkaConsumer in CachedKafkaConsumer (kafka-0-10-sql)

[ https://issues.apache.org/jira/browse/SPARK-23623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403100#comment-16403100 ] Apache Spark commented on SPARK-23623: -- User 'tdas' has created a pull request for this issue:

[jira] [Commented] (SPARK-16087) Spark Hangs When Using Union With Persisted Hadoop RDD

[ https://issues.apache.org/jira/browse/SPARK-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403098#comment-16403098 ] Kevin Conaway commented on SPARK-16087: --- [~srowen] I finally got around to digging in to this after

[jira] [Resolved] (SPARK-23683) FileCommitProtocol.instantiate to require 3-arg constructor for dynamic partition overwrite

[ https://issues.apache.org/jira/browse/SPARK-23683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23683. - Resolution: Fixed Assignee: Steve Loughran Fix Version/s: 2.4.0 >

[jira] [Commented] (SPARK-23646) pyspark DataFrameWriter ignores customized settings?

[ https://issues.apache.org/jira/browse/SPARK-23646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402653#comment-16402653 ] Chuan-Heng Hsiao commented on SPARK-23646: -- hi [~hyukjin.kwon],   Sorry to bother you again.  

[jira] [Comment Edited] (SPARK-5243) Spark will hang if (driver memory + executor memory) exceeds limit on a 1-worker cluster

[ https://issues.apache.org/jira/browse/SPARK-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402398#comment-16402398 ] Meenakshi sundaram sekar edited comment on SPARK-5243 at 3/16/18 7:47 PM:

[jira] [Commented] (SPARK-5243) Spark will hang if (driver memory + executor memory) exceeds limit on a 1-worker cluster

[ https://issues.apache.org/jira/browse/SPARK-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402398#comment-16402398 ] Meenakshi sundaram sekar commented on SPARK-5243: - Hi - i faced this issue when i had one

[jira] [Commented] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402369#comment-16402369 ] Matthew Porter commented on SPARK-5928: --- Hitting this issue in production as well and it can be very

[jira] [Resolved] (SPARK-15009) PySpark CountVectorizerModel should be able to construct from vocabulary list

[ https://issues.apache.org/jira/browse/SPARK-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-15009. - Resolution: Fixed Fix Version/s: 2.4.0 Target Version/s: 2.4.0 Thanks for fixing this

[jira] [Assigned] (SPARK-15009) PySpark CountVectorizerModel should be able to construct from vocabulary list

[ https://issues.apache.org/jira/browse/SPARK-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-15009: --- Assignee: Bryan Cutler > PySpark CountVectorizerModel should be able to construct from vocabulary

[jira] [Created] (SPARK-23714) Add metrics for cached KafkaConsumer

Ted Yu created SPARK-23714: -- Summary: Add metrics for cached KafkaConsumer Key: SPARK-23714 URL: https://issues.apache.org/jira/browse/SPARK-23714 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23646) pyspark DataFrameWriter ignores customized settings?

[ https://issues.apache.org/jira/browse/SPARK-23646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402300#comment-16402300 ] Chuan-Heng Hsiao commented on SPARK-23646: -- hi [~hyukjin.kwon],   I posted to the dev mailing

[jira] [Resolved] (SPARK-23623) Avoid concurrent use of cached KafkaConsumer in CachedKafkaConsumer (kafka-0-10-sql)

[ https://issues.apache.org/jira/browse/SPARK-23623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23623. -- Resolution: Fixed Fix Version/s: 2.4.0 > Avoid concurrent use of cached KafkaConsumer

[jira] [Commented] (SPARK-22371) dag-scheduler-event-loop thread stopped with error Attempted to access garbage collected accumulator 5605982

[ https://issues.apache.org/jira/browse/SPARK-22371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402271#comment-16402271 ] Ben Doerr commented on SPARK-22371: --- We've seen this for the first time on 2.3.0.  >

[jira] [Commented] (SPARK-23680) entrypoint.sh does not accept arbitrary UIDs, returning as an error

[ https://issues.apache.org/jira/browse/SPARK-23680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402266#comment-16402266 ] Erik Erlandson commented on SPARK-23680: commit workflow indicates to set the Assignee, however I

[jira] [Created] (SPARK-23713) Clean-up UnsafeWriter classes

Herman van Hovell created SPARK-23713: - Summary: Clean-up UnsafeWriter classes Key: SPARK-23713 URL: https://issues.apache.org/jira/browse/SPARK-23713 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-23680) entrypoint.sh does not accept arbitrary UIDs, returning as an error

[ https://issues.apache.org/jira/browse/SPARK-23680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson resolved SPARK-23680. Resolution: Fixed Target Version/s: 2.3.1, 2.4.0 merged to master >

[jira] [Created] (SPARK-23712) Investigate replacing Code Generated UnsafeRowJoiner with an Interpreted version

Herman van Hovell created SPARK-23712: - Summary: Investigate replacing Code Generated UnsafeRowJoiner with an Interpreted version Key: SPARK-23712 URL: https://issues.apache.org/jira/browse/SPARK-23712

[jira] [Resolved] (SPARK-18371) Spark Streaming backpressure bug - generates a batch with large number of records

[ https://issues.apache.org/jira/browse/SPARK-18371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger resolved SPARK-18371. Resolution: Fixed Fix Version/s: 2.4.0 > Spark Streaming backpressure bug -

[jira] [Assigned] (SPARK-18371) Spark Streaming backpressure bug - generates a batch with large number of records

[ https://issues.apache.org/jira/browse/SPARK-18371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger reassigned SPARK-18371: -- Assignee: Sebastian Arzt > Spark Streaming backpressure bug - generates a batch with

[jira] [Commented] (SPARK-14652) pyspark streaming driver unable to cleanup metadata for cached RDDs leading to driver OOM

[ https://issues.apache.org/jira/browse/SPARK-14652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402230#comment-16402230 ] Zoltán Zvara commented on SPARK-14652: -- Reproduced the problem in Spark 2.3.0: * In one online ML

[jira] [Created] (SPARK-23711) Add fallback to interpreted execution logic

Herman van Hovell created SPARK-23711: - Summary: Add fallback to interpreted execution logic Key: SPARK-23711 URL: https://issues.apache.org/jira/browse/SPARK-23711 Project: Spark Issue

[jira] [Resolved] (SPARK-23581) Add an interpreted version of GenerateUnsafeProjection

[ https://issues.apache.org/jira/browse/SPARK-23581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23581. --- Resolution: Fixed Fix Version/s: 2.4.0 > Add an interpreted version of

[jira] [Commented] (SPARK-23677) Selecting columns from joined DataFrames with the same origin yields wrong results

[ https://issues.apache.org/jira/browse/SPARK-23677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402225#comment-16402225 ] Martin Mauch commented on SPARK-23677: -- Or rather https://issues.apache.org/jira/browse/SPARK-10892

[jira] [Resolved] (SPARK-23677) Selecting columns from joined DataFrames with the same origin yields wrong results

[ https://issues.apache.org/jira/browse/SPARK-23677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Mauch resolved SPARK-23677. -- Resolution: Duplicate > Selecting columns from joined DataFrames with the same origin yields

[jira] [Commented] (SPARK-23677) Selecting columns from joined DataFrames with the same origin yields wrong results

[ https://issues.apache.org/jira/browse/SPARK-23677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402221#comment-16402221 ] Martin Mauch commented on SPARK-23677: -- [~maropu] right, it's that one. I did search for it before,

[jira] [Commented] (SPARK-9597) Add Spark Streaming + MQTT Integration Guide

[ https://issues.apache.org/jira/browse/SPARK-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402172#comment-16402172 ] Patrick Alwell commented on SPARK-9597: --- Looks like the library moved to Apache Bahir: 

[jira] [Commented] (SPARK-9597) Add Spark Streaming + MQTT Integration Guide

[ https://issues.apache.org/jira/browse/SPARK-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402175#comment-16402175 ] Sean Owen commented on SPARK-9597: -- Correct, it was removed for lack of demand or people to work on

[jira] [Commented] (SPARK-9597) Add Spark Streaming + MQTT Integration Guide

[ https://issues.apache.org/jira/browse/SPARK-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402167#comment-16402167 ] Patrick Alwell commented on SPARK-9597: --- [~srowen] Why was the library removed from Spark 2.X? Did

[jira] [Resolved] (SPARK-23553) Tests should not assume the default value of `spark.sql.sources.default`

[ https://issues.apache.org/jira/browse/SPARK-23553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23553. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.4.0 2.3.1

[jira] [Commented] (SPARK-22183) Inconsistency in LIKE escaping between literal values and column-based ones

[ https://issues.apache.org/jira/browse/SPARK-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402152#comment-16402152 ] Adrien Lavoillotte commented on SPARK-22183: [~michal.db] it would indeed in the future. We

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Commented] (SPARK-22183) Inconsistency in LIKE escaping between literal values and column-based ones

[ https://issues.apache.org/jira/browse/SPARK-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402122#comment-16402122 ] Michal Szafranski commented on SPARK-22183: --- [~instanceof me] Would it work for your use case

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402117#comment-16402117 ] Yuming Wang commented on SPARK-23710: - Hive materialized view may be supported after upgrading to

[jira] [Commented] (SPARK-22183) Inconsistency in LIKE escaping between literal values and column-based ones

[ https://issues.apache.org/jira/browse/SPARK-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402116#comment-16402116 ] Adrien Lavoillotte commented on SPARK-22183: Yes, thanks for the suggestion but our software

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402110#comment-16402110 ] Yuming Wang commented on SPARK-23710: - We don't need to consider Kryo's problem because {{Hive

[jira] [Created] (SPARK-23710) Upgrade Hive to 2.3.2

Yuming Wang created SPARK-23710: --- Summary: Upgrade Hive to 2.3.2 Key: SPARK-23710 URL: https://issues.apache.org/jira/browse/SPARK-23710 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23638) Spark on k8s: spark.kubernetes.initContainer.image has no effect

[ https://issues.apache.org/jira/browse/SPARK-23638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402070#comment-16402070 ] Yinan Li commented on SPARK-23638: -- The Kubernetes-specific submission client will only add an

[jira] [Comment Edited] (SPARK-23255) Add user guide and examples for DataFrame image reading functions

[ https://issues.apache.org/jira/browse/SPARK-23255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402052#comment-16402052 ] Wesley Hoffman edited comment on SPARK-23255 at 3/16/18 3:31 PM: - Formal

[jira] [Commented] (SPARK-23255) Add user guide and examples for DataFrame image reading functions

[ https://issues.apache.org/jira/browse/SPARK-23255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402052#comment-16402052 ] Wesley Hoffman commented on SPARK-23255: Formal documentation for this feature would be wonderful

[jira] [Comment Edited] (SPARK-22183) Inconsistency in LIKE escaping between literal values and column-based ones

[ https://issues.apache.org/jira/browse/SPARK-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402042#comment-16402042 ] Michal Szafranski edited comment on SPARK-22183 at 3/16/18 3:20 PM:

[jira] [Commented] (SPARK-22183) Inconsistency in LIKE escaping between literal values and column-based ones

[ https://issues.apache.org/jira/browse/SPARK-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402042#comment-16402042 ] Michal Szafranski commented on SPARK-22183: --- As for the reporters use case, using explicit

[jira] [Commented] (SPARK-23696) StructType.fromString swallows exceptions from DataType.fromJson

[ https://issues.apache.org/jira/browse/SPARK-23696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401979#comment-16401979 ] Simeon H.K. Fitch commented on SPARK-23696: --- Not sure what you mean by "we", but I'm just using

[jira] [Created] (SPARK-23709) BaggedPoint.convertToBaggedRDDSamplingWithReplacement does not guarantee the sum of weights

Olivier Sannier created SPARK-23709: --- Summary: BaggedPoint.convertToBaggedRDDSamplingWithReplacement does not guarantee the sum of weights Key: SPARK-23709 URL: https://issues.apache.org/jira/browse/SPARK-23709

[jira] [Commented] (SPARK-21962) Distributed Tracing in Spark

[ https://issues.apache.org/jira/browse/SPARK-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401791#comment-16401791 ] Steve Loughran commented on SPARK-21962: HTrace instrumentation would be the strategy to look at

[jira] [Commented] (SPARK-23644) SHS with proxy doesn't show applications

[ https://issues.apache.org/jira/browse/SPARK-23644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401782#comment-16401782 ] Apache Spark commented on SPARK-23644: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-22183) Inconsistency in LIKE escaping between literal values and column-based ones

[ https://issues.apache.org/jira/browse/SPARK-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401738#comment-16401738 ] Michal Szafranski edited comment on SPARK-22183 at 3/16/18 11:02 AM: -

[jira] [Commented] (SPARK-22183) Inconsistency in LIKE escaping between literal values and column-based ones

[ https://issues.apache.org/jira/browse/SPARK-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401738#comment-16401738 ] Michal Szafranski commented on SPARK-22183: --- This seems to be an issue with

[jira] [Commented] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401717#comment-16401717 ] Apache Spark commented on SPARK-5498: - User 'liutang123' has created a pull request for this issue:

[jira] [Assigned] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5498: --- Assignee: jeanlyn (was: Apache Spark) > [SPARK-SQL]when the partition schema does not match

[jira] [Assigned] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5498: --- Assignee: Apache Spark (was: jeanlyn) > [SPARK-SQL]when the partition schema does not match

[jira] [Comment Edited] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401701#comment-16401701 ] Lijia Liu edited comment on SPARK-5498 at 3/16/18 10:10 AM: In our cluster, we

[jira] [Reopened] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijia Liu reopened SPARK-5498: -- In our cluster, we use hive 1.2, spark 2.2, hadoop 2.7. When we read hive table use spark, we will get the

[jira] [Updated] (SPARK-5498) [SPARK-SQL]when the partition schema does not match table schema,it throws java.lang.ClassCastException and so on

[ https://issues.apache.org/jira/browse/SPARK-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijia Liu updated SPARK-5498: - Affects Version/s: 2.2.0 Fix Version/s: 3.0.0 > [SPARK-SQL]when the partition schema does not

[jira] [Commented] (SPARK-23665) Add adaptive algorithm to select query result collect method

[ https://issues.apache.org/jira/browse/SPARK-23665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401677#comment-16401677 ] Takeshi Yamamuro commented on SPARK-23665: -- I feel most database users explicitly use cursors

[jira] [Updated] (SPARK-23708) Comment of ShutdownHookManager.addShutdownHook is error

[ https://issues.apache.org/jira/browse/SPARK-23708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-23708: - Issue Type: Improvement (was: Bug) > Comment of ShutdownHookManager.addShutdownHook is error >

[jira] [Commented] (SPARK-23708) Comment of ShutdownHookManager.addShutdownHook is error

[ https://issues.apache.org/jira/browse/SPARK-23708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401673#comment-16401673 ] Apache Spark commented on SPARK-23708: -- User 'caneGuy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23708) Comment of ShutdownHookManager.addShutdownHook is error

[ https://issues.apache.org/jira/browse/SPARK-23708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23708: Assignee: Apache Spark > Comment of ShutdownHookManager.addShutdownHook is error >

[jira] [Updated] (SPARK-23708) Comment of ShutdownHookManager.addShutdownHook is error

[ https://issues.apache.org/jira/browse/SPARK-23708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-23708: - Issue Type: Bug (was: Improvement) > Comment of ShutdownHookManager.addShutdownHook is error >

[jira] [Assigned] (SPARK-23708) Comment of ShutdownHookManager.addShutdownHook is error

[ https://issues.apache.org/jira/browse/SPARK-23708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23708: Assignee: (was: Apache Spark) > Comment of ShutdownHookManager.addShutdownHook is

[jira] [Created] (SPARK-23708) Comment of ShutdownHookManager.addShutdownHook is error

zhoukang created SPARK-23708: Summary: Comment of ShutdownHookManager.addShutdownHook is error Key: SPARK-23708 URL: https://issues.apache.org/jira/browse/SPARK-23708 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23662) Support selective tests in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-23662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-23662. -- Resolution: Not A Problem > Support selective tests in SQLQueryTestSuite >

[jira] [Updated] (SPARK-23697) Accumulators of Spark 1.x no longer work with Spark 2.x

[ https://issues.apache.org/jira/browse/SPARK-23697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-23697: --- Affects Version/s: 2.3.0 > Accumulators of Spark 1.x no longer work with Spark 2.x >

[jira] [Resolved] (SPARK-23635) Spark executor env variable is overwritten by same name AM env variable

[ https://issues.apache.org/jira/browse/SPARK-23635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23635. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20799

[jira] [Assigned] (SPARK-23635) Spark executor env variable is overwritten by same name AM env variable

[ https://issues.apache.org/jira/browse/SPARK-23635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23635: --- Assignee: Saisai Shao > Spark executor env variable is overwritten by same name AM env

[jira] [Assigned] (SPARK-23707) Fresh 'initRange' name to avoid method name conflicts

[ https://issues.apache.org/jira/browse/SPARK-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23707: Assignee: Apache Spark > Fresh 'initRange' name to avoid method name conflicts >

[jira] [Assigned] (SPARK-23707) Fresh 'initRange' name to avoid method name conflicts

[ https://issues.apache.org/jira/browse/SPARK-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23707: Assignee: (was: Apache Spark) > Fresh 'initRange' name to avoid method name conflicts

[jira] [Commented] (SPARK-23707) Fresh 'initRange' name to avoid method name conflicts

[ https://issues.apache.org/jira/browse/SPARK-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401575#comment-16401575 ] Apache Spark commented on SPARK-23707: -- User 'ConeyLiu' has created a pull request for this issue:

[jira] [Created] (SPARK-23707) Fresh 'initRange' name to avoid method name conflicts

Xianyang Liu created SPARK-23707: Summary: Fresh 'initRange' name to avoid method name conflicts Key: SPARK-23707 URL: https://issues.apache.org/jira/browse/SPARK-23707 Project: Spark Issue

[jira] [Updated] (SPARK-23697) Accumulators of Spark 1.x no longer work with Spark 2.x

[ https://issues.apache.org/jira/browse/SPARK-23697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-23697: --- Description: I've noticed that accumulators of Spark 1.x no longer work with Spark

  1   2   >