[jira] [Commented] (SPARK-26093) Read Avro: ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource

[ https://issues.apache.org/jira/browse/SPARK-26093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690413#comment-16690413 ] Hyukjin Kwon commented on SPARK-26093: -- Please follow

[jira] [Resolved] (SPARK-26093) Read Avro: ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource

[ https://issues.apache.org/jira/browse/SPARK-26093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26093. -- Resolution: Not A Problem > Read Avro: ClassNotFoundException: >

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690399#comment-16690399 ] Hyukjin Kwon commented on SPARK-26019: -- It should be great if we can know what condition and code

[jira] [Assigned] (SPARK-26097) Show partitioning details in DAG UI

[ https://issues.apache.org/jira/browse/SPARK-26097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26097: Assignee: (was: Apache Spark) > Show partitioning details in DAG UI >

[jira] [Commented] (SPARK-26097) Show partitioning details in DAG UI

[ https://issues.apache.org/jira/browse/SPARK-26097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690382#comment-16690382 ] Apache Spark commented on SPARK-26097: -- User 'idanz' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26097) Show partitioning details in DAG UI

[ https://issues.apache.org/jira/browse/SPARK-26097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26097: Assignee: Apache Spark > Show partitioning details in DAG UI >

[jira] [Updated] (SPARK-26097) Show partitioning details in DAG UI

[ https://issues.apache.org/jira/browse/SPARK-26097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Idan Zalzberg updated SPARK-26097: -- Attachment: image (8).png > Show partitioning details in DAG UI >

[jira] [Created] (SPARK-26097) Show partitioning details in DAG UI

Idan Zalzberg created SPARK-26097: - Summary: Show partitioning details in DAG UI Key: SPARK-26097 URL: https://issues.apache.org/jira/browse/SPARK-26097 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-26043) Make SparkHadoopUtil private to Spark

[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26043: Assignee: Apache Spark > Make SparkHadoopUtil private to Spark >

[jira] [Commented] (SPARK-26026) Published Scaladoc jars missing from Maven Central

[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690373#comment-16690373 ] Sean Owen commented on SPARK-26026: --- Disregard my previous comment. It was because it caused the 2.12

[jira] [Issue Comment Deleted] (SPARK-26026) Published Scaladoc jars missing from Maven Central

[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26026: -- Comment: was deleted (was: Ah, I think it was this: {code}

[jira] [Comment Edited] (SPARK-26026) Published Scaladoc jars missing from Maven Central

[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688997#comment-16688997 ] Sean Owen edited comment on SPARK-26026 at 11/17/18 4:31 AM: - Ah, I think it

[jira] [Assigned] (SPARK-26043) Make SparkHadoopUtil private to Spark

[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26043: Assignee: (was: Apache Spark) > Make SparkHadoopUtil private to Spark >

[jira] [Commented] (SPARK-26043) Make SparkHadoopUtil private to Spark

[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690370#comment-16690370 ] Apache Spark commented on SPARK-26043: -- User 'srowen' has created a pull request for this issue:

[jira] [Updated] (SPARK-26043) Make SparkHadoopUtil private to Spark

[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26043: -- Priority: Minor (was: Major) > Make SparkHadoopUtil private to Spark >

[jira] [Resolved] (SPARK-16647) sparksql1.6.2 on yarn with hive metastore1.0.0 thows "alter_table_with_cascade" exception

[ https://issues.apache.org/jira/browse/SPARK-16647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16647. --- Resolution: Won't Fix Spark 1.x is long since not supported, so I'd only pursue this if it is

[jira] [Assigned] (SPARK-26090) Resolve most miscellaneous deprecation and build warnings for Spark 3

[ https://issues.apache.org/jira/browse/SPARK-26090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26090: Assignee: Apache Spark (was: Sean Owen) > Resolve most miscellaneous deprecation and

[jira] [Commented] (SPARK-26090) Resolve most miscellaneous deprecation and build warnings for Spark 3

[ https://issues.apache.org/jira/browse/SPARK-26090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690213#comment-16690213 ] Apache Spark commented on SPARK-26090: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26090) Resolve most miscellaneous deprecation and build warnings for Spark 3

[ https://issues.apache.org/jira/browse/SPARK-26090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26090: Assignee: Sean Owen (was: Apache Spark) > Resolve most miscellaneous deprecation and

[jira] [Commented] (SPARK-26090) Resolve most miscellaneous deprecation and build warnings for Spark 3

[ https://issues.apache.org/jira/browse/SPARK-26090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690212#comment-16690212 ] Apache Spark commented on SPARK-26090: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-26033) Break large ml/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690187#comment-16690187 ] Apache Spark commented on SPARK-26033: -- User 'BryanCutler' has created a pull request for this

[jira] [Assigned] (SPARK-26033) Break large ml/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26033: Assignee: Bryan Cutler (was: Apache Spark) > Break large ml/tests.py files into smaller

[jira] [Assigned] (SPARK-26033) Break large ml/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26033: Assignee: Apache Spark (was: Bryan Cutler) > Break large ml/tests.py files into smaller

[jira] [Commented] (SPARK-8288) ScalaReflection should also try apply methods defined in companion objects when inferring schema from a Product type

[ https://issues.apache.org/jira/browse/SPARK-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690141#comment-16690141 ] Apache Spark commented on SPARK-8288: - User 'drewrobb' has created a pull request for this issue:

[jira] [Resolved] (SPARK-26095) make-distribution.sh is hanging in jenkins

[ https://issues.apache.org/jira/browse/SPARK-26095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26095. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23061

[jira] [Assigned] (SPARK-26095) make-distribution.sh is hanging in jenkins

[ https://issues.apache.org/jira/browse/SPARK-26095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26095: -- Assignee: Marcelo Vanzin > make-distribution.sh is hanging in jenkins >

[jira] [Created] (SPARK-26096) k8s integration tests should run R tests

Marcelo Vanzin created SPARK-26096: -- Summary: k8s integration tests should run R tests Key: SPARK-26096 URL: https://issues.apache.org/jira/browse/SPARK-26096 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-26092) Use CheckpointFileManager to write the streaming metadata file

[ https://issues.apache.org/jira/browse/SPARK-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-26092. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 3.0.0

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690123#comment-16690123 ] Ruslan Dautkhanov commented on SPARK-26019: --- That user said he has seen this error 4-5 times,

[jira] [Commented] (SPARK-26094) Streaming WAL should create parent dirs

[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690075#comment-16690075 ] Imran Rashid commented on SPARK-26094: -- when playing around with this, I noticed another difference

[jira] [Commented] (SPARK-26094) Streaming WAL should create parent dirs

[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690076#comment-16690076 ] Imran Rashid commented on SPARK-26094: -- cc [~xiaochen] > Streaming WAL should create parent dirs >

[jira] [Commented] (SPARK-25871) Streaming WAL should not use hdfs erasure coding, regardless of FS defaults

[ https://issues.apache.org/jira/browse/SPARK-25871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690039#comment-16690039 ] Imran Rashid commented on SPARK-25871: -- this introduced a regression, SPARK-26094 > Streaming WAL

[jira] [Commented] (SPARK-26095) make-distribution.sh is hanging in jenkins

[ https://issues.apache.org/jira/browse/SPARK-26095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690045#comment-16690045 ] Apache Spark commented on SPARK-26095: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26095) make-distribution.sh is hanging in jenkins

[ https://issues.apache.org/jira/browse/SPARK-26095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26095: Assignee: Apache Spark > make-distribution.sh is hanging in jenkins >

[jira] [Assigned] (SPARK-26095) make-distribution.sh is hanging in jenkins

[ https://issues.apache.org/jira/browse/SPARK-26095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26095: Assignee: (was: Apache Spark) > make-distribution.sh is hanging in jenkins >

[jira] [Created] (SPARK-26095) make-distribution.sh is hanging in jenkins

Marcelo Vanzin created SPARK-26095: -- Summary: make-distribution.sh is hanging in jenkins Key: SPARK-26095 URL: https://issues.apache.org/jira/browse/SPARK-26095 Project: Spark Issue Type:

[jira] [Created] (SPARK-26094) Streaming WAL should create parent dirs

Imran Rashid created SPARK-26094: Summary: Streaming WAL should create parent dirs Key: SPARK-26094 URL: https://issues.apache.org/jira/browse/SPARK-26094 Project: Spark Issue Type:

[jira] [Created] (SPARK-26093) Read Avro: ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource

Dagang Wei created SPARK-26093: -- Summary: Read Avro: ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource Key: SPARK-26093 URL: https://issues.apache.org/jira/browse/SPARK-26093

[jira] [Commented] (SPARK-26092) Use CheckpointFileManager to write the streaming metadata file

[ https://issues.apache.org/jira/browse/SPARK-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689919#comment-16689919 ] Apache Spark commented on SPARK-26092: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26092) Use CheckpointFileManager to write the streaming metadata file

[ https://issues.apache.org/jira/browse/SPARK-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26092: Assignee: (was: Apache Spark) > Use CheckpointFileManager to write the streaming

[jira] [Assigned] (SPARK-26092) Use CheckpointFileManager to write the streaming metadata file

[ https://issues.apache.org/jira/browse/SPARK-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26092: Assignee: Apache Spark > Use CheckpointFileManager to write the streaming metadata file

[jira] [Commented] (SPARK-26092) Use CheckpointFileManager to write the streaming metadata file

[ https://issues.apache.org/jira/browse/SPARK-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689920#comment-16689920 ] Apache Spark commented on SPARK-26092: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-26092) Use CheckpointFileManager to write the streaming metadata file

[ https://issues.apache.org/jira/browse/SPARK-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26092: - Issue Type: Bug (was: Test) > Use CheckpointFileManager to write the streaming metadata file >

[jira] [Created] (SPARK-26092) Use CheckpointFileManager to write the streaming metadata file

Shixiong Zhu created SPARK-26092: Summary: Use CheckpointFileManager to write the streaming metadata file Key: SPARK-26092 URL: https://issues.apache.org/jira/browse/SPARK-26092 Project: Spark

[jira] [Assigned] (SPARK-26091) Upgrade to 2.3.4 for Hive Metastore Client 2.3

[ https://issues.apache.org/jira/browse/SPARK-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26091: Assignee: (was: Apache Spark) > Upgrade to 2.3.4 for Hive Metastore Client 2.3 >

[jira] [Commented] (SPARK-26091) Upgrade to 2.3.4 for Hive Metastore Client 2.3

[ https://issues.apache.org/jira/browse/SPARK-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689894#comment-16689894 ] Apache Spark commented on SPARK-26091: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-26091) Upgrade to 2.3.4 for Hive Metastore Client 2.3

[ https://issues.apache.org/jira/browse/SPARK-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26091: Assignee: Apache Spark > Upgrade to 2.3.4 for Hive Metastore Client 2.3 >

[jira] [Created] (SPARK-26091) Upgrade to 2.3.4 for Hive Metastore Client 2.3

Dongjoon Hyun created SPARK-26091: - Summary: Upgrade to 2.3.4 for Hive Metastore Client 2.3 Key: SPARK-26091 URL: https://issues.apache.org/jira/browse/SPARK-26091 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25905) BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers

[ https://issues.apache.org/jira/browse/SPARK-25905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689860#comment-16689860 ] Apache Spark commented on SPARK-25905: -- User 'wypoon' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25905) BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers

[ https://issues.apache.org/jira/browse/SPARK-25905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25905: Assignee: (was: Apache Spark) > BlockManager should expose getRemoteManagedBuffer to

[jira] [Assigned] (SPARK-25905) BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers

[ https://issues.apache.org/jira/browse/SPARK-25905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25905: Assignee: Apache Spark > BlockManager should expose getRemoteManagedBuffer to avoid

[jira] [Updated] (SPARK-21559) Remove Mesos fine-grained mode

[ https://issues.apache.org/jira/browse/SPARK-21559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21559: Target Version/s: 3.0.0 Labels: release-notes (was: ) > Remove Mesos fine-grained

[jira] [Resolved] (SPARK-26069) Flaky test: RpcIntegrationSuite.sendRpcWithStreamFailures

[ https://issues.apache.org/jira/browse/SPARK-26069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-26069. -- Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > Flaky test:

[jira] [Commented] (SPARK-22865) Publish Official Apache Spark Docker images

[ https://issues.apache.org/jira/browse/SPARK-22865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689695#comment-16689695 ] Luciano Resende commented on SPARK-22865: - Any update on this issue?  [~akorzhuev] seems to have

[jira] [Comment Edited] (SPARK-22865) Publish Official Apache Spark Docker images

[ https://issues.apache.org/jira/browse/SPARK-22865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689695#comment-16689695 ] Luciano Resende edited comment on SPARK-22865 at 11/16/18 5:23 PM: ---

[jira] [Commented] (SPARK-26045) Error in the spark 2.4 release package with the spark-avro_2.11 depdency

[ https://issues.apache.org/jira/browse/SPARK-26045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689641#comment-16689641 ] Sean Owen commented on SPARK-26045: --- This looks like you are using a different version of Avro than

[jira] [Created] (SPARK-26090) Resolve most miscellaneous deprecation and build warnings for Spark 3

Sean Owen created SPARK-26090: - Summary: Resolve most miscellaneous deprecation and build warnings for Spark 3 Key: SPARK-26090 URL: https://issues.apache.org/jira/browse/SPARK-26090 Project: Spark

[jira] [Updated] (SPARK-26052) Spark should output a _SUCCESS file for every partition correctly written

[ https://issues.apache.org/jira/browse/SPARK-26052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26052: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Spark should output a

[jira] [Resolved] (SPARK-26073) remove invalid comment as we don't use it anymore

[ https://issues.apache.org/jira/browse/SPARK-26073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26073. --- Resolution: Fixed Assignee: caoxuewen Fix Version/s: 3.0.0 Resolved by

[jira] [Resolved] (SPARK-1517) Publish nightly snapshots of documentation, maven artifacts, and binary builds

[ https://issues.apache.org/jira/browse/SPARK-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1517. -- Resolution: Won't Fix Ancient JIRA here ... I think we did publish these and don't know that we do

[jira] [Comment Edited] (SPARK-23709) BaggedPoint.convertToBaggedRDDSamplingWithReplacement does not guarantee the sum of weights

[ https://issues.apache.org/jira/browse/SPARK-23709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689633#comment-16689633 ] Olivier Sannier edited comment on SPARK-23709 at 11/16/18 4:30 PM: ---

[jira] [Commented] (SPARK-23709) BaggedPoint.convertToBaggedRDDSamplingWithReplacement does not guarantee the sum of weights

[ https://issues.apache.org/jira/browse/SPARK-23709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689633#comment-16689633 ] Olivier Sannier commented on SPARK-23709: - Well, sure, but nobody answered on the mailing list

[jira] [Updated] (SPARK-22481) CatalogImpl.refreshTable is slow

[ https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22481: -- Priority: Major (was: Critical) Issue Type: Improvement (was: Bug) > CatalogImpl.refreshTable

[jira] [Resolved] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

[ https://issues.apache.org/jira/browse/SPARK-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12963. --- Resolution: Not A Problem > In cluster mode,spark_local_ip will cause driver exception:Service

[jira] [Updated] (SPARK-20697) MSCK REPAIR TABLE resets the Storage Information for bucketed hive tables.

[ https://issues.apache.org/jira/browse/SPARK-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20697: -- Priority: Major (was: Critical) This sounds like Hive functionality though; is it even resolvable in

[jira] [Updated] (SPARK-23530) It's not appropriate to let the original master exit while the leader of zookeeper shutdown

[ https://issues.apache.org/jira/browse/SPARK-23530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23530: -- Priority: Major (was: Critical) What is the consequence though? > It's not appropriate to let the

[jira] [Resolved] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5594. -- Resolution: Cannot Reproduce Closing a very old issue here; reopen if it happens on master >

[jira] [Resolved] (SPARK-23709) BaggedPoint.convertToBaggedRDDSamplingWithReplacement does not guarantee the sum of weights

[ https://issues.apache.org/jira/browse/SPARK-23709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23709. --- Resolution: Not A Problem This is a question for the mailing list rather than JIRA at this stage.

[jira] [Updated] (SPARK-24985) Executing SQL with "Full Outer Join" on top of large tables when there is data skew met OOM

[ https://issues.apache.org/jira/browse/SPARK-24985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24985: -- Priority: Major (was: Critical) > Executing SQL with "Full Outer Join" on top of large tables when

[jira] [Updated] (SPARK-24906) Adaptively set split size for columnar file to ensure the task read data size fit expectation

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24906: -- Priority: Major (was: Critical) > Adaptively set split size for columnar file to ensure the task

[jira] [Updated] (SPARK-25309) Sci-kit Learn like Auto Pipeline Parallelization in Spark

[ https://issues.apache.org/jira/browse/SPARK-25309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25309: -- Priority: Minor (was: Critical) > Sci-kit Learn like Auto Pipeline Parallelization in Spark >

[jira] [Updated] (SPARK-25917) Spark UI's executors page loads forever when memoryMetrics in None. Fix is to JSON ignore memorymetrics when it is None.

[ https://issues.apache.org/jira/browse/SPARK-25917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25917: -- Priority: Major (was: Critical) Issue Type: Improvement (was: Bug) > Spark UI's executors

[jira] [Updated] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25548: -- Priority: Major (was: Critical) > In the PruneFileSourcePartitions optimizer, replace the

[jira] [Updated] (SPARK-24703) Unable to multiply calendar interval

[ https://issues.apache.org/jira/browse/SPARK-24703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24703: -- Priority: Minor (was: Critical) Issue Type: Improvement (was: Bug) Unless it works in Hive or

[jira] [Closed] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-14492. - > Spark SQL 1.6.0 does not work with external Hive metastore version lower than > 1.2.0; its not backwards

[jira] [Resolved] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14492. --- Resolution: Not A Problem This should stay closed. The reports here are about building with varying

[jira] [Resolved] (SPARK-26034) Break large mllib/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26034. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23056

[jira] [Assigned] (SPARK-25934) Mesos: SPARK_CONF_DIR should not be propogated by spark submit

[ https://issues.apache.org/jira/browse/SPARK-25934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25934: - Assignee: Matt Molek > Mesos: SPARK_CONF_DIR should not be propogated by spark submit >

[jira] [Resolved] (SPARK-25934) Mesos: SPARK_CONF_DIR should not be propogated by spark submit

[ https://issues.apache.org/jira/browse/SPARK-25934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25934. --- Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 2.3.3

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689556#comment-16689556 ] Imran Rashid commented on SPARK-4105: - fyi I opened SPARK-26089 for handling issues in large corrupt

[jira] [Commented] (SPARK-26089) Handle large corrupt shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689554#comment-16689554 ] Imran Rashid commented on SPARK-26089: -- cc [~joshrosen] [~zsxwing] as you looked at SPARK-4105

[jira] [Created] (SPARK-26089) Handle large corrupt shuffle blocks

Imran Rashid created SPARK-26089: Summary: Handle large corrupt shuffle blocks Key: SPARK-26089 URL: https://issues.apache.org/jira/browse/SPARK-26089 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-26088) DataSourceV2 should expose row count and attribute statistics

Martin Junghanns created SPARK-26088: Summary: DataSourceV2 should expose row count and attribute statistics Key: SPARK-26088 URL: https://issues.apache.org/jira/browse/SPARK-26088 Project: Spark

[jira] [Resolved] (SPARK-25799) DataSourceApiV2 scan reuse does not respect options

[ https://issues.apache.org/jira/browse/SPARK-25799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Kießling resolved SPARK-25799. -- Resolution: Fixed Fix Version/s: 2.4.0 > DataSourceApiV2 scan reuse does not respect

[jira] [Commented] (SPARK-25799) DataSourceApiV2 scan reuse does not respect options

[ https://issues.apache.org/jira/browse/SPARK-25799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689409#comment-16689409 ] Max Kießling commented on SPARK-25799: -- Apparently this issues is already resolved with the new

[jira] [Commented] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689368#comment-16689368 ] Apache Spark commented on SPARK-26078: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26078: Assignee: Apache Spark > WHERE .. IN fails to filter rows when used in combination with

[jira] [Commented] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689370#comment-16689370 ] Apache Spark commented on SPARK-26078: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26078: Assignee: (was: Apache Spark) > WHERE .. IN fails to filter rows when used in

[jira] [Updated] (SPARK-26087) Spark on K8S jobs fail when returning large result sets in client mode under certain conditions

[ https://issues.apache.org/jira/browse/SPARK-26087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-26087: Description: *Short description*: Jobs on Spark on k8s fail when certain conditions are met:

[jira] [Commented] (SPARK-26087) Spark on K8S jobs fail when returning large result sets in client mode under certain conditions

[ https://issues.apache.org/jira/browse/SPARK-26087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689358#comment-16689358 ] Piotr Mrowczynski commented on SPARK-26087: --- Corresponding error message is: {code:java}

[jira] [Comment Edited] (SPARK-26087) Spark on K8S jobs fail when returning large result sets in client mode under certain conditions

[ https://issues.apache.org/jira/browse/SPARK-26087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689358#comment-16689358 ] Piotr Mrowczynski edited comment on SPARK-26087 at 11/16/18 12:30 PM:

[jira] [Created] (SPARK-26087) Spark on K8S jobs fail when returning large result sets in client mode under certain conditions

Luca Canali created SPARK-26087: --- Summary: Spark on K8S jobs fail when returning large result sets in client mode under certain conditions Key: SPARK-26087 URL: https://issues.apache.org/jira/browse/SPARK-26087

[jira] [Commented] (SPARK-26084) AggregateExpression.references fails on unresolved expression trees

[ https://issues.apache.org/jira/browse/SPARK-26084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689312#comment-16689312 ] Herman van Hovell commented on SPARK-26084: --- [~simeons] since you have already propose a

[jira] [Commented] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689268#comment-16689268 ] Liang-Chi Hsieh commented on SPARK-26078: - I simply have a look for it, but don't have great fix

[jira] [Commented] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689262#comment-16689262 ] Marco Gaido commented on SPARK-26078: - I'll investigate this immediately, thanks [~cloud_fan]. >

[jira] [Commented] (SPARK-26045) Error in the spark 2.4 release package with the spark-avro_2.11 depdency

[ https://issues.apache.org/jira/browse/SPARK-26045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689217#comment-16689217 ] Marco Gaido commented on SPARK-26045: - [~o.garcia] can you please create a PR for this? > Error in

[jira] [Commented] (SPARK-26059) Spark standalone mode, does not correctly record a failed Spark Job.

[ https://issues.apache.org/jira/browse/SPARK-26059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689210#comment-16689210 ] Prashant Sharma commented on SPARK-26059: - Well to reproduce it, we need to submit a failing job

[jira] [Commented] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689200#comment-16689200 ] Neeraj Bhadani commented on SPARK-26075: [~hyukjin.kwon] I haven't tried this with Spark 2.4.

[jira] [Commented] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689202#comment-16689202 ] Hyukjin Kwon commented on SPARK-26075: -- It should be helpful if that can be verified. > Cannot

  1   2   >