[jira] [Commented] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329362#comment-16329362 ] Marcelo Vanzin commented on SPARK-23135: (By fine I mean the table renders correctly; the

[jira] [Commented] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329353#comment-16329353 ] Marcelo Vanzin commented on SPARK-23135: I'll try to take a look at the code, but do you have

[jira] [Commented] (SPARK-23020) Re-enable Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329306#comment-16329306 ] Apache Spark commented on SPARK-23020: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329293#comment-16329293 ] Burak Yavuz commented on SPARK-23135: - cc [~vanzin] > Accumulators don't show up properly in the

[jira] [Updated] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-23135: Environment:       was: Didn't do a lot of digging but may be caused by:

[jira] [Updated] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-23135: Attachment: webUIAccumulatorRegression.png > Accumulators don't show up properly in the Stages

[jira] [Updated] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-23135: Description: Didn't do a lot of digging but may be caused by:

[jira] [Created] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

Burak Yavuz created SPARK-23135: --- Summary: Accumulators don't show up properly in the Stages page anymore Key: SPARK-23135 URL: https://issues.apache.org/jira/browse/SPARK-23135 Project: Spark

[jira] [Assigned] (SPARK-23133) Spark options are not passed to the Executor in Docker context

[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23133: Assignee: (was: Apache Spark) > Spark options are not passed to the Executor in

[jira] [Assigned] (SPARK-23133) Spark options are not passed to the Executor in Docker context

[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23133: Assignee: Apache Spark > Spark options are not passed to the Executor in Docker context >

[jira] [Commented] (SPARK-23133) Spark options are not passed to the Executor in Docker context

[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329189#comment-16329189 ] Apache Spark commented on SPARK-23133: -- User 'andrusha' has created a pull request for this issue:

[jira] [Commented] (SPARK-23011) Support alternative function form with group aggregate pandas UDF

[ https://issues.apache.org/jira/browse/SPARK-23011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329177#comment-16329177 ] Apache Spark commented on SPARK-23011: -- User 'icexelloss' has created a pull request for this issue:

[jira] [Commented] (SPARK-8682) Range Join for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-8682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329150#comment-16329150 ] Ruslan Dautkhanov commented on SPARK-8682: -- Range joins need some serious optimization in Spark.

[jira] [Updated] (SPARK-23134) WebUI is showing the cache table details even after cache idle timeout

[ https://issues.apache.org/jira/browse/SPARK-23134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shahid K I updated SPARK-23134: --- Description: After cachedExecutorIdleTimeout, WebUI shows the cached partition details in the

[jira] [Updated] (SPARK-23134) WebUI is showing the cache table details even after cache idle timeout

[ https://issues.apache.org/jira/browse/SPARK-23134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shahid K I updated SPARK-23134: --- Environment:  Run Cache command with below configuration to cache the RDD blocks  

[jira] [Updated] (SPARK-23134) WebUI is showing the cache table details even after cache idle timeout

[ https://issues.apache.org/jira/browse/SPARK-23134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shahid K I updated SPARK-23134: --- Environment:  Run Cache command with below configuration to cache the RDD blocks  

[jira] [Created] (SPARK-23134) WebUI is showing the cache table details even after cache idle timeout

Shahid K I created SPARK-23134: -- Summary: WebUI is showing the cache table details even after cache idle timeout Key: SPARK-23134 URL: https://issues.apache.org/jira/browse/SPARK-23134 Project: Spark

[jira] [Updated] (SPARK-23131) Stackoverflow using ML and Kryo serializer

[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peigen updated SPARK-23131: --- Description: When trying to use GeneralizedLinearRegression model and set SparkConf to use

[jira] [Commented] (SPARK-23131) Stackoverflow using ML and Kryo serializer

[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329100#comment-16329100 ] Peigen commented on SPARK-23131: I realize this happens when I try to serialize the model using

[jira] [Created] (SPARK-23133) Spark options are not passed to the Executor in Docker context

Andrew Korzhuev created SPARK-23133: --- Summary: Spark options are not passed to the Executor in Docker context Key: SPARK-23133 URL: https://issues.apache.org/jira/browse/SPARK-23133 Project: Spark

[jira] [Assigned] (SPARK-23020) Re-enable Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23020: Assignee: Apache Spark (was: Marcelo Vanzin) > Re-enable Flaky Test: >

[jira] [Assigned] (SPARK-23020) Re-enable Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23020: Assignee: Marcelo Vanzin (was: Apache Spark) > Re-enable Flaky Test: >

[jira] [Commented] (SPARK-23020) Re-enable Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329065#comment-16329065 ] Marcelo Vanzin commented on SPARK-23020: Bummer. I'll try to take another look later today. >

[jira] [Commented] (SPARK-22980) Using pandas_udf when inputs are not Pandas's Series or DataFrame

[ https://issues.apache.org/jira/browse/SPARK-22980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328936#comment-16328936 ] Li Jin commented on SPARK-22980: I agree with [~cloud_fan]. I think it's enough to document each args to

[jira] [Commented] (SPARK-21697) NPE & ExceptionInInitializerError trying to load UTF from HDFS

[ https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328928#comment-16328928 ] Steve Loughran commented on SPARK-21697: No, it's spark's ability to have hdfs:// URLs on the

[jira] [Commented] (SPARK-22884) ML test for StructuredStreaming: spark.ml.clustering

[ https://issues.apache.org/jira/browse/SPARK-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328898#comment-16328898 ] Sandor Murakozi commented on SPARK-22884: - Is there anybody working on this? If not I'm happy to

[jira] [Commented] (SPARK-22886) ML test for StructuredStreaming: spark.ml.recommendation

[ https://issues.apache.org/jira/browse/SPARK-22886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328874#comment-16328874 ] Gabor Somogyi commented on SPARK-22886: --- I would like to work on this. Please notify me if somebody

[jira] [Commented] (SPARK-23131) Stackoverflow using ML and Kryo serializer

[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328868#comment-16328868 ] Sean Owen commented on SPARK-23131: --- This requires updating Twitter Chill too, really, to 0.9.2. Have

[jira] [Commented] (SPARK-23076) When we call cache() on RDD which depends on ShuffleRowRDD, we will get an error result

[ https://issues.apache.org/jira/browse/SPARK-23076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328801#comment-16328801 ] Sean Owen commented on SPARK-23076: --- You're relying on behavior that this class doesn't provide. It

[jira] [Resolved] (SPARK-23076) When we call cache() on RDD which depends on ShuffleRowRDD, we will get an error result

[ https://issues.apache.org/jira/browse/SPARK-23076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23076. --- Resolution: Not A Problem > When we call cache() on RDD which depends on ShuffleRowRDD, we will get

[jira] [Assigned] (SPARK-21783) Turn on ORC filter push-down by default

[ https://issues.apache.org/jira/browse/SPARK-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21783: --- Assignee: Dongjoon Hyun > Turn on ORC filter push-down by default >

[jira] [Resolved] (SPARK-21783) Turn on ORC filter push-down by default

[ https://issues.apache.org/jira/browse/SPARK-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21783. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20265

[jira] [Commented] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328787#comment-16328787 ] Sean Owen commented on SPARK-23125: --- The error message you cite, which is from the version in use,

[jira] [Commented] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328782#comment-16328782 ] zhaoshijie commented on SPARK-23125: spark 2.2 use kafka version is 0.10.0.1 and I don't think config

[jira] [Commented] (SPARK-21697) NPE & ExceptionInInitializerError trying to load UTF from HDFS

[ https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328777#comment-16328777 ] Sean Owen commented on SPARK-21697: --- Isn't this an HDFS problem? what could Spark do about it?  > NPE

[jira] [Resolved] (SPARK-23126) I used the Project operator and modified the source. After compiling successfully, and testing the jars, I got the exception. Maybe the phenomenon is related with impli

[ https://issues.apache.org/jira/browse/SPARK-23126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23126. --- Resolution: Invalid Fix Version/s: (was: 2.2.0) Target Version/s: (was: 2.2.0)

[jira] [Resolved] (SPARK-23123) Unable to run Spark Job with Hadoop NameNode Federation using ViewFS

[ https://issues.apache.org/jira/browse/SPARK-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23123. --- Resolution: Not A Problem > Unable to run Spark Job with Hadoop NameNode Federation using ViewFS >

[jira] [Resolved] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23125. --- Resolution: Not A Problem Probably, but this is a Kafka config issue. If you're not using matched

[jira] [Resolved] (SPARK-15401) Spark Thrift server creates empty directories in tmp directory on the driver

[ https://issues.apache.org/jira/browse/SPARK-15401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-15401. - Resolution: Duplicate > Spark Thrift server creates empty directories in tmp directory on the

[jira] [Commented] (SPARK-15401) Spark Thrift server creates empty directories in tmp directory on the driver

[ https://issues.apache.org/jira/browse/SPARK-15401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328738#comment-16328738 ] Marco Gaido commented on SPARK-15401: - this should have been fixed in SPARK-22793. > Spark Thrift

[jira] [Resolved] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-23130. - Resolution: Duplicate > Spark Thrift does not clean-up temporary files (/tmp/*_resources and >

[jira] [Commented] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328734#comment-16328734 ] Marco Gaido commented on SPARK-23130: - The "_resources" files leak should have been fixed by 

[jira] [Assigned] (SPARK-23132) Run ml.image doctests in tests

[ https://issues.apache.org/jira/browse/SPARK-23132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23132: Assignee: Apache Spark > Run ml.image doctests in tests > --

[jira] [Assigned] (SPARK-23132) Run ml.image doctests in tests

[ https://issues.apache.org/jira/browse/SPARK-23132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23132: Assignee: (was: Apache Spark) > Run ml.image doctests in tests >

[jira] [Commented] (SPARK-23132) Run ml.image doctests in tests

[ https://issues.apache.org/jira/browse/SPARK-23132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328714#comment-16328714 ] Apache Spark commented on SPARK-23132: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-23132) Run ml.image doctests in tests

[ https://issues.apache.org/jira/browse/SPARK-23132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23132: - Environment: (was: Seems currently we don't actually run the doctests in \{{ml.image.py}}.

[jira] [Created] (SPARK-23132) Run ml.image doctests in tests

Hyukjin Kwon created SPARK-23132: Summary: Run ml.image doctests in tests Key: SPARK-23132 URL: https://issues.apache.org/jira/browse/SPARK-23132 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-23132) Run ml.image doctests in tests

[ https://issues.apache.org/jira/browse/SPARK-23132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23132: - Description: Seems currently we don't actually run the doctests in  {{ml.image.py}}. It'd be

[jira] [Updated] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoshijie updated SPARK-23125: --- Description: I find DirectKafkaInputDStream(kafka010) Offset commit failed when batch time is more

[jira] [Updated] (SPARK-23131) Stackoverflow using ML and Kryo serializer

[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peigen updated SPARK-23131: --- Priority: Minor (was: Critical) > Stackoverflow using ML and Kryo serializer >

[jira] [Updated] (SPARK-23131) Stackoverflow using ML and Kryo serializer

[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peigen updated SPARK-23131: --- Description: When trying to use GeneralizedLinearRegression model and set SparkConf to use

[jira] [Updated] (SPARK-23131) Stackoverflow using ML and Kryo serializer

[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peigen updated SPARK-23131: --- Environment: (was: When trying to use GeneralizedLinearRegression model and set SparkConf to use

[jira] [Created] (SPARK-23131) Stackoverflow using ML and Kryo serializer

Peigen created SPARK-23131: -- Summary: Stackoverflow using ML and Kryo serializer Key: SPARK-23131 URL: https://issues.apache.org/jira/browse/SPARK-23131 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23123) Unable to run Spark Job with Hadoop NameNode Federation using ViewFS

[ https://issues.apache.org/jira/browse/SPARK-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328651#comment-16328651 ] Steve Loughran commented on SPARK-23123: I've never looked at ViewFS internals before, so treat

[jira] [Assigned] (SPARK-23127) Update FeatureHasher user guide for catCols parameter

[ https://issues.apache.org/jira/browse/SPARK-23127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23127: Assignee: Apache Spark > Update FeatureHasher user guide for catCols parameter >

[jira] [Commented] (SPARK-23127) Update FeatureHasher user guide for catCols parameter

[ https://issues.apache.org/jira/browse/SPARK-23127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328584#comment-16328584 ] Apache Spark commented on SPARK-23127: -- User 'MLnick' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23127) Update FeatureHasher user guide for catCols parameter

[ https://issues.apache.org/jira/browse/SPARK-23127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23127: Assignee: (was: Apache Spark) > Update FeatureHasher user guide for catCols parameter

[jira] [Updated] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoshijie updated SPARK-23125: --- Description: I find DirectKafkaInputDStream(kafka010) Offset commit failed when batch time is more

[jira] [Updated] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Roberts updated SPARK-23130: - Labels: thrift (was: ) > Spark Thrift does not clean-up temporary files (/tmp/*_resources and

[jira] [Commented] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328568#comment-16328568 ] Sean Roberts commented on SPARK-23130: -- * SPARK-15401: Similar report for the "_resources" files *

[jira] [Updated] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Roberts updated SPARK-23130: - Description: Spark Thrift is not cleaning up /tmp for files & directories named like:

[jira] [Updated] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Roberts updated SPARK-23130: - Environment: * Spark versions: 1.6.3, 2.1.0, 2.2.0 * Hadoop distributions: HDP 2.5 - 2.6.3.0 *

[jira] [Updated] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Roberts updated SPARK-23130: - Environment: * Hadoop distributions: HDP 2.5 - 2.6.3.0 * OS: Seen on SLES12, RHEL 7.3 & RHEL

[jira] [Created] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

Sean Roberts created SPARK-23130: Summary: Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout) Key: SPARK-23130 URL: https://issues.apache.org/jira/browse/SPARK-23130

[jira] [Assigned] (SPARK-23129) Lazy init DiskMapIterator#deserializeStream to reduce memory usage when ExternalAppendOnlyMap spill too much times

[ https://issues.apache.org/jira/browse/SPARK-23129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23129: Assignee: Apache Spark > Lazy init DiskMapIterator#deserializeStream to reduce memory

[jira] [Commented] (SPARK-23129) Lazy init DiskMapIterator#deserializeStream to reduce memory usage when ExternalAppendOnlyMap spill too much times

[ https://issues.apache.org/jira/browse/SPARK-23129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328543#comment-16328543 ] Apache Spark commented on SPARK-23129: -- User 'caneGuy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23129) Lazy init DiskMapIterator#deserializeStream to reduce memory usage when ExternalAppendOnlyMap spill too much times

[ https://issues.apache.org/jira/browse/SPARK-23129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23129: Assignee: (was: Apache Spark) > Lazy init DiskMapIterator#deserializeStream to reduce

[jira] [Created] (SPARK-23129) Lazy init DiskMapIterator#deserializeStream to reduce memory usage when ExternalAppendOnlyMap spill too much times

zhoukang created SPARK-23129: Summary: Lazy init DiskMapIterator#deserializeStream to reduce memory usage when ExternalAppendOnlyMap spill too much times Key: SPARK-23129 URL:

[jira] [Created] (SPARK-23128) Introduce QueryStage to improve adaptive execution in Spark SQL

Carson Wang created SPARK-23128: --- Summary: Introduce QueryStage to improve adaptive execution in Spark SQL Key: SPARK-23128 URL: https://issues.apache.org/jira/browse/SPARK-23128 Project: Spark

[jira] [Updated] (SPARK-23127) Update FeatureHasher user guide for catCols parameter

[ https://issues.apache.org/jira/browse/SPARK-23127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-23127: --- Description: SPARK-22801 added the {{categoricalCols}} parameter and updated the Scala and

[jira] [Created] (SPARK-23127) Update FeatureHasher user guide for catCols parameter

Nick Pentreath created SPARK-23127: -- Summary: Update FeatureHasher user guide for catCols parameter Key: SPARK-23127 URL: https://issues.apache.org/jira/browse/SPARK-23127 Project: Spark

[jira] [Created] (SPARK-23126) I used the Project operator and modified the source. After compiling successfully, and testing the jars, I got the exception. Maybe the phenomenon is related with implic

xuetao created SPARK-23126: -- Summary: I used the Project operator and modified the source. After compiling successfully, and testing the jars, I got the exception. Maybe the phenomenon is related with implicits Key: SPARK-23126

[jira] [Updated] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoshijie updated SPARK-23125: --- Description: I find DirectKafkaInputDStream(kafka010) Offset commit failed when batch time is more

[jira] [Updated] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoshijie updated SPARK-23125: --- Description: I find DirectKafkaInputDStream(kafka010) Offset commit failed when batch time is more

[jira] [Updated] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoshijie updated SPARK-23125: --- Description: I find DirectKafkaInputDStream(kafka010) Offset commit failed when batch time is more

[jira] [Updated] (SPARK-23020) Re-enable Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal updated SPARK-23020: --- Summary: Re-enable Flaky Test:

[jira] [Commented] (SPARK-23020) Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328455#comment-16328455 ] Apache Spark commented on SPARK-23020: -- User 'sameeragarwal' has created a pull request for this

[jira] [Assigned] (SPARK-23020) Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23020: Assignee: Marcelo Vanzin (was: Apache Spark) > Flaky Test:

[jira] [Assigned] (SPARK-23020) Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23020: Assignee: Apache Spark (was: Marcelo Vanzin) > Flaky Test:

[jira] [Updated] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoshijie updated SPARK-23125: --- Description: I find DirectKafkaInputDStream(kafka010) Offset commit failed when batch time is more

[jira] [Commented] (SPARK-23118) SparkR 2.3 QA: Programming guide, migration guide, vignettes updates

[ https://issues.apache.org/jira/browse/SPARK-23118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328430#comment-16328430 ] Felix Cheung commented on SPARK-23118: -- did this and opened SPARK-21616 > SparkR 2.3 QA:

[jira] [Created] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

zhaoshijie created SPARK-23125: -- Summary: Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout. Key: SPARK-23125 URL: https://issues.apache.org/jira/browse/SPARK-23125

[jira] [Resolved] (SPARK-23062) EXCEPT documentation should make it clear that it's EXCEPT DISTINCT

[ https://issues.apache.org/jira/browse/SPARK-23062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23062. - Resolution: Fixed Assignee: Henry Robinson Fix Version/s: 2.3.0 > EXCEPT documentation

[jira] [Commented] (SPARK-23115) SparkR 2.3 QA: New R APIs and API docs

[ https://issues.apache.org/jira/browse/SPARK-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328422#comment-16328422 ] Felix Cheung commented on SPARK-23115: -- did this, and opened this

[jira] [Comment Edited] (SPARK-23115) SparkR 2.3 QA: New R APIs and API docs

[ https://issues.apache.org/jira/browse/SPARK-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328422#comment-16328422 ] Felix Cheung edited comment on SPARK-23115 at 1/17/18 8:01 AM: --- did this,

[jira] [Commented] (SPARK-23114) Spark R 2.3 QA umbrella

[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328419#comment-16328419 ] Felix Cheung commented on SPARK-23114: -- sure, [~josephkb] > Spark R 2.3 QA umbrella >

<    1   2