[jira] [Assigned] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-3990: Assignee: Xiangrui Meng kryo.KryoException caused by ALS.trainImplicit in pyspark

[jira] [Updated] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3990: - Description: When we tried ALS.trainImplicit() in pyspark environment, it only works for

[jira] [Updated] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3990: - Priority: Critical (was: Major) kryo.KryoException caused by ALS.trainImplicit in pyspark

[jira] [Commented] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176623#comment-14176623 ] Xiangrui Meng commented on SPARK-3990: -- [~gen] Could you try the following and see

[jira] [Updated] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3995: - Assignee: Jeremy Freeman [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

[jira] [Updated] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3995: - Target Version/s: 1.1.1, 1.2.0 [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

[jira] [Commented] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176637#comment-14176637 ] Xiangrui Meng commented on SPARK-3995: -- [~freeman-lab] Thanks for catching the bug!

[jira] [Created] (SPARK-4005) handle message replies in receive instead of in the individual private methods

Zhang, Liye created SPARK-4005: -- Summary: handle message replies in receive instead of in the individual private methods Key: SPARK-4005 URL: https://issues.apache.org/jira/browse/SPARK-4005 Project:

[jira] [Commented] (SPARK-4005) handle message replies in receive instead of in the individual private methods

[ https://issues.apache.org/jira/browse/SPARK-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176676#comment-14176676 ] Apache Spark commented on SPARK-4005: - User 'liyezhang556520' has created a pull

[jira] [Commented] (SPARK-1042) spark cleans all java broadcast variables when it hits the spark.cleaner.ttl

[ https://issues.apache.org/jira/browse/SPARK-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176708#comment-14176708 ] Tal Sliwowicz commented on SPARK-1042: -- [~qqsun8819] I think the issue was resolved

[jira] [Created] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

Tal Sliwowicz created SPARK-4006: Summary: Spark Driver crashes whenever an Executor is registered twice Key: SPARK-4006 URL: https://issues.apache.org/jira/browse/SPARK-4006 Project: Spark

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tal Sliwowicz updated SPARK-4006: - Description: This is a huge robustness issue for us, in mission critical , time sensitive (real

[jira] [Updated] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2429: --- Attachment: benchmark2.html HI [~rnowling], I improved the performance of my implementation. Could

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176747#comment-14176747 ] Saisai Shao commented on SPARK-3633: From my test, I think this problem might be

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tal Sliwowicz updated SPARK-4006: - Description: This is a huge robustness issue for us (Taboola), in mission critical , time

[jira] [Commented] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176774#comment-14176774 ] Tal Sliwowicz commented on SPARK-4006: -- Fixed in -

[jira] [Updated] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2429: --- Attachment: 2014-10-20_divisive-hierarchical-clustering.pdf I made a slide to explain the abstraction

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD commented on SPARK-3967: -- Hi Ryan, Thanks for your help. You

[jira] [Commented] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176788#comment-14176788 ] Gen TANG commented on SPARK-3990: - [~mengxr] I tried the code that you provided and it

[jira] [Commented] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176798#comment-14176798 ] Apache Spark commented on SPARK-4006: - User 'tsliwowicz' has created a pull request

[jira] [Updated] (SPARK-3968) Use parquet-mr filter2 api in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Datta updated SPARK-3968: -- Description: The parquet-mr project has introduced a new filter api , along with several fixes (like

[jira] [Created] (SPARK-4007) EOF exception to load from HDFS an JavaRDD

Cristian Galán created SPARK-4007: - Summary: EOF exception to load from HDFS an JavaRDD Key: SPARK-4007 URL: https://issues.apache.org/jira/browse/SPARK-4007 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4007) EOF exception to load an JavaRDD from HDFS

[ https://issues.apache.org/jira/browse/SPARK-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cristian Galán updated SPARK-4007: -- Priority: Major (was: Critical) Environment: hadoop-client-2.30 hadoop-hdfs-2.30

[jira] [Created] (SPARK-4008) Fix kryo with fold in KryoSerializerSuite

Shixiong Zhu created SPARK-4008: --- Summary: Fix kryo with fold in KryoSerializerSuite Key: SPARK-4008 URL: https://issues.apache.org/jira/browse/SPARK-4008 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-4008) g

[ https://issues.apache.org/jira/browse/SPARK-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4008: Summary: g (was: Fix kryo with fold in KryoSerializerSuite) g - Key:

[jira] [Commented] (SPARK-4007) EOF exception to load an JavaRDD from HDFS

[ https://issues.apache.org/jira/browse/SPARK-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176824#comment-14176824 ] Sean Owen commented on SPARK-4007: -- Is this going to have any information associated to

[jira] [Commented] (SPARK-4008) Fix kryo with fold in KryoSerializerSuite

[ https://issues.apache.org/jira/browse/SPARK-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176829#comment-14176829 ] Apache Spark commented on SPARK-4008: - User 'zsxwing' has created a pull request for

[jira] [Comment Edited] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD edited comment on SPARK-3967 at 10/20/14 12:20 PM:

[jira] [Comment Edited] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD edited comment on SPARK-3967 at 10/20/14 12:19 PM:

[jira] [Comment Edited] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD edited comment on SPARK-3967 at 10/20/14 12:20 PM:

[jira] [Comment Edited] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD edited comment on SPARK-3967 at 10/20/14 12:21 PM:

[jira] [Created] (SPARK-4009) HiveTableScan should use makeRDDForTable instead of makeRDDForPartitionedTable for partitioned table when partitionPruningPred is None

YanTang Zhai created SPARK-4009: --- Summary: HiveTableScan should use makeRDDForTable instead of makeRDDForPartitionedTable for partitioned table when partitionPruningPred is None Key: SPARK-4009 URL:

[jira] [Commented] (SPARK-4009) HiveTableScan should use makeRDDForTable instead of makeRDDForPartitionedTable for partitioned table when partitionPruningPred is None

[ https://issues.apache.org/jira/browse/SPARK-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176888#comment-14176888 ] Apache Spark commented on SPARK-4009: - User 'YanTangZhai' has created a pull request

[jira] [Closed] (SPARK-4007) EOF exception to load an JavaRDD from HDFS

[ https://issues.apache.org/jira/browse/SPARK-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cristian Galán closed SPARK-4007. - Resolution: Invalid Don't need fix because it's not a problem of Spark, the objects are saved in

[jira] [Created] (SPARK-4010) spark UI returns 500 in yarn-client mode

Guoqiang Li created SPARK-4010: -- Summary: spark UI returns 500 in yarn-client mode Key: SPARK-4010 URL: https://issues.apache.org/jira/browse/SPARK-4010 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4010) spark UI returns 500 in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-4010: --- Component/s: Web UI spark UI returns 500 in yarn-client mode

[jira] [Commented] (SPARK-4002) JavaKafkaStreamSuite.testKafkaStream fails on OSX

[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176969#comment-14176969 ] Ryan Williams commented on SPARK-4002: -- Hey Saisai, my last post mentioned that I'd

[jira] [Updated] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2429: --- Attachment: (was: 2014-10-20_divisive-hierarchical-clustering.pdf) Hierarchical Implementation

[jira] [Updated] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2429: --- Attachment: 2014-10-20_divisive-hierarchical-clustering.pdf Hierarchical Implementation of KMeans

[jira] [Updated] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2429: --- Attachment: (was: 2014-10-20_divisive-hierarchical-clustering.pdf) Hierarchical Implementation

[jira] [Updated] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2429: --- Attachment: 2014-10-20_divisive-hierarchical-clustering.pdf Hierarchical Implementation of KMeans

[jira] [Updated] (SPARK-4010) Spark UI returns 500 in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-4010: --- Summary: Spark UI returns 500 in yarn-client mode (was: spark UI returns 500 in yarn-client mode )

[jira] [Commented] (SPARK-4002) JavaKafkaStreamSuite.testKafkaStream fails on OSX

[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176995#comment-14176995 ] Sean Owen commented on SPARK-4002: -- FWIW It doesn't fail for me from master right now if

[jira] [Commented] (SPARK-4010) Spark UI returns 500 in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176996#comment-14176996 ] Apache Spark commented on SPARK-4010: - User 'witgo' has created a pull request for

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177000#comment-14177000 ] Ryan Williams commented on SPARK-3967: -- Cool, I'll add it there as well [~preaudc],

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177040#comment-14177040 ] Christophe PRÉAUD commented on SPARK-3967: -- That's fine, thanks! Spark

[jira] [Updated] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4001: - Assignee: Jacky Li Add Apriori algorithm to Spark MLlib

[jira] [Updated] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4001: - Target Version/s: (was: 1.2.0) Add Apriori algorithm to Spark MLlib

[jira] [Updated] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4001: - Affects Version/s: (was: 1.1.0) Add Apriori algorithm to Spark MLlib

[jira] [Commented] (SPARK-4002) JavaKafkaStreamSuite.testKafkaStream fails on OSX

[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177076#comment-14177076 ] Ryan Williams commented on SPARK-4002: -- Also, if i were to try to bisect this, any

[jira] [Commented] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177081#comment-14177081 ] Xiangrui Meng commented on SPARK-4001: -- [~jackylk] Could you provide references and

[jira] [Commented] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177085#comment-14177085 ] Xiangrui Meng commented on SPARK-3990: -- Sure, please link to this JIRA so we can keep

[jira] [Resolved] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3948. --- Resolution: Fixed Fix Version/s: 1.1.1 1.2.0 Issue resolved by pull request

[jira] [Commented] (SPARK-3537) Statistics for cached RDDs

[ https://issues.apache.org/jira/browse/SPARK-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177146#comment-14177146 ] Apache Spark commented on SPARK-3537: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3914) InMemoryRelation should inherit statistics of its child to enable broadcast join

[ https://issues.apache.org/jira/browse/SPARK-3914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177161#comment-14177161 ] Apache Spark commented on SPARK-3914: - User 'liancheng' has created a pull request for

[jira] [Updated] (SPARK-4010) Spark UI returns 500 in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4010: - Affects Version/s: 1.2.0 Spark UI returns 500 in yarn-client mode

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177170#comment-14177170 ] Reynold Xin commented on SPARK-3948: How often does this bug manifest? If it is often

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177173#comment-14177173 ] Josh Rosen commented on SPARK-3948: --- [~rxin] The patch here should actually fix the

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177195#comment-14177195 ] Reynold Xin commented on SPARK-3948: Ok then it sounds good. Sort-based shuffle can

[jira] [Created] (SPARK-4011) tighten the visibility of the members in Worker class

Nan Zhu created SPARK-4011: -- Summary: tighten the visibility of the members in Worker class Key: SPARK-4011 URL: https://issues.apache.org/jira/browse/SPARK-4011 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-3986) Fix package names to fit their directory names.

[ https://issues.apache.org/jira/browse/SPARK-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3986. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2835

[jira] [Updated] (SPARK-3986) Fix package names to fit their directory names.

[ https://issues.apache.org/jira/browse/SPARK-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3986: Assignee: Takuya Ueshin Fix package names to fit their directory names.

[jira] [Resolved] (SPARK-576) Design and develop a more precise progress estimator

[ https://issues.apache.org/jira/browse/SPARK-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-576. -- Resolution: Won't Fix Closing this as Won't Fix; see our discussion at

[jira] [Resolved] (SPARK-3736) Workers should reconnect to Master if disconnected

[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3736. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2828

[jira] [Created] (SPARK-4012) Uncaught OOM in ContextCleaner

Nan Zhu created SPARK-4012: -- Summary: Uncaught OOM in ContextCleaner Key: SPARK-4012 URL: https://issues.apache.org/jira/browse/SPARK-4012 Project: Spark Issue Type: Bug Components: Spark

[jira] [Resolved] (SPARK-3467) Python BatchedSerializer should dynamically lower batch size for large objects

[ https://issues.apache.org/jira/browse/SPARK-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-3467. --- Resolution: Fixed Fix Version/s: 1.2.0 This is fixed by

[jira] [Created] (SPARK-4013) Do not create multiple actor systems on each executor

Andrew Or created SPARK-4013: Summary: Do not create multiple actor systems on each executor Key: SPARK-4013 URL: https://issues.apache.org/jira/browse/SPARK-4013 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4013) Do not create multiple actor systems on each executor

[ https://issues.apache.org/jira/browse/SPARK-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4013: - Description: This causes many more error messages to be logged on the driver than necessary when an

[jira] [Created] (SPARK-4014) TaskContext.attemptId returns taskId

Yin Huai created SPARK-4014: --- Summary: TaskContext.attemptId returns taskId Key: SPARK-4014 URL: https://issues.apache.org/jira/browse/SPARK-4014 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177365#comment-14177365 ] Gen TANG commented on SPARK-3990: - Yeah, I will try. But I am afraid that I couldn't fix

[jira] [Commented] (SPARK-3815) LPAD function does not work in where predicate

[ https://issues.apache.org/jira/browse/SPARK-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177396#comment-14177396 ] Yana Kadiyska commented on SPARK-3815: -- Venkata, I am building master and I am still

[jira] [Comment Edited] (SPARK-3815) LPAD function does not work in where predicate

[ https://issues.apache.org/jira/browse/SPARK-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177396#comment-14177396 ] Yana Kadiyska edited comment on SPARK-3815 at 10/20/14 7:58 PM:

[jira] [Created] (SPARK-4015) Documentation in the streaming context references non-existent function

holdenk created SPARK-4015: -- Summary: Documentation in the streaming context references non-existent function Key: SPARK-4015 URL: https://issues.apache.org/jira/browse/SPARK-4015 Project: Spark

[jira] [Commented] (SPARK-3889) JVM dies with SIGBUS, resulting in ConnectionManager failed ACK

[ https://issues.apache.org/jira/browse/SPARK-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177410#comment-14177410 ] Zach Fry commented on SPARK-3889: - Any chance of getting this backported to 1.1.1? If

[jira] [Resolved] (SPARK-3207) Choose splits for continuous features in DecisionTree more adaptively

[ https://issues.apache.org/jira/browse/SPARK-3207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3207. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2780

[jira] [Updated] (SPARK-4014) TaskContext.attemptId returns taskId

[ https://issues.apache.org/jira/browse/SPARK-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4014: Priority: Minor (was: Major) TaskContext.attemptId returns taskId

[jira] [Updated] (SPARK-3467) Python BatchedSerializer should dynamically lower batch size for large objects

[ https://issues.apache.org/jira/browse/SPARK-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3467: - Assignee: Davies Liu Python BatchedSerializer should dynamically lower batch size for large

[jira] [Commented] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177537#comment-14177537 ] Davies Liu commented on SPARK-3990: --- The default serializer change was introduced by

[jira] [Reopened] (SPARK-2652) Turning default configurations for PySpark

[ https://issues.apache.org/jira/browse/SPARK-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-2652: --- This introduced some regression in MLlib, so I would like to revert the change for default serializer in

[jira] [Commented] (SPARK-4013) Do not create multiple actor systems on each executor

[ https://issues.apache.org/jira/browse/SPARK-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177539#comment-14177539 ] Apache Spark commented on SPARK-4013: - User 'andrewor14' has created a pull request

[jira] [Created] (SPARK-4016) Allow user to optionally show additional, advanced metrics in the UI

Kay Ousterhout created SPARK-4016: - Summary: Allow user to optionally show additional, advanced metrics in the UI Key: SPARK-4016 URL: https://issues.apache.org/jira/browse/SPARK-4016 Project: Spark

[jira] [Created] (SPARK-4017) Progress bar in console

Davies Liu created SPARK-4017: - Summary: Progress bar in console Key: SPARK-4017 URL: https://issues.apache.org/jira/browse/SPARK-4017 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-4018) RDD.reduce failing with java.lang.ClassCastException: org.apache.spark.SparkContext$$anonfun$26 cannot be cast to scala.Function2

Haithem Turki created SPARK-4018: Summary: RDD.reduce failing with java.lang.ClassCastException: org.apache.spark.SparkContext$$anonfun$26 cannot be cast to scala.Function2 Key: SPARK-4018 URL:

[jira] [Resolved] (SPARK-3906) Support joins of multiple tables in SparkSQL (SQLContext, not HiveQL)

[ https://issues.apache.org/jira/browse/SPARK-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3906. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2767

[jira] [Commented] (SPARK-4012) Uncaught OOM in ContextCleaner

[ https://issues.apache.org/jira/browse/SPARK-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177569#comment-14177569 ] Apache Spark commented on SPARK-4012: - User 'CodingCat' has created a pull request for

[jira] [Resolved] (SPARK-3800) BindingException when grouping on nested fields

[ https://issues.apache.org/jira/browse/SPARK-3800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3800. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2658

[jira] [Updated] (SPARK-4019) Repartitioning with more than 2000 partitions drops all data

[ https://issues.apache.org/jira/browse/SPARK-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4019: - Priority: Critical (was: Major) Repartitioning with more than 2000 partitions drops all data

[jira] [Created] (SPARK-4019) Repartitioning with more than 2000 partitions drops all data

Xiangrui Meng created SPARK-4019: Summary: Repartitioning with more than 2000 partitions drops all data Key: SPARK-4019 URL: https://issues.apache.org/jira/browse/SPARK-4019 Project: Spark

[jira] [Updated] (SPARK-4019) Repartitioning with more than 2000 partitions drops all data

[ https://issues.apache.org/jira/browse/SPARK-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4019: - Component/s: Spark Core Description: {code} sc.makeRDD(0 until 10,

[jira] [Updated] (SPARK-3966) Fix nullabilities of Cast related to DateType.

[ https://issues.apache.org/jira/browse/SPARK-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3966: Assignee: Takuya Ueshin Fix nullabilities of Cast related to DateType.

[jira] [Commented] (SPARK-4018) RDD.reduce failing with java.lang.ClassCastException: org.apache.spark.SparkContext$$anonfun$26 cannot be cast to scala.Function2

[ https://issues.apache.org/jira/browse/SPARK-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177592#comment-14177592 ] Sean Owen commented on SPARK-4018: -- Your sample code is Java, but the error seems to

[jira] [Resolved] (SPARK-3966) Fix nullabilities of Cast related to DateType.

[ https://issues.apache.org/jira/browse/SPARK-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3966. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2820

[jira] [Updated] (SPARK-4013) Do not run multiple actor systems on each executor

[ https://issues.apache.org/jira/browse/SPARK-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4013: - Summary: Do not run multiple actor systems on each executor (was: Do not create multiple actor systems

[jira] [Commented] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177606#comment-14177606 ] Xiangrui Meng commented on SPARK-3990: -- [~davies] Is there a JIRA that we can link?

[jira] [Commented] (SPARK-4019) Repartitioning with more than 2000 partitions drops all data

[ https://issues.apache.org/jira/browse/SPARK-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177617#comment-14177617 ] Josh Rosen commented on SPARK-4019: --- This issue is caused by a bug in

[jira] [Updated] (SPARK-4014) TaskContext.attemptId returns taskId

[ https://issues.apache.org/jira/browse/SPARK-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4014: Description: In TaskRunner, we assign the taskId of a task to the attempId of the corresponding

[jira] [Commented] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177640#comment-14177640 ] Xiangrui Meng commented on SPARK-4001: -- No. Add Apriori algorithm to Spark MLlib

[jira] [Commented] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177643#comment-14177643 ] Jacky Li commented on SPARK-4001: - Maybe there is a misunderstand, I do not mean to use it

[jira] [Commented] (SPARK-4018) RDD.reduce failing with java.lang.ClassCastException: org.apache.spark.SparkContext$$anonfun$26 cannot be cast to scala.Function2

[ https://issues.apache.org/jira/browse/SPARK-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177648#comment-14177648 ] Haithem Turki commented on SPARK-4018: -- Not using the shell but spinning up a

[jira] [Commented] (SPARK-4001) Add Apriori algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177658#comment-14177658 ] Xiangrui Meng commented on SPARK-4001: -- I'm asking because I'm not very familiar with

  1   2   >