[jira] [Commented] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116667#comment-14116667 ] Sean Owen commented on SPARK-3324: -- Let me try to sketch what's funky about the

[jira] [Created] (SPARK-3328) --with-tachyon build is broken

Elijah Epifanov created SPARK-3328: -- Summary: --with-tachyon build is broken Key: SPARK-3328 URL: https://issues.apache.org/jira/browse/SPARK-3328 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3328) --with-tachyon build is broken

[ https://issues.apache.org/jira/browse/SPARK-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elijah Epifanov updated SPARK-3328: --- Description: cp: tachyon-0.5.0/target/tachyon-0.5.0-jar-with-dependencies.jar: No such file

[jira] [Commented] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116782#comment-14116782 ] Yi Tian commented on SPARK-3324: Thanks [~srowen] explain that. In my beginning idea, I'd

[jira] [Commented] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116785#comment-14116785 ] Yi Tian commented on SPARK-3324: BTW, [~pwendell] there is another problem for IntelliJ.

[jira] [Commented] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116801#comment-14116801 ] Sean Owen commented on SPARK-3324: -- [~tianyi] I seem to remember having a similar

[jira] [Created] (SPARK-3329) HiveQuerySuite SET tests depend on map orderings

William Benton created SPARK-3329: - Summary: HiveQuerySuite SET tests depend on map orderings Key: SPARK-3329 URL: https://issues.apache.org/jira/browse/SPARK-3329 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3329) HiveQuerySuite SET tests depend on map orderings

[ https://issues.apache.org/jira/browse/SPARK-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116821#comment-14116821 ] Apache Spark commented on SPARK-3329: - User 'willb' has created a pull request for

[jira] [Created] (SPARK-3330) Successive test runs with different profiles fail SparkSubmitSuite

Sean Owen created SPARK-3330: Summary: Successive test runs with different profiles fail SparkSubmitSuite Key: SPARK-3330 URL: https://issues.apache.org/jira/browse/SPARK-3330 Project: Spark

[jira] [Created] (SPARK-3331) PEP8 tests fail in release because they check unzipped py4j code

Sean Owen created SPARK-3331: Summary: PEP8 tests fail in release because they check unzipped py4j code Key: SPARK-3331 URL: https://issues.apache.org/jira/browse/SPARK-3331 Project: Spark

[jira] [Updated] (SPARK-3331) PEP8 tests fail because they check unzipped py4j code

[ https://issues.apache.org/jira/browse/SPARK-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3331: - Summary: PEP8 tests fail because they check unzipped py4j code (was: PEP8 tests fail in release because

[jira] [Updated] (SPARK-3331) PEP8 tests fail because they check unzipped py4j code

[ https://issues.apache.org/jira/browse/SPARK-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3331: - Description: PEP8 tests run on files under ./python, but unzipped py4j code is found at

[jira] [Commented] (SPARK-3330) Successive test runs with different profiles fail SparkSubmitSuite

[ https://issues.apache.org/jira/browse/SPARK-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116829#comment-14116829 ] Apache Spark commented on SPARK-3330: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-3331) PEP8 tests fail because they check unzipped py4j code

[ https://issues.apache.org/jira/browse/SPARK-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116830#comment-14116830 ] Apache Spark commented on SPARK-3331: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116868#comment-14116868 ] Nicholas Chammas commented on SPARK-2870: - [~marmbrus], [~davies], [~yhuai] - We

[jira] [Created] (SPARK-3332) Tags shouldn't be the main strategy for machine membership on clusters

Allan Douglas R. de Oliveira created SPARK-3332: --- Summary: Tags shouldn't be the main strategy for machine membership on clusters Key: SPARK-3332 URL:

[jira] [Commented] (SPARK-3332) Tags shouldn't be the main strategy for machine membership on clusters

[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116890#comment-14116890 ] Apache Spark commented on SPARK-3332: - User 'douglaz' has created a pull request for

[jira] [Updated] (SPARK-3010) fix redundant conditional

[ https://issues.apache.org/jira/browse/SPARK-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3010: - Assignee: wangfei fix redundant conditional - Key:

[jira] [Resolved] (SPARK-3010) fix redundant conditional

[ https://issues.apache.org/jira/browse/SPARK-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3010. -- Resolution: Fixed Fix Version/s: (was: 1.1.0) 1.2.0

[jira] [Updated] (SPARK-3010) fix redundant conditional

[ https://issues.apache.org/jira/browse/SPARK-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3010: - Priority: Trivial (was: Major) fix redundant conditional -

[jira] [Created] (SPARK-3333) Large number of partitions causes OOM

Nicholas Chammas created SPARK-: --- Summary: Large number of partitions causes OOM Key: SPARK- URL: https://issues.apache.org/jira/browse/SPARK- Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116895#comment-14116895 ] Nicholas Chammas commented on SPARK-: - Note: I have not yet confirmed that

[jira] [Updated] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-: Description: Here’s a repro for PySpark: {code} a = sc.parallelize([Nick, John, Bob]) a =

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116902#comment-14116902 ] Patrick Wendell commented on SPARK-: Hey [~nchammas] - I don't think anything

[jira] [Updated] (SPARK-3332) Tagging is not atomic with launching instances on EC2

[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3332: --- Summary: Tagging is not atomic with launching instances on EC2 (was: Tags shouldn't be the

[jira] [Commented] (SPARK-3332) Tagging is not atomic with launching instances on EC2

[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116906#comment-14116906 ] Patrick Wendell commented on SPARK-3332: I changed the title slightly - I think

[jira] [Updated] (SPARK-3332) Tagging is not atomic with launching instances on EC2

[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3332: --- Target Version/s: 1.2.0 Tagging is not atomic with launching instances on EC2

[jira] [Commented] (SPARK-3332) Tagging is not atomic with launching instances on EC2

[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116910#comment-14116910 ] Apache Spark commented on SPARK-3332: - User 'JoshRosen' has created a pull request for

[jira] [Updated] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2870: Target Version/s: 1.2.0 Thorough schema inference directly on RDDs of Python dictionaries

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116915#comment-14116915 ] Michael Armbrust commented on SPARK-2870: - Yeah, thanks for pinging me. I've

[jira] [Closed] (SPARK-3317) The loss of regularization in Updater should use the oldWeights

[ https://issues.apache.org/jira/browse/SPARK-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai closed SPARK-3317. -- Resolution: Won't Fix The loss of regularization in Updater should use the oldWeights

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116923#comment-14116923 ] Matei Zaharia commented on SPARK-: -- The slowdown might be partly due to adding

[jira] [Resolved] (SPARK-3320) Batched in-memory column buffer building doesn't work for SchemaRDDs with empty partitions

[ https://issues.apache.org/jira/browse/SPARK-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3320. - Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Cheng Lian Batched

[jira] [Comment Edited] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116935#comment-14116935 ] Davies Liu edited comment on SPARK- at 8/31/14 11:26 PM: -

[jira] [Comment Edited] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116935#comment-14116935 ] Davies Liu edited comment on SPARK- at 8/31/14 11:27 PM: -

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116936#comment-14116936 ] Josh Rosen commented on SPARK-: --- I agree with Davies; I think this is a more general

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116942#comment-14116942 ] Matei Zaharia commented on SPARK-: -- I see, that makes sense. Large number of

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116948#comment-14116948 ] Josh Rosen commented on SPARK-: --- Working on doing some manual bisecting to find the

[jira] [Commented] (SPARK-3332) Tagging is not atomic with launching instances on EC2

[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116950#comment-14116950 ] Allan Douglas R. de Oliveira commented on SPARK-3332: - [~pwendell],

[jira] [Comment Edited] (SPARK-3332) Tagging is not atomic with launching instances on EC2

[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116950#comment-14116950 ] Allan Douglas R. de Oliveira edited comment on SPARK-3332 at 9/1/14 12:49 AM:

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116969#comment-14116969 ] Josh Rosen commented on SPARK-: --- Looks like the issue was introduced somewhere

[jira] [Commented] (SPARK-3168) The ServletContextHandler of webui lacks a SessionManager

[ https://issues.apache.org/jira/browse/SPARK-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116974#comment-14116974 ] meiyoula commented on SPARK-3168: - To use CAS for single sign-on, i add some filters of

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116978#comment-14116978 ] Nicholas Chammas commented on SPARK-: - For the record, I got the OOM in a

[jira] [Comment Edited] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116978#comment-14116978 ] Nicholas Chammas edited comment on SPARK- at 9/1/14 2:09 AM:

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116985#comment-14116985 ] Josh Rosen commented on SPARK-: --- I'll resume work on this later tonight, but just

[jira] [Updated] (SPARK-3334) Spark causes mesos-master memory leak

[ https://issues.apache.org/jira/browse/SPARK-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iven Hsu updated SPARK-3334: Description: The {{akkaFrameSize}} is set to {{Long.MaxValue}} in MesosBackend to workaround SPARK-1112,

[jira] [Created] (SPARK-3334) Spark causes mesos-master memory leak

Iven Hsu created SPARK-3334: --- Summary: Spark causes mesos-master memory leak Key: SPARK-3334 URL: https://issues.apache.org/jira/browse/SPARK-3334 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3334) Spark causes mesos-master memory leak

[ https://issues.apache.org/jira/browse/SPARK-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iven Hsu updated SPARK-3334: Description: The {{akkaFrameSize}} is set to {{Long.MaxValue}} in MesosBackend to workaround SPARK-1112,

[jira] [Assigned] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reassigned SPARK-3324: -- Assignee: Patrick Wendell YARN module has nonstandard structure which cause compile

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117016#comment-14117016 ] Josh Rosen commented on SPARK-: --- It looks like

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117019#comment-14117019 ] Davies Liu commented on SPARK-: --- @joserosen This should not be the culprit, it just

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117021#comment-14117021 ] Josh Rosen commented on SPARK-: --- Good point. I guess culprit was the wrong word,

[jira] [Resolved] (SPARK-2536) Update the MLlib page of Spark website

[ https://issues.apache.org/jira/browse/SPARK-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2536. -- Resolution: Done Update the MLlib page of Spark website

[jira] [Resolved] (SPARK-3205) input format for text records saved with in-record delimiter and newline characters escaped

[ https://issues.apache.org/jira/browse/SPARK-3205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3205. -- Resolution: Later Moved the implementation to https://github.com/mengxr/redshift-input-format.

[jira] [Commented] (SPARK-3090) Avoid not stopping SparkContext with YARN Client mode

[ https://issues.apache.org/jira/browse/SPARK-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117036#comment-14117036 ] Kousuke Saruta commented on SPARK-3090: --- It sound good idea that SparkContext