[jira] [Resolved] (SPARK-24652) Strange ALS Implementation for Implicit Feedback

[ https://issues.apache.org/jira/browse/SPARK-24652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam resolved SPARK-24652. --- Resolution: Not A Problem > Strange ALS Implementation for Implicit Feedback > -

[jira] [Created] (SPARK-24652) Strange ALS Implementation for Implicit Feedback

Jerry Lam created SPARK-24652: - Summary: Strange ALS Implementation for Implicit Feedback Key: SPARK-24652 URL: https://issues.apache.org/jira/browse/SPARK-24652 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073077#comment-16073077 ] Jerry Lam commented on SPARK-21109: --- The schema of the Dataset[my_case] is defined by t

[jira] [Commented] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073071#comment-16073071 ] Jerry Lam commented on SPARK-21109: --- The update doc is unclear because case classes alr

[jira] [Commented] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073066#comment-16073066 ] Jerry Lam commented on SPARK-21109: --- Does the order of the columns part of the schema?

[jira] [Commented] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073046#comment-16073046 ] Jerry Lam commented on SPARK-21109: --- I checked the scala doc for Dataset about union an

[jira] [Commented] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073037#comment-16073037 ] Jerry Lam commented on SPARK-21109: --- When I said they have the same schema is that they

[jira] [Comment Edited] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072989#comment-16072989 ] Jerry Lam edited comment on SPARK-21109 at 7/4/17 12:24 AM: I

[jira] [Reopened] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam reopened SPARK-21109: --- I'm not sure if I understand your reply correctly but both data1 and data2 have the same schema if you p

[jira] [Updated] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam updated SPARK-21109: -- Description: To reproduce the issue: {code} case class my_case(id0: Long, id1: Int, id2: Int, id3: Stri

[jira] [Updated] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

[ https://issues.apache.org/jira/browse/SPARK-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam updated SPARK-21109: -- Description: To reproduce the issue: {code} case class my_case(id0: Long, id1: Int, id2: Int, id3: Stri

[jira] [Created] (SPARK-21109) union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe

Jerry Lam created SPARK-21109: - Summary: union two dataset[A] don't work as expected if one of the datasets is originated from a dataframe Key: SPARK-21109 URL: https://issues.apache.org/jira/browse/SPARK-21109

[jira] [Created] (SPARK-14309) Dataframe returns wrong results due to parsing incorrectly

Jerry Lam created SPARK-14309: - Summary: Dataframe returns wrong results due to parsing incorrectly Key: SPARK-14309 URL: https://issues.apache.org/jira/browse/SPARK-14309 Project: Spark Issue Ty

[jira] [Commented] (SPARK-10951) Support private S3 repositories using spark-submit via --repositories flag

[ https://issues.apache.org/jira/browse/SPARK-10951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060451#comment-15060451 ] Jerry Lam commented on SPARK-10951: --- Any change to have this feature in 1.6? :) > Supp

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045354#comment-15045354 ] Jerry Lam commented on SPARK-8118: -- Hi Justin, thanks for sharing this. Do you know if yo

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14983330#comment-14983330 ] Jerry Lam commented on SPARK-4823: -- Hi [~debasish83], I wonder if this is still work in p

[jira] [Commented] (SPARK-8597) DataFrame partitionBy memory pressure scales extremely poorly

[ https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973531#comment-14973531 ] Jerry Lam commented on SPARK-8597: -- FYI ... The solution described here solves the proble

[jira] [Comment Edited] (SPARK-8890) Reduce memory consumption for dynamic partition insert

[ https://issues.apache.org/jira/browse/SPARK-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973529#comment-14973529 ] Jerry Lam edited comment on SPARK-8890 at 10/26/15 1:02 AM: Hi

[jira] [Comment Edited] (SPARK-8890) Reduce memory consumption for dynamic partition insert

[ https://issues.apache.org/jira/browse/SPARK-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973529#comment-14973529 ] Jerry Lam edited comment on SPARK-8890 at 10/26/15 12:58 AM: -

[jira] [Commented] (SPARK-8890) Reduce memory consumption for dynamic partition insert

[ https://issues.apache.org/jira/browse/SPARK-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973529#comment-14973529 ] Jerry Lam commented on SPARK-8890: -- Hi guys, sorry by injecting comments into the closed

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971496#comment-14971496 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 6:16 PM: Th

[jira] [Commented] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971496#comment-14971496 ] Jerry Lam commented on SPARK-4940: -- Thanks Martin, Yes, beefing up the executor works bu

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971228#comment-14971228 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 4:16 PM: I

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971228#comment-14971228 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 4:15 PM: I

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971228#comment-14971228 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 4:02 PM: I

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971228#comment-14971228 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 4:01 PM: I

[jira] [Commented] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971228#comment-14971228 ] Jerry Lam commented on SPARK-4940: -- I just want to weight in the importance of this issue

[jira] [Comment Edited] (SPARK-10309) Some tasks failed with Unable to acquire memory

[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965510#comment-14965510 ] Jerry Lam edited comment on SPARK-10309 at 10/20/15 6:25 PM: -

[jira] [Commented] (SPARK-10309) Some tasks failed with Unable to acquire memory

[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965510#comment-14965510 ] Jerry Lam commented on SPARK-10309: --- Same issue, I got the following stacktrace: 15/10

[jira] [Updated] (SPARK-10951) Support private S3 repositories using spark-submit via --repositories flag

[ https://issues.apache.org/jira/browse/SPARK-10951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam updated SPARK-10951: -- Summary: Support private S3 repositories using spark-submit via --repositories flag (was: Support priv

[jira] [Updated] (SPARK-10951) Support private S3 pepositories using spark-submit via --repositories flag

[ https://issues.apache.org/jira/browse/SPARK-10951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam updated SPARK-10951: -- Summary: Support private S3 pepositories using spark-submit via --repositories flag (was: Support S3 p

[jira] [Created] (SPARK-10951) Support S3 pepository using spark-submit via --repositories flag

Jerry Lam created SPARK-10951: - Summary: Support S3 pepository using spark-submit via --repositories flag Key: SPARK-10951 URL: https://issues.apache.org/jira/browse/SPARK-10951 Project: Spark I

[jira] [Updated] (SPARK-10731) The head() implementation of dataframe is very slow

[ https://issues.apache.org/jira/browse/SPARK-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam updated SPARK-10731: -- Affects Version/s: 1.4.1 > The head() implementation of dataframe is very slow > --

[jira] [Updated] (SPARK-10731) The head() implementation of dataframe is very slow

[ https://issues.apache.org/jira/browse/SPARK-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam updated SPARK-10731: -- Labels: pyspark (was: ) > The head() implementation of dataframe is very slow > --

[jira] [Created] (SPARK-10731) The head() implementation of dataframe is very slow

Jerry Lam created SPARK-10731: - Summary: The head() implementation of dataframe is very slow Key: SPARK-10731 URL: https://issues.apache.org/jira/browse/SPARK-10731 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876198#comment-14876198 ] Jerry Lam edited comment on SPARK-8118 at 9/18/15 7:30 PM: --- I'm

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876198#comment-14876198 ] Jerry Lam commented on SPARK-8118: -- I'm trying to turn off parquet logging by adding thes

[jira] [Commented] (SPARK-8009) [Mesos] Allow provisioning of executor logging configuration

[ https://issues.apache.org/jira/browse/SPARK-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804676#comment-14804676 ] Jerry Lam commented on SPARK-8009: -- No. I just tested the spark.mesos.uris. Downloading l

[jira] [Commented] (SPARK-4561) PySparkSQL's Row.asDict() should convert nested rows to dictionaries

[ https://issues.apache.org/jira/browse/SPARK-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623628#comment-14623628 ] Jerry Lam commented on SPARK-4561: -- I wonder if this will be fixed soon? > PySparkSQL's

[jira] [Commented] (SPARK-2443) Reading from Partitioned Tables is Slow

[ https://issues.apache.org/jira/browse/SPARK-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060790#comment-14060790 ] Jerry Lam commented on SPARK-2443: -- I wonder if this fix can be easily merged into the cu

[jira] [Created] (SPARK-2448) Table name is not getting applied to their attributes after "registerAsTable"

Jerry Lam created SPARK-2448: Summary: Table name is not getting applied to their attributes after "registerAsTable" Key: SPARK-2448 URL: https://issues.apache.org/jira/browse/SPARK-2448 Project: Spark