[jira] [Commented] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503128#comment-15503128 ] Sean Owen commented on SPARK-17593: --- I'm not sure this is a Spark problem. It seems S3 specific. Try

[jira] [Commented] (SPARK-16121) ListingFileCatalog does not list in parallel anymore

[ https://issues.apache.org/jira/browse/SPARK-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503278#comment-15503278 ] Gaurav Shah commented on SPARK-16121: - Thanks [~srowen] > ListingFileCatalog does not list in

[jira] [Issue Comment Deleted] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Shah updated SPARK-17593: Comment: was deleted (was: Thanks [~srowen] my spark code does use `s3n` ) > list files on s3

[jira] [Commented] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503164#comment-15503164 ] Gaurav Shah commented on SPARK-17593: - Thanks [~srowen] my spark code does use `s3n` > list files

[jira] [Created] (SPARK-17593) list files on s3 very slow

Gaurav Shah created SPARK-17593: --- Summary: list files on s3 very slow Key: SPARK-17593 URL: https://issues.apache.org/jira/browse/SPARK-17593 Project: Spark Issue Type: Bug Affects

[jira] [Commented] (SPARK-17582) Dead executors shouldn't show in the SparkUI

[ https://issues.apache.org/jira/browse/SPARK-17582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503461#comment-15503461 ] Thomas Graves commented on SPARK-17582: --- Yes they are meant to be there, status is DEAD. This

[jira] [Commented] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503207#comment-15503207 ] Gaurav Shah commented on SPARK-17593: - Thanks [~srowen] tried after your comment, but that didn't

[jira] [Updated] (SPARK-17585) PySpark SparkContext.addFile supports adding files recursively

[ https://issues.apache.org/jira/browse/SPARK-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17585: Description: Users would like to add a directory as dependency in some cases, they can use

[jira] [Updated] (SPARK-17585) PySpark SparkContext.addFile supports adding files recursively

[ https://issues.apache.org/jira/browse/SPARK-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17585: Description: PySpark {{SparkContext.addFile}} should support adding files recursively under a

[jira] [Issue Comment Deleted] (SPARK-17591) Fix/investigate the failure of tests in Scala On Windows

[ https://issues.apache.org/jira/browse/SPARK-17591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagadeesan A S updated SPARK-17591: --- Comment: was deleted (was: [~hyukjin.kwon] i would like to work on this issue. Maybe I can

[jira] [Commented] (SPARK-17588) java.lang.AssertionError: assertion failed: lapack.dppsv returned 105. when running glm using gaussian link function.

[ https://issues.apache.org/jira/browse/SPARK-17588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502574#comment-15502574 ] Sean Owen commented on SPARK-17588: --- You can ignore the warnings, they're just about netlib falling

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502493#comment-15502493 ] Yun Ni commented on SPARK-5992: --- Hi Joseph, I have made an initial PR based on the design doc:

[jira] [Updated] (SPARK-17558) Bump Hadoop 2.7 version from 2.7.2 to 2.7.3

[ https://issues.apache.org/jira/browse/SPARK-17558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17558: -- Priority: Trivial (was: Major) Issue Type: Improvement (was: New Feature) > Bump Hadoop 2.7

[jira] [Resolved] (SPARK-17558) Bump Hadoop 2.7 version from 2.7.2 to 2.7.3

[ https://issues.apache.org/jira/browse/SPARK-17558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17558. --- Resolution: Fixed Assignee: Steve Loughran (was: Reynold Xin) Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-17163: Assignee: Seth Hendrickson > Merge MLOR into a single LOR interface >

[jira] [Resolved] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-17163. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14834

[jira] [Commented] (SPARK-17597) HiveContext cannot create a table named sort

[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505316#comment-15505316 ] Hyukjin Kwon commented on SPARK-17597: -- BTW, I can't reproduce this against master. It seems this

[jira] [Created] (SPARK-17602) PySpark - Performance Optimization Large Size of Broadcast Variable

Xiao Ming Bao created SPARK-17602: - Summary: PySpark - Performance Optimization Large Size of Broadcast Variable Key: SPARK-17602 URL: https://issues.apache.org/jira/browse/SPARK-17602 Project: Spark

[jira] [Assigned] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17603: Assignee: (was: Apache Spark) > Utilize Hive-generated Statistics For Partitioned

[jira] [Created] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

Xiao Li created SPARK-17603: --- Summary: Utilize Hive-generated Statistics For Partitioned Tables Key: SPARK-17603 URL: https://issues.apache.org/jira/browse/SPARK-17603 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17603: Assignee: Apache Spark > Utilize Hive-generated Statistics For Partitioned Tables >

[jira] [Commented] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505580#comment-15505580 ] Apache Spark commented on SPARK-17603: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-17597) HiveContext cannot create a table named sort

[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505316#comment-15505316 ] Hyukjin Kwon edited comment on SPARK-17597 at 9/20/16 2:15 AM: --- BTW, I

[jira] [Commented] (SPARK-17597) HiveContext cannot create a table named sort

[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505340#comment-15505340 ] Hyukjin Kwon commented on SPARK-17597: -- [~saif.a.ellafi] Do you mind if I ask to confirm that it

[jira] [Comment Edited] (SPARK-17597) HiveContext cannot create a table named sort

[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505316#comment-15505316 ] Hyukjin Kwon edited comment on SPARK-17597 at 9/20/16 2:16 AM: --- BTW, I

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505409#comment-15505409 ] Apache Spark commented on SPARK-17549: -- User 'yhuai' has created a pull request for this issue:

[jira] [Closed] (SPARK-17054) SparkR can not run in yarn-cluster mode on mac os

[ https://issues.apache.org/jira/browse/SPARK-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang closed SPARK-17054. -- Resolution: Won't Fix Close it as it is resolved somewhere else. > SparkR can not run in

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505411#comment-15505411 ] Yin Huai commented on SPARK-17549: -- Forgot to say. Thank you for the investigation! Should we first get

[jira] [Resolved] (SPARK-17160) GetExternalRowField does not properly escape field names, causing generated code not to compile

[ https://issues.apache.org/jira/browse/SPARK-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17160. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-17528) MutableProjection should not cache content from the input row

[ https://issues.apache.org/jira/browse/SPARK-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17528: Target Version/s: 2.1.0 (was: 2.0.1, 2.1.0) > MutableProjection should not cache content from the

[jira] [Resolved] (SPARK-17513) StreamExecution should discard unneeded metadata

[ https://issues.apache.org/jira/browse/SPARK-17513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17513. - Resolution: Fixed Assignee: Frederick Reiss Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505404#comment-15505404 ] Yin Huai commented on SPARK-17549: -- [~vanzin] Let's revert this patch for now. So, this part will be the

[jira] [Updated] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17603: Issue Type: Improvement (was: Bug) > Utilize Hive-generated Statistics For Partitioned Tables >

[jira] [Commented] (SPARK-17527) mergeSchema with `_OPTIONAL_` metadata fails

[ https://issues.apache.org/jira/browse/SPARK-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503595#comment-15503595 ] Liang-Chi Hsieh commented on SPARK-17527: - Of course. > mergeSchema with `_OPTIONAL_` metadata

[jira] [Created] (SPARK-17595) Inefficient selection in Word2VecModel.findSynonyms

William Benton created SPARK-17595: -- Summary: Inefficient selection in Word2VecModel.findSynonyms Key: SPARK-17595 URL: https://issues.apache.org/jira/browse/SPARK-17595 Project: Spark

[jira] [Commented] (SPARK-1018) take and collect don't work on HadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503617#comment-15503617 ] Christophe Bismuth commented on SPARK-1018: --- Hi, I've spent few hours trying to understand why

[jira] [Created] (SPARK-17594) Bug in left-outer join

Virgil Palanciuc created SPARK-17594: Summary: Bug in left-outer join Key: SPARK-17594 URL: https://issues.apache.org/jira/browse/SPARK-17594 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-17594) Bug in left-outer join

[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17594. --- Resolution: Duplicate Have a look through JIRA first please. > Bug in left-outer join >

[jira] [Commented] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503663#comment-15503663 ] Steve Loughran commented on SPARK-17593: Looking at the dir tree, anything you could do to

[jira] [Commented] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503668#comment-15503668 ] Gaurav Shah commented on SPARK-17593: - Thanks [~ste...@apache.org] S3 is definitely slower than hdfs

[jira] [Resolved] (SPARK-17259) Hadoop 2.7 profile to depend on Hadoop 2.7.3

[ https://issues.apache.org/jira/browse/SPARK-17259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-17259. Resolution: Duplicate > Hadoop 2.7 profile to depend on Hadoop 2.7.3 >

[jira] [Commented] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503583#comment-15503583 ] Steve Loughran commented on SPARK-17593: Sean is right: this is primarily S3, or more

[jira] [Updated] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Shah updated SPARK-17593: Description: lets say we have following partitioned data: {code} events_v3 --

[jira] [Commented] (SPARK-17594) Bug in left-outer join

[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503564#comment-15503564 ] Virgil Palanciuc commented on SPARK-17594: -- FWIW, they may not be related; my program works if I

[jira] [Updated] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Shah updated SPARK-17593: Description: lets say we have following partitioned data: {code} events_v3 --

[jira] [Commented] (SPARK-17594) Bug in left-outer join

[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503666#comment-15503666 ] Virgil Palanciuc commented on SPARK-17594: -- Sorry. In my defence, I started to submit the bug

[jira] [Commented] (SPARK-17527) mergeSchema with `_OPTIONAL_` metadata fails

[ https://issues.apache.org/jira/browse/SPARK-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503581#comment-15503581 ] Gaurav Shah commented on SPARK-17527: - Can I do that in two days ? stuck with something else as of

[jira] [Commented] (SPARK-17593) list files on s3 very slow

[ https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503675#comment-15503675 ] Gaurav Shah commented on SPARK-17593: - I definitely agree that flattening out will help, ( not sure

[jira] [Assigned] (SPARK-17595) Inefficient selection in Word2VecModel.findSynonyms

[ https://issues.apache.org/jira/browse/SPARK-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17595: Assignee: (was: Apache Spark) > Inefficient selection in Word2VecModel.findSynonyms >

[jira] [Assigned] (SPARK-14082) Add support for GPU resource when running on Mesos

[ https://issues.apache.org/jira/browse/SPARK-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14082: Assignee: Apache Spark > Add support for GPU resource when running on Mesos >

[jira] [Commented] (SPARK-14082) Add support for GPU resource when running on Mesos

[ https://issues.apache.org/jira/browse/SPARK-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503928#comment-15503928 ] Apache Spark commented on SPARK-14082: -- User 'tnachen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17595) Inefficient selection in Word2VecModel.findSynonyms

[ https://issues.apache.org/jira/browse/SPARK-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17595: Assignee: Apache Spark > Inefficient selection in Word2VecModel.findSynonyms >

[jira] [Updated] (SPARK-17473) jdbc docker tests are failing with java.lang.AbstractMethodError:

[ https://issues.apache.org/jira/browse/SPARK-17473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17473: --- Assignee: Suresh Thalamati > jdbc docker tests are failing with java.lang.AbstractMethodError: >

[jira] [Resolved] (SPARK-17438) Master UI should show the correct core limit when `ApplicationInfo.executorLimit` is set

[ https://issues.apache.org/jira/browse/SPARK-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17438. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Master UI should show the

[jira] [Updated] (SPARK-17589) Fix test case `create external table`

[ https://issues.apache.org/jira/browse/SPARK-17589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17589: - Assignee: Xiao Li > Fix test case `create external table` > - > >

[jira] [Reopened] (SPARK-17594) Bug in left-outer join

[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virgil Palanciuc reopened SPARK-17594: -- Reopening with the proper example > Bug in left-outer join > -- > >

[jira] [Commented] (SPARK-5377) Dynamically add jar into Spark Driver's classpath.

[ https://issues.apache.org/jira/browse/SPARK-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504217#comment-15504217 ] Jon Morra commented on SPARK-5377: -- I would like to revisit this issue as well. Some of our jobs which

[jira] [Resolved] (SPARK-17589) Fix test case `create external table`

[ https://issues.apache.org/jira/browse/SPARK-17589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17589. -- Resolution: Fixed Fix Version/s: 2.0.1 Issue resolved by pull request 15145

[jira] [Assigned] (SPARK-14082) Add support for GPU resource when running on Mesos

[ https://issues.apache.org/jira/browse/SPARK-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14082: Assignee: (was: Apache Spark) > Add support for GPU resource when running on Mesos >

[jira] [Commented] (SPARK-17595) Inefficient selection in Word2VecModel.findSynonyms

[ https://issues.apache.org/jira/browse/SPARK-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503929#comment-15503929 ] Apache Spark commented on SPARK-17595: -- User 'willb' has created a pull request for this issue:

[jira] [Assigned] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5992: --- Assignee: Apache Spark > Locality Sensitive Hashing (LSH) for MLlib >

[jira] [Assigned] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5992: --- Assignee: (was: Apache Spark) > Locality Sensitive Hashing (LSH) for MLlib >

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503971#comment-15503971 ] Apache Spark commented on SPARK-5992: - User 'Yunni' has created a pull request for this issue:

[jira] [Commented] (SPARK-17588) java.lang.AssertionError: assertion failed: lapack.dppsv returned 105. when running glm using gaussian link function.

[ https://issues.apache.org/jira/browse/SPARK-17588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504083#comment-15504083 ] sai pavan kumar chitti commented on SPARK-17588: here is the output of schema().

[jira] [Resolved] (SPARK-17473) jdbc docker tests are failing with java.lang.AbstractMethodError:

[ https://issues.apache.org/jira/browse/SPARK-17473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17473. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Fixed by [~tsuresh]'s PR

[jira] [Created] (SPARK-17596) Streaming job lacks Scala runtime methods

Evgeniy Tsvigun created SPARK-17596: --- Summary: Streaming job lacks Scala runtime methods Key: SPARK-17596 URL: https://issues.apache.org/jira/browse/SPARK-17596 Project: Spark Issue Type:

[jira] [Updated] (SPARK-17594) Bug in left-outer join

[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virgil Palanciuc updated SPARK-17594: - Description: I have a bug where I think a left-join returns wrong results, by mistakenly

[jira] [Updated] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

[ https://issues.apache.org/jira/browse/SPARK-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16295: --- Assignee: (was: Cheng Lian) > Extract SQL programming guide example snippets from source files

[jira] [Resolved] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

[ https://issues.apache.org/jira/browse/SPARK-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-16295. Resolution: Fixed Fix Version/s: 2.0.1 > Extract SQL programming guide example snippets

[jira] [Updated] (SPARK-16439) Incorrect information in SQL Query details

[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16439: --- Assignee: Davies Liu (was: Maciej Bryński) > Incorrect information in SQL Query details >

[jira] [Assigned] (SPARK-17494) Floor function rounds up during join

[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-17494: -- Assignee: Davies Liu > Floor function rounds up during join >

[jira] [Commented] (SPARK-17588) java.lang.AssertionError: assertion failed: lapack.dppsv returned 105. when running glm using gaussian link function.

[ https://issues.apache.org/jira/browse/SPARK-17588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504369#comment-15504369 ] sai pavan kumar chitti commented on SPARK-17588: input is a single csv file of size

[jira] [Commented] (SPARK-16296) add null check for key when create map data in encoder

[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504395#comment-15504395 ] Josh Rosen commented on SPARK-16296: [~cloud_fan], I notice that this issue is targeted at 2.0.1 but

[jira] [Commented] (SPARK-17594) Bug in left-outer join

[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504285#comment-15504285 ] Virgil Palanciuc commented on SPARK-17594: -- It's not - initially my example was about the

[jira] [Commented] (SPARK-17365) Kill multiple executors together to reduce lock contention

[ https://issues.apache.org/jira/browse/SPARK-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504347#comment-15504347 ] Apache Spark commented on SPARK-17365: -- User 'dhruve' has created a pull request for this issue:

[jira] [Commented] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504410#comment-15504410 ] Josh Rosen commented on SPARK-15698: [~jerryshao] [~zsxwing], should 2.0.1 really be a target version

[jira] [Issue Comment Deleted] (SPARK-12635) More efficient (column batch) serialization for Python/R

[ https://issues.apache.org/jira/browse/SPARK-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12635: Comment: was deleted (was: User 'nongli' has created a pull request for this issue:

[jira] [Created] (SPARK-17597) HiveContext cannot create a table named sot

Saif Addin Ellafi created SPARK-17597: - Summary: HiveContext cannot create a table named sot Key: SPARK-17597 URL: https://issues.apache.org/jira/browse/SPARK-17597 Project: Spark Issue

[jira] [Resolved] (SPARK-16439) Incorrect information in SQL Query details

[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16439. Resolution: Fixed Fix Version/s: (was: 2.0.0) 2.2.0

[jira] [Commented] (SPARK-17597) HiveContext cannot create a table named sort

[ https://issues.apache.org/jira/browse/SPARK-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504340#comment-15504340 ] Saif Addin Ellafi commented on SPARK-17597: --- Regarded it as a problem since the table actually

[jira] [Commented] (SPARK-17594) Bug in left-outer join

[ https://issues.apache.org/jira/browse/SPARK-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504331#comment-15504331 ] Virgil Palanciuc commented on SPARK-17594: -- Hmmm.. no, I can't reproduce it with 2.1.0-SNAPSHOT.

[jira] [Updated] (SPARK-16439) Incorrect information in SQL Query details

[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16439: --- Fix Version/s: (was: 2.2.0) 2.1.0 > Incorrect information in SQL Query

[jira] [Commented] (SPARK-17596) Streaming job lacks Scala runtime methods

[ https://issues.apache.org/jira/browse/SPARK-17596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504337#comment-15504337 ] Sean Owen commented on SPARK-17596: --- This sounds like a Scala version mismatch problem, or a packaging

[jira] [Updated] (SPARK-16323) Avoid unnecessary cast when doing integral divide

[ https://issues.apache.org/jira/browse/SPARK-16323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16323: --- Target Version/s: 2.1.0 (was: 2.0.1, 2.1.0) > Avoid unnecessary cast when doing integral divide >

[jira] [Commented] (SPARK-16323) Avoid unnecessary cast when doing integral divide

[ https://issues.apache.org/jira/browse/SPARK-16323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504403#comment-15504403 ] Josh Rosen commented on SPARK-16323: FYI I'm going to untarget this from 2.0.1 because this is only a

[jira] [Updated] (SPARK-17494) Floor/ceil of decimal returns wrong result if it's in compact format

[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-17494: --- Summary: Floor/ceil of decimal returns wrong result if it's in compact format (was: Floor function

[jira] [Commented] (SPARK-17494) Floor/ceil of decimal returns wrong result if it's in compact format

[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504803#comment-15504803 ] Apache Spark commented on SPARK-17494: -- User 'davies' has created a pull request for this issue:

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504811#comment-15504811 ] Apache Spark commented on SPARK-17477: -- User 'wgtmac' has created a pull request for this issue:

[jira] [Updated] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-17100: --- Fix Version/s: (was: 2.2.0) 2.1.0 > pyspark filter on a udf column after join

[jira] [Updated] (SPARK-17057) ProbabilisticClassifierModels' thresholds should be > 0 and sum < 1 to match randomForest cutoff

[ https://issues.apache.org/jira/browse/SPARK-17057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17057: -- Assignee: Sean Owen Affects Version/s: 2.0.0 Priority: Minor (was: Major)

[jira] [Commented] (SPARK-17563) Add org/apache/spark/JavaSparkListener to make Spark-2.0.0 work with Hive-2.X.X

[ https://issues.apache.org/jira/browse/SPARK-17563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502698#comment-15502698 ] Oleksiy Sayankin commented on SPARK-17563: -- Found existing issue

[jira] [Commented] (SPARK-17582) Dead executors shouldn't show in the SparkUI

[ https://issues.apache.org/jira/browse/SPARK-17582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502770#comment-15502770 ] Sean Owen commented on SPARK-17582: --- I think they are supposed to, so you can see what happened to

[jira] [Updated] (SPARK-17297) Clarify window/slide duration as absolute time, not relative to a calendar

[ https://issues.apache.org/jira/browse/SPARK-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17297: -- Priority: Trivial (was: Minor) > Clarify window/slide duration as absolute time, not relative to a

[jira] [Resolved] (SPARK-17297) Clarify window/slide duration as absolute time, not relative to a calendar

[ https://issues.apache.org/jira/browse/SPARK-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17297. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-9862) Join: Handling data skew

[ https://issues.apache.org/jira/browse/SPARK-9862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502799#comment-15502799 ] wangyuhu commented on SPARK-9862: - I‘m working on this > Join: Handling data skew >

[jira] [Commented] (SPARK-17549) InMemoryRelation doesn't scale to large tables

[ https://issues.apache.org/jira/browse/SPARK-17549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505060#comment-15505060 ] Marcelo Vanzin commented on SPARK-17549: Replying to myself: yes, this seems to be a real

[jira] [Commented] (SPARK-17601) SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long

[ https://issues.apache.org/jira/browse/SPARK-17601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505066#comment-15505066 ] Hyukjin Kwon commented on SPARK-17601: -- We might have to avoid to open multiple related issues. I

[jira] [Resolved] (SPARK-16296) add null check for key when create map data in encoder

[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16296. - Resolution: Fixed Fix Version/s: 2.1.0 Target Version/s: 2.1.0 (was: 2.0.1) >

[jira] [Commented] (SPARK-16296) add null check for key when create map data in encoder

[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505099#comment-15505099 ] Wenchen Fan commented on SPARK-16296: - This is a minor issue and it's hard to fix it without

[jira] [Commented] (SPARK-10815) API design: data sources and sinks

[ https://issues.apache.org/jira/browse/SPARK-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505102#comment-15505102 ] Frederick Reiss commented on SPARK-10815: - I'm confused by the current description of this task.

  1   2   >