[jira] [Assigned] (SPARK-19748) refresh for InMemoryFileIndex with FileStatusCache does not work correctly

[ https://issues.apache.org/jira/browse/SPARK-19748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19748: Assignee: Apache Spark > refresh for InMemoryFileIndex with FileStatusCache does not work

[jira] [Assigned] (SPARK-19748) refresh for InMemoryFileIndex with FileStatusCache does not work correctly

[ https://issues.apache.org/jira/browse/SPARK-19748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19748: Assignee: (was: Apache Spark) > refresh for InMemoryFileIndex with FileStatusCache

[jira] [Commented] (SPARK-19748) refresh for InMemoryFileIndex with FileStatusCache does not work correctly

[ https://issues.apache.org/jira/browse/SPARK-19748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885307#comment-15885307 ] Apache Spark commented on SPARK-19748: -- User 'windpiger' has created a pull request for this issue:

[jira] [Commented] (SPARK-19747) Consolidate code in ML aggregators

[ https://issues.apache.org/jira/browse/SPARK-19747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885298#comment-15885298 ] Nick Pentreath commented on SPARK-19747: Big +1 for this! I agree we really should be able to

[jira] [Updated] (SPARK-19742) When using SparkSession to write a dataset to Hive the schema is ignored

[ https://issues.apache.org/jira/browse/SPARK-19742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navin Goel updated SPARK-19742: --- Summary: When using SparkSession to write a dataset to Hive the schema is ignored (was: When using

[jira] [Created] (SPARK-19748) refresh for InMemoryFileIndex with FileStatusCache does not work correctly

Song Jun created SPARK-19748: Summary: refresh for InMemoryFileIndex with FileStatusCache does not work correctly Key: SPARK-19748 URL: https://issues.apache.org/jira/browse/SPARK-19748 Project: Spark

[jira] [Commented] (SPARK-19747) Consolidate code in ML aggregators

[ https://issues.apache.org/jira/browse/SPARK-19747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885185#comment-15885185 ] Seth Hendrickson commented on SPARK-19747: -- BTW, I have a rough prototype which at least

[jira] [Created] (SPARK-19747) Consolidate code in ML aggregators

Seth Hendrickson created SPARK-19747: Summary: Consolidate code in ML aggregators Key: SPARK-19747 URL: https://issues.apache.org/jira/browse/SPARK-19747 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19741) ClassCastException when using Dataset with type containing value types

[ https://issues.apache.org/jira/browse/SPARK-19741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885171#comment-15885171 ] Kazuaki Ishizaki commented on SPARK-19741: -- Would it be possible to attache a pair of the error

[jira] [Assigned] (SPARK-19746) LogisticAggregator is inefficient in indexing

[ https://issues.apache.org/jira/browse/SPARK-19746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19746: Assignee: (was: Apache Spark) > LogisticAggregator is inefficient in indexing >

[jira] [Assigned] (SPARK-19746) LogisticAggregator is inefficient in indexing

[ https://issues.apache.org/jira/browse/SPARK-19746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19746: Assignee: Apache Spark > LogisticAggregator is inefficient in indexing >

[jira] [Commented] (SPARK-19746) LogisticAggregator is inefficient in indexing

[ https://issues.apache.org/jira/browse/SPARK-19746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885159#comment-15885159 ] Apache Spark commented on SPARK-19746: -- User 'sethah' has created a pull request for this issue:

[jira] [Created] (SPARK-19746) LogisticAggregator is inefficient in indexing

Seth Hendrickson created SPARK-19746: Summary: LogisticAggregator is inefficient in indexing Key: SPARK-19746 URL: https://issues.apache.org/jira/browse/SPARK-19746 Project: Spark Issue

[jira] [Commented] (SPARK-16931) PySpark access to data-frame bucketing api

[ https://issues.apache.org/jira/browse/SPARK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885152#comment-15885152 ] Apache Spark commented on SPARK-16931: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885134#comment-15885134 ] Shawn Lavelle commented on SPARK-13219: --- Is https://issues.apache.org/jira/browse/SPARK-19609

[jira] [Commented] (SPARK-12532) Join-key Pushdown via Predicate Transitivity

[ https://issues.apache.org/jira/browse/SPARK-12532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885133#comment-15885133 ] Shawn Lavelle commented on SPARK-12532: --- is this replaced/superceded by

[jira] [Commented] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885131#comment-15885131 ] Shawn Lavelle commented on SPARK-19730: --- My request here seems related to

[jira] [Assigned] (SPARK-19745) SVCAggregator serializes coefficients

[ https://issues.apache.org/jira/browse/SPARK-19745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19745: Assignee: (was: Apache Spark) > SVCAggregator serializes coefficients >

[jira] [Assigned] (SPARK-19745) SVCAggregator serializes coefficients

[ https://issues.apache.org/jira/browse/SPARK-19745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19745: Assignee: Apache Spark > SVCAggregator serializes coefficients >

[jira] [Commented] (SPARK-19745) SVCAggregator serializes coefficients

[ https://issues.apache.org/jira/browse/SPARK-19745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885122#comment-15885122 ] Apache Spark commented on SPARK-19745: -- User 'sethah' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885108#comment-15885108 ] Shawn Lavelle edited comment on SPARK-19730 at 2/27/17 4:32 AM:

[jira] [Created] (SPARK-19745) SVCAggregator serializes coefficients

Seth Hendrickson created SPARK-19745: Summary: SVCAggregator serializes coefficients Key: SPARK-19745 URL: https://issues.apache.org/jira/browse/SPARK-19745 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14471) The alias created in SELECT could be used in GROUP BY and followed expressions

[ https://issues.apache.org/jira/browse/SPARK-14471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885116#comment-15885116 ] Hyukjin Kwon commented on SPARK-14471: -- Still happens in the current master -

[jira] [Commented] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885108#comment-15885108 ] Shawn Lavelle commented on SPARK-19730: --- Reference: The query is just as simple as "*select * from

[jira] [Resolved] (SPARK-15224) Can not delete jar and list jar in spark Thrift server

[ https://issues.apache.org/jira/browse/SPARK-15224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15224. -- Resolution: Duplicate I am resolving this as a duplicate of SPARK-15763 Please reopen this if

[jira] [Commented] (SPARK-14273) Add FileFormat.isSplittable to indicate whether a format is splittable

[ https://issues.apache.org/jira/browse/SPARK-14273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885086#comment-15885086 ] Hyukjin Kwon commented on SPARK-14273: -- Hi [~yhuai] and [~liancheng], it seems there is

[jira] [Assigned] (SPARK-19727) Spark SQL round function modifies original column

[ https://issues.apache.org/jira/browse/SPARK-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19727: Assignee: Apache Spark > Spark SQL round function modifies original column >

[jira] [Assigned] (SPARK-19727) Spark SQL round function modifies original column

[ https://issues.apache.org/jira/browse/SPARK-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19727: Assignee: (was: Apache Spark) > Spark SQL round function modifies original column >

[jira] [Assigned] (SPARK-19727) Spark SQL round function modifies original column

[ https://issues.apache.org/jira/browse/SPARK-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19727: Assignee: Apache Spark > Spark SQL round function modifies original column >

[jira] [Assigned] (SPARK-19727) Spark SQL round function modifies original column

[ https://issues.apache.org/jira/browse/SPARK-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19727: Assignee: (was: Apache Spark) > Spark SQL round function modifies original column >

[jira] [Commented] (SPARK-19727) Spark SQL round function modifies original column

[ https://issues.apache.org/jira/browse/SPARK-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885029#comment-15885029 ] Apache Spark commented on SPARK-19727: -- User 'wojtek-szymanski' has created a pull request for this

[jira] [Commented] (SPARK-19727) Spark SQL round function modifies original column

[ https://issues.apache.org/jira/browse/SPARK-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885012#comment-15885012 ] Wojciech Szymanski commented on SPARK-19727: I can look at this > Spark SQL round function

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884979#comment-15884979 ] Mridul Muralidharan commented on SPARK-5159: For 1.6.x and earlier, we found that doAs

[jira] [Assigned] (SPARK-16931) PySpark access to data-frame bucketing api

[ https://issues.apache.org/jira/browse/SPARK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16931: Assignee: (was: Apache Spark) > PySpark access to data-frame bucketing api >

[jira] [Assigned] (SPARK-16931) PySpark access to data-frame bucketing api

[ https://issues.apache.org/jira/browse/SPARK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16931: Assignee: Apache Spark > PySpark access to data-frame bucketing api >

[jira] [Assigned] (SPARK-19594) StreamingQueryListener fails to handle QueryTerminatedEvent if more then one listeners exists

[ https://issues.apache.org/jira/browse/SPARK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-19594: Assignee: Eyal Zituny > StreamingQueryListener fails to handle QueryTerminatedEvent if

[jira] [Resolved] (SPARK-19594) StreamingQueryListener fails to handle QueryTerminatedEvent if more then one listeners exists

[ https://issues.apache.org/jira/browse/SPARK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19594. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 >

[jira] [Created] (SPARK-19744) Find a better alternative to netlib-java

Demi Marie Obenour created SPARK-19744: -- Summary: Find a better alternative to netlib-java Key: SPARK-19744 URL: https://issues.apache.org/jira/browse/SPARK-19744 Project: Spark Issue

[jira] [Commented] (SPARK-13802) Fields order in Row(**kwargs) is not consistent with Schema.toInternal method

[ https://issues.apache.org/jira/browse/SPARK-13802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884939#comment-15884939 ] Maciej Szymkiewicz commented on SPARK-13802: [~szymonm] Do you have anything particular in

[jira] [Commented] (SPARK-17265) EdgeRDD Difference throws an exception

[ https://issues.apache.org/jira/browse/SPARK-17265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884936#comment-15884936 ] Joseph K. Bradley commented on SPARK-17265: --- Are you able to post code (and data or generated

[jira] [Commented] (SPARK-13802) Fields order in Row(**kwargs) is not consistent with Schema.toInternal method

[ https://issues.apache.org/jira/browse/SPARK-13802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884919#comment-15884919 ] Szymon Matejczyk commented on SPARK-13802: -- I understand we cannot do anything about

[jira] [Comment Edited] (SPARK-19731) IN Operator should support arrays

[ https://issues.apache.org/jira/browse/SPARK-19731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884904#comment-15884904 ] Shawn Lavelle edited comment on SPARK-19731 at 2/26/17 8:47 PM: I

[jira] [Commented] (SPARK-19731) IN Operator should support arrays

[ https://issues.apache.org/jira/browse/SPARK-19731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884904#comment-15884904 ] Shawn Lavelle commented on SPARK-19731: --- I recognize the possibility of an ambiguity problem, but I

[jira] [Assigned] (SPARK-19348) pyspark.ml.Pipeline gets corrupted under multi threaded use

[ https://issues.apache.org/jira/browse/SPARK-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-19348: - Shepherd: Joseph K. Bradley Assignee: Bryan Cutler

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884892#comment-15884892 ] Vlad Sadilovski edited comment on SPARK-12216 at 2/26/17 7:58 PM: -- I

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884892#comment-15884892 ] Vlad Sadilovski edited comment on SPARK-12216 at 2/26/17 7:57 PM: -- I

[jira] [Commented] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884892#comment-15884892 ] Vlad Sadilovski commented on SPARK-12216: - I found following bug while installing spark on

[jira] [Updated] (SPARK-19743) Exception when creating more than one implicit Encoder in REPL

[ https://issues.apache.org/jira/browse/SPARK-19743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-19743: --- Component/s: (was: SQL) Spark Shell > Exception when creating more than

[jira] [Commented] (SPARK-19743) Exception when creating more than one implicit Encoder in REPL

[ https://issues.apache.org/jira/browse/SPARK-19743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884826#comment-15884826 ] Maciej Bryński commented on SPARK-19743: Possible solution could be to change spark val from

[jira] [Created] (SPARK-19743) Exception when creating more than one implicit Encoder in REPL

Maciej Bryński created SPARK-19743: -- Summary: Exception when creating more than one implicit Encoder in REPL Key: SPARK-19743 URL: https://issues.apache.org/jira/browse/SPARK-19743 Project: Spark

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884809#comment-15884809 ] Kohki Nishio commented on SPARK-19675: -- here's a pr for this issue

[jira] [Assigned] (SPARK-18646) ExecutorClassLoader for spark-shell does not honor spark.executor.userClassPathFirst

[ https://issues.apache.org/jira/browse/SPARK-18646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18646: Assignee: Apache Spark > ExecutorClassLoader for spark-shell does not honor >

[jira] [Commented] (SPARK-18646) ExecutorClassLoader for spark-shell does not honor spark.executor.userClassPathFirst

[ https://issues.apache.org/jira/browse/SPARK-18646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884807#comment-15884807 ] Apache Spark commented on SPARK-18646: -- User 'taroplus' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18646) ExecutorClassLoader for spark-shell does not honor spark.executor.userClassPathFirst

[ https://issues.apache.org/jira/browse/SPARK-18646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18646: Assignee: (was: Apache Spark) > ExecutorClassLoader for spark-shell does not honor >

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884790#comment-15884790 ] Ofir Manor commented on SPARK-18057: Sure - [~marmbrus] - what do you think? it resolved my Kafka

[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884789#comment-15884789 ] jin xing commented on SPARK-19659: -- [~irashid] I've uploaded a design doc, please take a look and give

[jira] [Updated] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger updated SPARK-18057: --- I think you should ask Michael and / or Ryan what their plan is. > Update structured

[jira] [Updated] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated SPARK-19659: - Attachment: SPARK-19659-design-v1.pdf > Fetch big blocks to disk when shuffle-read >

[jira] [Commented] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884770#comment-15884770 ] Herman van Hovell commented on SPARK-19730: --- On a high level I agree that this is useful.

[jira] [Commented] (SPARK-19731) IN Operator should support arrays

[ https://issues.apache.org/jira/browse/SPARK-19731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884763#comment-15884763 ] Herman van Hovell commented on SPARK-19731: --- Why is array_contains not sufficient? {{IN}} by

[jira] [Assigned] (SPARK-19721) Good error message for version mismatch in log files

[ https://issues.apache.org/jira/browse/SPARK-19721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19721: Assignee: Apache Spark > Good error message for version mismatch in log files >

[jira] [Assigned] (SPARK-19721) Good error message for version mismatch in log files

[ https://issues.apache.org/jira/browse/SPARK-19721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19721: Assignee: (was: Apache Spark) > Good error message for version mismatch in log files

[jira] [Commented] (SPARK-19721) Good error message for version mismatch in log files

[ https://issues.apache.org/jira/browse/SPARK-19721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884726#comment-15884726 ] Apache Spark commented on SPARK-19721: -- User 'lw-lin' has created a pull request for this issue:

[jira] [Created] (SPARK-19742) When using SparkSession to write a taset to Hive the schema is ignored

Navin Goel created SPARK-19742: -- Summary: When using SparkSession to write a taset to Hive the schema is ignored Key: SPARK-19742 URL: https://issues.apache.org/jira/browse/SPARK-19742 Project: Spark

[jira] [Created] (SPARK-19741) ClassCastException when using Dataset with type containing value types

Lior Regev created SPARK-19741: -- Summary: ClassCastException when using Dataset with type containing value types Key: SPARK-19741 URL: https://issues.apache.org/jira/browse/SPARK-19741 Project: Spark

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884619#comment-15884619 ] Ofir Manor commented on SPARK-18057: [~c...@koeninger.org] I think my previous issue with upgrading

[jira] [Commented] (SPARK-5984) TimSort broken

[ https://issues.apache.org/jira/browse/SPARK-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884620#comment-15884620 ] Apache Spark commented on SPARK-5984: - User 'xiaoyesoso' has created a pull request for this issue:

[jira] [Updated] (SPARK-19740) Spark executor always runs as root when running on mesos

[ https://issues.apache.org/jira/browse/SPARK-19740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19740: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Spark executor always runs