[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440786#comment-15440786 ] Miao Wang commented on SPARK-17110: --- I set up a two-node cluster, one master, one worker, 48 cores. 1G

[jira] [Updated] (SPARK-17276) Stop environment parameters flooding Jenkins build output

[ https://issues.apache.org/jira/browse/SPARK-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-17276: Attachment: Screen Shot 2016-08-26 at 10.52.07 PM.png > Stop environment parameters flooding Jenkins build

[jira] [Commented] (SPARK-17276) Stop environment parameters flooding Jenkins build output

[ https://issues.apache.org/jira/browse/SPARK-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440782#comment-15440782 ] Xin Ren commented on SPARK-17276: - I'm working on it. > Stop environment parameters flooding Jenkins

[jira] [Created] (SPARK-17276) Stop environment parameters flooding Jenkins build output

Xin Ren created SPARK-17276: --- Summary: Stop environment parameters flooding Jenkins build output Key: SPARK-17276 URL: https://issues.apache.org/jira/browse/SPARK-17276 Project: Spark Issue Type:

[jira] [Commented] (SPARK-17274) Move join optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440750#comment-15440750 ] Apache Spark commented on SPARK-17274: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17274) Move join optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17274: Assignee: Reynold Xin (was: Apache Spark) > Move join optimizer rules into a separate

[jira] [Assigned] (SPARK-17274) Move join optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17274: Assignee: Apache Spark (was: Reynold Xin) > Move join optimizer rules into a separate

[jira] [Commented] (SPARK-17275) Flaky test: org.apache.spark.deploy.RPackageUtilsSuite.jars that don't exist are skipped and print warning

[ https://issues.apache.org/jira/browse/SPARK-17275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440747#comment-15440747 ] Yin Huai commented on SPARK-17275: -- cc [~felixcheung] [~shivaram] > Flaky test:

[jira] [Created] (SPARK-17275) Flaky test: org.apache.spark.deploy.RPackageUtilsSuite.jars that don't exist are skipped and print warning

Yin Huai created SPARK-17275: Summary: Flaky test: org.apache.spark.deploy.RPackageUtilsSuite.jars that don't exist are skipped and print warning Key: SPARK-17275 URL:

[jira] [Created] (SPARK-17274) Move join optimizer rules into a separate file

Reynold Xin created SPARK-17274: --- Summary: Move join optimizer rules into a separate file Key: SPARK-17274 URL: https://issues.apache.org/jira/browse/SPARK-17274 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-17273) Move expression optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17273: Assignee: Reynold Xin (was: Apache Spark) > Move expression optimizer rules into a

[jira] [Commented] (SPARK-17273) Move expression optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440742#comment-15440742 ] Apache Spark commented on SPARK-17273: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17273) Move expression optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17273: Assignee: Apache Spark (was: Reynold Xin) > Move expression optimizer rules into a

[jira] [Assigned] (SPARK-17272) Move subquery optimizer rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17272: Assignee: Apache Spark (was: Reynold Xin) > Move subquery optimizer rules into its own

[jira] [Assigned] (SPARK-17272) Move subquery optimizer rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17272: Assignee: Reynold Xin (was: Apache Spark) > Move subquery optimizer rules into its own

[jira] [Created] (SPARK-17273) Move expression optimizer rules into a separate file

Reynold Xin created SPARK-17273: --- Summary: Move expression optimizer rules into a separate file Key: SPARK-17273 URL: https://issues.apache.org/jira/browse/SPARK-17273 Project: Spark Issue

[jira] [Commented] (SPARK-17272) Move subquery optimizer rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440733#comment-15440733 ] Apache Spark commented on SPARK-17272: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-17272) Move subquery optimizer rules into its own file

Reynold Xin created SPARK-17272: --- Summary: Move subquery optimizer rules into its own file Key: SPARK-17272 URL: https://issues.apache.org/jira/browse/SPARK-17272 Project: Spark Issue Type:

[jira] [Commented] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440725#comment-15440725 ] Apache Spark commented on SPARK-17270: -- User 'rxin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-17269) Move finish analysis stage into its own file

[ https://issues.apache.org/jira/browse/SPARK-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17269. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Move finish analysis

[jira] [Resolved] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17270. -- Resolution: Fixed Fix Version/s: 2.1.0 https://github.com/apache/spark/pull/14839 has been

[jira] [Updated] (SPARK-10747) add support for NULLS FIRST|LAST in ORDER BY clause

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-10747: --- Summary: add support for NULLS FIRST|LAST in ORDER BY clause (was: add support for window specification to

[jira] [Updated] (SPARK-17268) Break Optimizer.scala apart

[ https://issues.apache.org/jira/browse/SPARK-17268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17268: Description: Optimizer.scala has become too large to maintain. We would need to break it apart

[jira] [Commented] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440656#comment-15440656 ] Xin Wu commented on SPARK-10747: This JIRA may be changed to support NULLS FIRST|LAST feature in ORDER BY

[jira] [Assigned] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10747: Assignee: Apache Spark > add support for window specification to include how NULLS are

[jira] [Commented] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440651#comment-15440651 ] Apache Spark commented on SPARK-10747: -- User 'xwu0226' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10747: Assignee: (was: Apache Spark) > add support for window specification to include how

[jira] [Updated] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-10747: --- Issue Type: New Feature (was: Improvement) > add support for window specification to include how NULLS are

[jira] [Commented] (SPARK-16963) Change Source API so that sources do not need to keep unbounded state

[ https://issues.apache.org/jira/browse/SPARK-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440598#comment-15440598 ] Frederick Reiss commented on SPARK-16963: - Updated the pull request to address some conflicting

[jira] [Assigned] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17271: Assignee: Apache Spark > Planner adds un-necessary Sort even if child ordering is

[jira] [Assigned] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17271: Assignee: (was: Apache Spark) > Planner adds un-necessary Sort even if child ordering

[jira] [Commented] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440557#comment-15440557 ] Apache Spark commented on SPARK-17271: -- User 'tejasapatil' has created a pull request for this

[jira] [Updated] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-17271: Description: Found a case when the planner is adding un-needed SORT operation due to bug in the

[jira] [Commented] (SPARK-16216) CSV data source does not write date and timestamp correctly

[ https://issues.apache.org/jira/browse/SPARK-16216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440546#comment-15440546 ] Apache Spark commented on SPARK-16216: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17266. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14837

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440505#comment-15440505 ] Sun Rui commented on SPARK-13525: - What's your spark cluster deployment mode? yarn or standalone? >

[jira] [Updated] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17266: - Assignee: Yin Huai > PrefixComparatorsSuite's "String prefix comparator" failed when both input >

[jira] [Created] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

Tejas Patil created SPARK-17271: --- Summary: Planner adds un-necessary Sort even if child ordering is semantically same as required ordering Key: SPARK-17271 URL: https://issues.apache.org/jira/browse/SPARK-17271

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440476#comment-15440476 ] Sun Rui commented on SPARK-13525: - Another guess: could you check "localhost" works for local TCP

[jira] [Commented] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440368#comment-15440368 ] Apache Spark commented on SPARK-17270: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17270: Assignee: Reynold Xin (was: Apache Spark) > Move object optimization rules into its own

[jira] [Assigned] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17270: Assignee: Apache Spark (was: Reynold Xin) > Move object optimization rules into its own

[jira] [Created] (SPARK-17270) Move object optimization rules into its own file

Reynold Xin created SPARK-17270: --- Summary: Move object optimization rules into its own file Key: SPARK-17270 URL: https://issues.apache.org/jira/browse/SPARK-17270 Project: Spark Issue Type:

[jira] [Commented] (SPARK-17269) Move finish analysis stage into its own file

[ https://issues.apache.org/jira/browse/SPARK-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440359#comment-15440359 ] Apache Spark commented on SPARK-17269: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17269) Move finish analysis stage into its own file

[ https://issues.apache.org/jira/browse/SPARK-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17269: Assignee: Reynold Xin (was: Apache Spark) > Move finish analysis stage into its own file

[jira] [Assigned] (SPARK-17269) Move finish analysis stage into its own file

[ https://issues.apache.org/jira/browse/SPARK-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17269: Assignee: Apache Spark (was: Reynold Xin) > Move finish analysis stage into its own file

[jira] [Created] (SPARK-17269) Move finish analysis stage into its own file

Reynold Xin created SPARK-17269: --- Summary: Move finish analysis stage into its own file Key: SPARK-17269 URL: https://issues.apache.org/jira/browse/SPARK-17269 Project: Spark Issue Type:

[jira] [Created] (SPARK-17268) Break Optimizer.scala apart

Reynold Xin created SPARK-17268: --- Summary: Break Optimizer.scala apart Key: SPARK-17268 URL: https://issues.apache.org/jira/browse/SPARK-17268 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-17244) Joins should not pushdown non-deterministic conditions

[ https://issues.apache.org/jira/browse/SPARK-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17244. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull request

[jira] [Updated] (SPARK-17244) Joins should not pushdown non-deterministic conditions

[ https://issues.apache.org/jira/browse/SPARK-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17244: - Assignee: Sameer Agarwal > Joins should not pushdown non-deterministic conditions >

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440183#comment-15440183 ] DB Tsai commented on SPARK-17163: - It relates to this [SPARK-17201], but seems that it's not a concern.

[jira] [Commented] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440146#comment-15440146 ] Apache Spark commented on SPARK-17266: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17266: Assignee: (was: Apache Spark) > PrefixComparatorsSuite's "String prefix comparator"

[jira] [Assigned] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17266: Assignee: Apache Spark > PrefixComparatorsSuite's "String prefix comparator" failed when

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440142#comment-15440142 ] Joseph K. Bradley commented on SPARK-17163: --- I was guessing that optimization would be more

[jira] [Updated] (SPARK-17267) Long running structured streaming requirements

[ https://issues.apache.org/jira/browse/SPARK-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17267: Priority: Blocker (was: Major) > Long running structured streaming requirements >

[jira] [Updated] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15698: Issue Type: Sub-task (was: Improvement) Parent: SPARK-17267 > Ability to remove old

[jira] [Updated] (SPARK-17235) MetadataLog should support purging old logs

[ https://issues.apache.org/jira/browse/SPARK-17235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17235: Issue Type: Sub-task (was: New Feature) Parent: SPARK-17267 > MetadataLog should support

[jira] [Updated] (SPARK-17165) FileStreamSource should not track the list of seen files indefinitely

[ https://issues.apache.org/jira/browse/SPARK-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17165: Issue Type: Sub-task (was: Bug) Parent: SPARK-17267 > FileStreamSource should not track

[jira] [Created] (SPARK-17267) Long running structured streaming requirements

Reynold Xin created SPARK-17267: --- Summary: Long running structured streaming requirements Key: SPARK-17267 URL: https://issues.apache.org/jira/browse/SPARK-17267 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-17235) MetadataLog should support purging old logs

[ https://issues.apache.org/jira/browse/SPARK-17235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17235. - Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.1.0

[jira] [Created] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

Yin Huai created SPARK-17266: Summary: PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings Key: SPARK-17266 URL: https://issues.apache.org/jira/browse/SPARK-17266

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440090#comment-15440090 ] Arihanth Jain commented on SPARK-13525: --- [~sunrui] I have tried "spark.sparkr.use.daemon" to false

[jira] [Commented] (SPARK-17044) Add window function test in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439925#comment-15439925 ] Dongjoon Hyun commented on SPARK-17044: --- Hi, [~rxin]. Could you review this issue? > Add window

[jira] [Updated] (SPARK-17044) Add window function test in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17044: -- Description: This issue adds a SQL query test for Window functions for new

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439857#comment-15439857 ] Apache Spark commented on SPARK-17243: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17243: Assignee: (was: Apache Spark) > Spark 2.0 history server summary page gets stuck at

[jira] [Assigned] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17243: Assignee: Apache Spark > Spark 2.0 history server summary page gets stuck at "loading

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439799#comment-15439799 ] Alex Bozarth commented on SPARK-17243: -- So I decided to work on this as a short break from my

[jira] [Commented] (SPARK-15882) Discuss distributed linear algebra in spark.ml package

[ https://issues.apache.org/jira/browse/SPARK-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439767#comment-15439767 ] Joseph K. Bradley commented on SPARK-15882: --- This seems like an important feature, but less

[jira] [Resolved] (SPARK-17246) Support BigDecimal literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17246. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Support BigDecimal

[jira] [Resolved] (SPARK-16967) Collect Mesos support code into a module/profile

[ https://issues.apache.org/jira/browse/SPARK-16967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16967. Resolution: Fixed Assignee: Michael Gummelt Fix Version/s: 2.1.0 > Collect

[jira] [Created] (SPARK-17265) EdgeRDD Difference throws an exception

Shishir Kharel created SPARK-17265: -- Summary: EdgeRDD Difference throws an exception Key: SPARK-17265 URL: https://issues.apache.org/jira/browse/SPARK-17265 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439613#comment-15439613 ] Dongjoon Hyun commented on SPARK-17261: --- Hi, [~dakghar] For me, those seems not to work even in

[jira] [Updated] (SPARK-17220) Upgrade Py4J to 0.10.3

[ https://issues.apache.org/jira/browse/SPARK-17220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-17220: - Component/s: PySpark > Upgrade Py4J to 0.10.3 > -- > >

[jira] [Updated] (SPARK-17265) EdgeRDD Difference throws an exception

[ https://issues.apache.org/jira/browse/SPARK-17265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shishir Kharel updated SPARK-17265: --- Description: Subtracting two edge RDD throws and exception. val difference =

[jira] [Commented] (SPARK-16501) spark.mesos.secret exposed on UI and command line

[ https://issues.apache.org/jira/browse/SPARK-16501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439609#comment-15439609 ] Eric Daniel commented on SPARK-16501: - Great to know, thanks! > spark.mesos.secret exposed on UI and

[jira] [Resolved] (SPARK-17207) Comparing Vector in relative tolerance or absolute tolerance in UnitTests error

[ https://issues.apache.org/jira/browse/SPARK-17207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-17207. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14785

[jira] [Updated] (SPARK-17165) FileStreamSource should not track the list of seen files indefinitely

[ https://issues.apache.org/jira/browse/SPARK-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17165: - Fix Version/s: 2.1.0 2.0.1 > FileStreamSource should not track the list of

[jira] [Resolved] (SPARK-17165) FileStreamSource should not track the list of seen files indefinitely

[ https://issues.apache.org/jira/browse/SPARK-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17165. -- Resolution: Fixed Assignee: Peter Lee > FileStreamSource should not track the list of

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439517#comment-15439517 ] Gang Wu commented on SPARK-17243: - Thanks [~ajbozarth]! Let me know when it is done. > Spark 2.0 history

[jira] [Updated] (SPARK-17250) Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

[ https://issues.apache.org/jira/browse/SPARK-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17250: - Assignee: Xiao Li (was: Apache Spark) > Remove HiveClient and setCurrentDatabase from

[jira] [Resolved] (SPARK-17250) Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

[ https://issues.apache.org/jira/browse/SPARK-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17250. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14821

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439499#comment-15439499 ] Seth Hendrickson commented on SPARK-17163: -- [~dbtsai] We can discuss these design points on the

[jira] [Updated] (SPARK-17192) Issuing an exception when users specify the partitioning columns without a given schema

[ https://issues.apache.org/jira/browse/SPARK-17192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17192: - Assignee: Xiao Li > Issuing an exception when users specify the partitioning columns without a > given

[jira] [Resolved] (SPARK-17192) Issuing an exception when users specify the partitioning columns without a given schema

[ https://issues.apache.org/jira/browse/SPARK-17192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17192. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14572

[jira] [Commented] (SPARK-17252) Performing arithmetic in VALUES can lead to ClassCastException / MatchErrors during query parsing

[ https://issues.apache.org/jira/browse/SPARK-17252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439478#comment-15439478 ] Josh Rosen commented on SPARK-17252: Looks like this issue only affects 2.0.0, so I'm going to

[jira] [Resolved] (SPARK-17252) Performing arithmetic in VALUES can lead to ClassCastException / MatchErrors during query parsing

[ https://issues.apache.org/jira/browse/SPARK-17252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17252. Resolution: Fixed Fix Version/s: 2.0.1 > Performing arithmetic in VALUES can lead to

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439475#comment-15439475 ] Apache Spark commented on SPARK-17163: -- User 'sethah' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17163: Assignee: Apache Spark > Merge MLOR into a single LOR interface >

[jira] [Assigned] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17163: Assignee: (was: Apache Spark) > Merge MLOR into a single LOR interface >

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439470#comment-15439470 ] Alex Bozarth commented on SPARK-17243: -- Thanks [~ste...@apache.org], this idea is great. [~wgtmac],

[jira] [Commented] (SPARK-17165) FileStreamSource should not track the list of seen files indefinitely

[ https://issues.apache.org/jira/browse/SPARK-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439391#comment-15439391 ] Frederick Reiss commented on SPARK-17165: - This problem is actually deeper than just

[jira] [Commented] (SPARK-17251) "ClassCastException: OuterReference cannot be cast to NamedExpression" for correlated subquery on the RHS of an IN operator

[ https://issues.apache.org/jira/browse/SPARK-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439360#comment-15439360 ] Herman van Hovell commented on SPARK-17251: --- Ok, I have taken a look at this one. We should

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439352#comment-15439352 ] Steve Loughran commented on SPARK-17243: The REST API actually lets you set a time range for

[jira] [Commented] (SPARK-16998) select($"column1", explode($"column2")) is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439336#comment-15439336 ] Takeshi Yamamuro commented on SPARK-16998: -- can we link this ticket to SPARK-15214? >

[jira] [Commented] (SPARK-16998) select($"column1", explode($"column2")) is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439331#comment-15439331 ] Takeshi Yamamuro commented on SPARK-16998: -- yea, no problem. thanks! > select($"column1",

[jira] [Resolved] (SPARK-17260) move CreateTables to HiveStrategies

[ https://issues.apache.org/jira/browse/SPARK-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17260. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14825

[jira] [Commented] (SPARK-16998) select($"column1", explode($"column2")) is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439276#comment-15439276 ] Herman van Hovell commented on SPARK-16998: --- [~maropu] Do you mind if I do it myself? I already

[jira] [Commented] (SPARK-17252) Performing arithmetic in VALUES can lead to ClassCastException / MatchErrors during query parsing

[ https://issues.apache.org/jira/browse/SPARK-17252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439269#comment-15439269 ] Herman van Hovell commented on SPARK-17252: --- I cannot reproduce this. I tried both on the

  1   2   >