[jira] [Commented] (SPARK-5841) Memory leak in DiskBlockManager

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327086#comment-14327086 ] Apache Spark commented on SPARK-5841: - User 'nishkamravi2' has created a pull request

[jira] [Updated] (SPARK-5906) Input read size can be incorrect when using Hadoop version <2.5

2015-02-18 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-5906: -- Description: When SparkSQL reads input data from parquet, there are many cases where it doesn't

[jira] [Updated] (SPARK-5906) Input read size can be incorrect when using Hadoop version <2.5

2015-02-18 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-5906: -- Summary: Input read size can be incorrect when using Hadoop version <2.5 (was: Input read size

[jira] [Updated] (SPARK-5906) Input read size incorrect when using Hadoop version <2.5

2015-02-18 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-5906: -- Description: When SparkSQL reads input data from parquet, there are many cases where it doesn't

[jira] [Closed] (SPARK-5906) Input read size incorrect for Parquet files

2015-02-18 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout closed SPARK-5906. - Resolution: Won't Fix Ah I see -- then I'm closing this, because I'm using Hadoop 2.0, which is wh

[jira] [Updated] (SPARK-5906) Input read size incorrect when using Hadoop version <2.5

2015-02-18 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-5906: -- Summary: Input read size incorrect when using Hadoop version <2.5 (was: Input read size incorre

[jira] [Commented] (SPARK-5881) RDD remains cached after the table gets overridden by "CACHE TABLE"

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327056#comment-14327056 ] Apache Spark commented on SPARK-5881: - User 'yhuai' has created a pull request for thi

[jira] [Commented] (SPARK-5342) Allow long running Spark apps to run on secure YARN/HDFS

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327055#comment-14327055 ] Apache Spark commented on SPARK-5342: - User 'harishreedharan' has created a pull reque

[jira] [Commented] (SPARK-5751) Flaky test: o.a.s.sql.hive.thriftserver.HiveThriftServer2Suite sometimes times out

2015-02-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327053#comment-14327053 ] Cheng Lian commented on SPARK-5751: --- Hey [~joshrosen], thanks for the investigation! I c

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327038#comment-14327038 ] Apache Spark commented on SPARK-1537: - User 'zhzhan' has created a pull request for th

[jira] [Assigned] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-5904: -- Assignee: Reynold Xin > DataFrame methods with varargs do not work in Java > --

[jira] [Updated] (SPARK-5859) fix Data Frame Python API

2015-02-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5859: --- Assignee: Davies Liu > fix Data Frame Python API > -- > > Key

[jira] [Commented] (SPARK-5751) Flaky test: o.a.s.sql.hive.thriftserver.HiveThriftServer2Suite sometimes times out

2015-02-18 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327017#comment-14327017 ] Josh Rosen commented on SPARK-5751: --- I took a brief look at this today and have a few th

[jira] [Commented] (SPARK-5906) Input read size incorrect for Parquet files

2015-02-18 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327013#comment-14327013 ] Sandy Ryza commented on SPARK-5906: --- Ah yeah always an infinite source of confusion. I

[jira] [Commented] (SPARK-5906) Input read size incorrect for Parquet files

2015-02-18 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327010#comment-14327010 ] Kay Ousterhout commented on SPARK-5906: --- Ah sorry I was conflating NewHadoopRDD with

[jira] [Commented] (SPARK-5906) Input read size incorrect for Parquet files

2015-02-18 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327007#comment-14327007 ] Sandy Ryza commented on SPARK-5906: --- That stack trace doesn't necessarily indicate to me

[jira] [Commented] (SPARK-5906) Input read size incorrect for Parquet files

2015-02-18 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327004#comment-14327004 ] Kay Ousterhout commented on SPARK-5906: --- Here's one stack trace from one of the Spar

[jira] [Commented] (SPARK-5906) Input read size incorrect for Parquet files

2015-02-18 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326992#comment-14326992 ] Sandy Ryza commented on SPARK-5906: --- Hmm, that's definitely not the expected behavior.

[jira] [Created] (SPARK-5906) Input read size incorrect for Parquet files

2015-02-18 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-5906: - Summary: Input read size incorrect for Parquet files Key: SPARK-5906 URL: https://issues.apache.org/jira/browse/SPARK-5906 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5832) Add Affinity Propagation clustering algorithm

2015-02-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326951#comment-14326951 ] Liang-Chi Hsieh commented on SPARK-5832: For clustering algorithm, O(N^2) complexi

[jira] [Updated] (SPARK-5076) Don't show "Cores" or "Memory Per Node" columns for completed applications

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5076: - Affects Version/s: 1.0.0 > Don't show "Cores" or "Memory Per Node" columns for completed applications > --

[jira] [Commented] (SPARK-5816) Add huge backward compatibility warning in DriverWrapper

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326942#comment-14326942 ] Apache Spark commented on SPARK-5816: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-5669) Spark assembly includes incompatibly licensed libgfortran, libgcc code via JBLAS

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326940#comment-14326940 ] Xiangrui Meng commented on SPARK-5669: -- [~srowen] I came across this "GCC Runtime Lib

[jira] [Commented] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326937#comment-14326937 ] Apache Spark commented on SPARK-5904: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-5533) Replace explicit dependency on org.codehaus.jackson

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326932#comment-14326932 ] Andrew Or commented on SPARK-5533: -- Ok, fair enough. I'm closing this as won't fix. > Re

[jira] [Closed] (SPARK-5533) Replace explicit dependency on org.codehaus.jackson

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5533. Resolution: Won't Fix > Replace explicit dependency on org.codehaus.jackson > --

[jira] [Updated] (SPARK-4571) History server shows negative time

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4571: - Priority: Major (was: Minor) > History server shows negative time > -- >

[jira] [Updated] (SPARK-4905) Flaky test: o.a.s.streaming.flume.FlumeStreamSuite.flume input stream

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4905: - Priority: Critical (was: Major) > Flaky test: o.a.s.streaming.flume.FlumeStreamSuite.flume input stream >

[jira] [Updated] (SPARK-4905) Flaky test: o.a.s.streaming.flume.FlumeStreamSuite.flume input stream

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4905: - Summary: Flaky test: o.a.s.streaming.flume.FlumeStreamSuite.flume input stream (was: Flaky FlumeStreamSui

[jira] [Updated] (SPARK-5074) Flaky test: o.a.s.scheduler.DAGSchedulerSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5074: - Priority: Critical (was: Minor) > Flaky test: o.a.s.scheduler.DAGSchedulerSuite > ---

[jira] [Updated] (SPARK-5083) Flaky test: o.a.s.scheduler.TaskResultGetterSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5083: - Priority: Critical (was: Minor) > Flaky test: o.a.s.scheduler.TaskResultGetterSuite > ---

[jira] [Updated] (SPARK-5083) Flaky test: o.a.s.scheduler.TaskResultGetterSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5083: - Summary: Flaky test: o.a.s.scheduler.TaskResultGetterSuite (was: Fix a flaky test in TaskResultGetterSuit

[jira] [Updated] (SPARK-5153) Flaky test: o.a.s.streaming.kafka.ReliableKafkaStreamSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5153: - Priority: Critical (was: Major) > Flaky test: o.a.s.streaming.kafka.ReliableKafkaStreamSuite > --

[jira] [Updated] (SPARK-5074) Flaky test: o.a.s.scheduler.DAGSchedulerSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5074: - Summary: Flaky test: o.a.s.scheduler.DAGSchedulerSuite (was: Fix a non-deterministic test in org.apache.s

[jira] [Updated] (SPARK-5153) Flaky test: o.a.s.streaming.kafka.ReliableKafkaStreamSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5153: - Summary: Flaky test: o.a.s.streaming.kafka.ReliableKafkaStreamSuite (was: flaky test of "Reliable Kafka i

[jira] [Commented] (SPARK-5345) Flaky test: o.a.s.deploy.history.FsHistoryProviderSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326911#comment-14326911 ] Andrew Or commented on SPARK-5345: -- The real fix should be here: https://github.com/apach

[jira] [Updated] (SPARK-5548) Flaky test: o.a.s.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5548: - Fix Version/s: (was: 1.3.0) > Flaky test: o.a.s.util.AkkaUtilsSuite.remote fetch ssl on - untrusted se

[jira] [Updated] (SPARK-5345) Flaky test: o.a.s.deploy.history.FsHistoryProviderSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5345: - Assignee: Kousuke Saruta > Flaky test: o.a.s.deploy.history.FsHistoryProviderSuite > -

[jira] [Updated] (SPARK-5548) Flaky test: o.a.s.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5548: - Target Version/s: (was: 1.3.0) > Flaky test: o.a.s.util.AkkaUtilsSuite.remote fetch ssl on - untrusted s

[jira] [Updated] (SPARK-5345) Flaky test: o.a.s.deploy.history.FsHistoryProviderSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5345: - Priority: Critical (was: Major) > Flaky test: o.a.s.deploy.history.FsHistoryProviderSuite > -

[jira] [Updated] (SPARK-5345) Flaky test: o.a.s.deploy.history.FsHistoryProviderSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5345: - Summary: Flaky test: o.a.s.deploy.history.FsHistoryProviderSuite (was: Fix unstable test case in FsHistor

[jira] [Updated] (SPARK-5559) Flaky test: o.a.s.streaming.flume.FlumeStreamSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5559: - Priority: Critical (was: Major) > Flaky test: o.a.s.streaming.flume.FlumeStreamSuite > --

[jira] [Updated] (SPARK-5548) Flaky test: o.a.s.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5548: - Summary: Flaky test: o.a.s.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server (was: Flaky test: o

[jira] [Updated] (SPARK-5559) Flaky test: o.a.s.streaming.flume.FlumeStreamSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5559: - Summary: Flaky test: o.a.s.streaming.flume.FlumeStreamSuite (was: Remove oppotunity we met flakiness when

[jira] [Updated] (SPARK-5548) Flaky test: o.a.s.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5548: - Priority: Critical (was: Major) > Flaky test: o.a.s.util.AkkaUtilsSuite.remote fetch ssl on - untrusted s

[jira] [Updated] (SPARK-5559) Flaky test: o.a.s.streaming.flume.FlumeStreamSuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5559: - Assignee: Kousuke Saruta > Flaky test: o.a.s.streaming.flume.FlumeStreamSuite > --

[jira] [Updated] (SPARK-5659) Flaky test: o.a.s.streaming.ReceiverSuite.block

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5659: - Summary: Flaky test: o.a.s.streaming.ReceiverSuite.block (was: Flaky Test: org.apache.spark.streaming.Rec

[jira] [Updated] (SPARK-5696) HiveThriftServer2Suite fails because of extra log4j.properties in the driver classpath

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5696: - Priority: Critical (was: Major) > HiveThriftServer2Suite fails because of extra log4j.properties in the d

[jira] [Updated] (SPARK-5626) Spurious test failures due to NullPointerException in EasyMock test code

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5626: - Priority: Critical (was: Major) > Spurious test failures due to NullPointerException in EasyMock test cod

[jira] [Updated] (SPARK-5679) Flaky tests in o.a.s.metrics.InputOutputMetricsSuite: input metrics with interleaved reads and input metrics with mixed read method

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5679: - Priority: Critical (was: Major) > Flaky tests in o.a.s.metrics.InputOutputMetricsSuite: input metrics wit

[jira] [Updated] (SPARK-5690) Flaky test: o.a.s.deploy.rest.StandaloneRestSubmitSuite.simple submit until completion

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5690: - Summary: Flaky test: o.a.s.deploy.rest.StandaloneRestSubmitSuite.simple submit until completion (was: Fla

[jira] [Updated] (SPARK-5679) Flaky tests in o.a.s.metrics.InputOutputMetricsSuite: input metrics with interleaved reads and input metrics with mixed read method

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5679: - Summary: Flaky tests in o.a.s.metrics.InputOutputMetricsSuite: input metrics with interleaved reads and in

[jira] [Updated] (SPARK-5731) Flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.basic stream receiving with multiple topics and smallest starting offset

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5731: - Summary: Flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.basic stream receiving with multiple top

[jira] [Commented] (SPARK-5690) Flaky test: org.apache.spark.deploy.rest.StandaloneRestSubmitSuite.simple submit until completion

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326903#comment-14326903 ] Andrew Or commented on SPARK-5690: -- Fixed in https://github.com/apache/spark/pull/4637 >

[jira] [Updated] (SPARK-5700) Bump jets3t version from 0.9.2 to 0.9.3 in hadoop-2.3 and hadoop-2.4 profiles

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5700: - Labels: (was: flaky-test) > Bump jets3t version from 0.9.2 to 0.9.3 in hadoop-2.3 and hadoop-2.4 profile

[jira] [Updated] (SPARK-5751) Flaky test: o.a.s.sql.hive.thriftserver.HiveThriftServer2Suite sometimes times out

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5751: - Priority: Critical (was: Major) > Flaky test: o.a.s.sql.hive.thriftserver.HiveThriftServer2Suite sometime

[jira] [Commented] (SPARK-5463) Fix Parquet filter push-down

2015-02-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326902#comment-14326902 ] Cheng Lian commented on SPARK-5463: --- No, the new release hasn't been cut yet. > Fix Par

[jira] [Updated] (SPARK-5751) Flaky test: o.a.s.sql.hive.thriftserver.HiveThriftServer2Suite sometimes times out

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5751: - Summary: Flaky test: o.a.s.sql.hive.thriftserver.HiveThriftServer2Suite sometimes times out (was: Flaky t

[jira] [Updated] (SPARK-5812) Flaky test: o.a.s.JavaAPISuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5812: - Summary: Flaky test: o.a.s.JavaAPISuite (was: Potential flaky test JavaAPISuite.glom) > Flaky test: o.a.

[jira] [Updated] (SPARK-5751) Flaky test: o.a.s.sql.hive.thriftserver.HiveThriftServer2Suite sometimes timeouts

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5751: - Summary: Flaky test: o.a.s.sql.hive.thriftserver.HiveThriftServer2Suite sometimes timeouts (was: First te

[jira] [Updated] (SPARK-5812) Flaky test: o.a.s.JavaAPISuite

2015-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5812: - Priority: Critical (was: Major) > Flaky test: o.a.s.JavaAPISuite > -- > >

[jira] [Resolved] (SPARK-2472) Spark SQL Thrift server sometimes assigns wrong job group name

2015-02-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-2472. --- Resolution: Fixed Assignee: Kay Ousterhout Target Version/s: 1.3.0 (was: 1.4.0)

[jira] [Commented] (SPARK-2472) Spark SQL Thrift server sometimes assigns wrong job group name

2015-02-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326893#comment-14326893 ] Cheng Lian commented on SPARK-2472: --- This ticket is fixed by [#4630|https://github.com/a

[jira] [Updated] (SPARK-2472) Spark SQL Thrift server sometimes assigns wrong job group name

2015-02-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-2472: -- Fix Version/s: 1.2.2 1.3.0 > Spark SQL Thrift server sometimes assigns wrong job grou

[jira] [Updated] (SPARK-5846) Spark SQL does not correctly set job description and scheduler pool

2015-02-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5846: -- Fix Version/s: 1.2.2 > Spark SQL does not correctly set job description and scheduler pool > ---

[jira] [Resolved] (SPARK-5846) Spark SQL does not correctly set job description and scheduler pool

2015-02-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-5846. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4630 [https://github.com/

[jira] [Updated] (SPARK-5881) RDD remains cached after the table gets overridden by "CACHE TABLE"

2015-02-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5881: Description: {code} val rdd = sc.parallelize((1 to 10).map(i => s"""{"a":$i, "b":"str${i}"}""")) sqlContext.

[jira] [Updated] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5904: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5166 > DataFrame methods with varargs do not wo

[jira] [Updated] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5904: --- Labels: DataFrame (was: ) > DataFrame methods with varargs do not work in Java >

[jira] [Updated] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5904: - Assignee: (was: Joseph K. Bradley) > DataFrame methods with varargs do not work in Jav

[jira] [Commented] (SPARK-5905) Improve RowMatrix user guide and doc.

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326828#comment-14326828 ] Xiangrui Meng commented on SPARK-5905: -- 1. If people say `an m x n matrix`, `m` shoul

[jira] [Commented] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326818#comment-14326818 ] Joseph K. Bradley commented on SPARK-5904: -- After conferring with others, it soun

[jira] [Assigned] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-5904: Assignee: Joseph K. Bradley > DataFrame methods with varargs do not work in Java >

[jira] [Created] (SPARK-5905) Improve RowMatrix user guide and doc.

2015-02-18 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-5905: Summary: Improve RowMatrix user guide and doc. Key: SPARK-5905 URL: https://issues.apache.org/jira/browse/SPARK-5905 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-5897) Add PIC code example to user guide

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5897: - Issue Type: Documentation (was: Bug) > Add PIC code example to user guide > -

[jira] [Resolved] (SPARK-5897) Add PIC code example to user guide

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5897. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by https://github.com/apache/spark/

[jira] [Commented] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326805#comment-14326805 ] Joseph K. Bradley commented on SPARK-5904: -- I'll work on this, but I'm not sure a

[jira] [Updated] (SPARK-5879) spary_ec2.py should expose/return master and slave lists (e.g. write to file)

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5879: - Fix Version/s: (was: 1.3.0) > spary_ec2.py should expose/return master and slave lists (e.g. w

[jira] [Reopened] (SPARK-5879) spary_ec2.py should expose/return master and slave lists (e.g. write to file)

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-5879: -- Sorry there was a typo in my PR. This issue is still open. > spary_ec2.py should expose/return mast

[jira] [Updated] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5904: - Description: DataFrame methods with varargs fail when called from Java due to a bug in Sca

[jira] [Updated] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5904: - Assignee: (was: Joseph K. Bradley) > DataFrame methods with varargs do not work in Jav

[jira] [Resolved] (SPARK-5879) spary_ec2.py should expose/return master and slave lists (e.g. write to file)

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5879. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4680 [https://githu

[jira] [Created] (SPARK-5904) DataFrame methods with varargs do not work in Java

2015-02-18 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5904: Summary: DataFrame methods with varargs do not work in Java Key: SPARK-5904 URL: https://issues.apache.org/jira/browse/SPARK-5904 Project: Spark Issu

[jira] [Commented] (SPARK-5873) Can't see partially analyzed plans

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326793#comment-14326793 ] Apache Spark commented on SPARK-5873: - User 'marmbrus' has created a pull request for

[jira] [Created] (SPARK-5903) Add support for Decimal type in SQL Table caching

2015-02-18 Thread Manoj Samel (JIRA)
Manoj Samel created SPARK-5903: -- Summary: Add support for Decimal type in SQL Table caching Key: SPARK-5903 URL: https://issues.apache.org/jira/browse/SPARK-5903 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-02-18 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326778#comment-14326778 ] Zhan Zhang commented on SPARK-1537: --- I have sent a PR with WIP for people who are intere

[jira] [Commented] (SPARK-5902) PipelineStage.transformSchema should be protected, not private

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326729#comment-14326729 ] Apache Spark commented on SPARK-5902: - User 'jkbradley' has created a pull request for

[jira] [Comment Edited] (SPARK-2344) Add Fuzzy C-Means algorithm to MLlib

2015-02-18 Thread Beniamino (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326719#comment-14326719 ] Beniamino edited comment on SPARK-2344 at 2/18/15 11:22 PM: Hi

[jira] [Created] (SPARK-5902) PipelineStage.transformSchema should be protected, not private

2015-02-18 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5902: Summary: PipelineStage.transformSchema should be protected, not private Key: SPARK-5902 URL: https://issues.apache.org/jira/browse/SPARK-5902 Project: Spark

[jira] [Commented] (SPARK-2344) Add Fuzzy C-Means algorithm to MLlib

2015-02-18 Thread Beniamino (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326719#comment-14326719 ] Beniamino commented on SPARK-2344: -- Hi Alex, Thank you for the link!!! I've posted a wor

[jira] [Updated] (SPARK-3066) Support recommendAll in matrix factorization model

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3066: - Target Version/s: 1.4.0 Assignee: Debasish Das > Support recommendAll in matrix factor

[jira] [Commented] (SPARK-5669) Spark assembly includes incompatibly licensed libgfortran, libgcc code via JBLAS

2015-02-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326706#comment-14326706 ] Xiangrui Meng commented on SPARK-5669: -- [~srowen] We will remove JBLAS from runtime i

[jira] [Commented] (SPARK-5722) Infer_schema_type incorrect for Integers in pyspark

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326705#comment-14326705 ] Apache Spark commented on SPARK-5722: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-5879) spary_ec2.py should expose/return master and slave lists (e.g. write to file)

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326699#comment-14326699 ] Apache Spark commented on SPARK-5879: - User 'mengxr' has created a pull request for th

[jira] [Created] (SPARK-5901) [PySpark] pickle classes in main module

2015-02-18 Thread Davies Liu (JIRA)
Davies Liu created SPARK-5901: - Summary: [PySpark] pickle classes in main module Key: SPARK-5901 URL: https://issues.apache.org/jira/browse/SPARK-5901 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-5900) Wrap the results returned by PIC and FPGrowth in case classes

2015-02-18 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-5900: Summary: Wrap the results returned by PIC and FPGrowth in case classes Key: SPARK-5900 URL: https://issues.apache.org/jira/browse/SPARK-5900 Project: Spark

[jira] [Updated] (SPARK-5896) toDF in python doesn't work with tuple/list

2015-02-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-5896: -- Summary: toDF in python doesn't work with tuple/list (was: toDF in python doesn't work with Strings) >

[jira] [Commented] (SPARK-5896) toDF in python doesn't work with tuple/list w/o names

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326655#comment-14326655 ] Apache Spark commented on SPARK-5896: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-5898) Can't create DataFrame from Pandas data frame

2015-02-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326654#comment-14326654 ] Apache Spark commented on SPARK-5898: - User 'davies' has created a pull request for th

[jira] [Updated] (SPARK-5896) toDF in python doesn't work with tuple/list w/o names

2015-02-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-5896: -- Summary: toDF in python doesn't work with tuple/list w/o names (was: toDF in python doesn't work with t

  1   2   >