[jira] [Commented] (SPARK-3299) [SQL] Public API in SQLContext to list tables

[ https://issues.apache.org/jira/browse/SPARK-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270346#comment-14270346 ] Apache Spark commented on SPARK-3299: - User 'bbejeck' has created a pull request for

[jira] [Closed] (SPARK-5122) Remove Shark from spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5122. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Nicholas Chammas Target

[jira] [Resolved] (SPARK-4636) Cluster By Distribute By output different with Hive

[ https://issues.apache.org/jira/browse/SPARK-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao resolved SPARK-4636. -- Resolution: Not a Problem The answer with highest score seems not correct, in might tested with the

[jira] [Reopened] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-3490: -- Alleviate port collisions during tests -- Key:

[jira] [Updated] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3490: - Target Version/s: 1.2.0, 1.1.1, 0.9.3, 1.0.3 (was: 1.1.1, 1.2.0) Alleviate port collisions during tests

[jira] [Commented] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270320#comment-14270320 ] Andrew Or commented on SPARK-3490: -- Reopening this for branches 0.9 and 1.0 Alleviate

[jira] [Closed] (SPARK-5007) Try random port when startServiceOnPort to reduce the chance of port collision

[ https://issues.apache.org/jira/browse/SPARK-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YanTang Zhai closed SPARK-5007. --- Resolution: Won't Fix Try random port when startServiceOnPort to reduce the chance of port collision

[jira] [Commented] (SPARK-2387) Remove the stage barrier for better resource utilization

[ https://issues.apache.org/jira/browse/SPARK-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270442#comment-14270442 ] Apache Spark commented on SPARK-2387: - User 'lianhuiwang' has created a pull request

[jira] [Comment Edited] (SPARK-4636) Cluster By Distribute By output different with Hive

[ https://issues.apache.org/jira/browse/SPARK-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270466#comment-14270466 ] Cheng Hao edited comment on SPARK-4636 at 1/9/15 2:57 AM: -- The

[jira] [Commented] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270256#comment-14270256 ] Apache Spark commented on SPARK-3490: - User 'andrewor14' has created a pull request

[jira] [Created] (SPARK-5160) Python module in jars

Davies Liu created SPARK-5160: - Summary: Python module in jars Key: SPARK-5160 URL: https://issues.apache.org/jira/browse/SPARK-5160 Project: Spark Issue Type: New Feature Components:

[jira] [Commented] (SPARK-4912) Persistent data source tables

[ https://issues.apache.org/jira/browse/SPARK-4912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270242#comment-14270242 ] Apache Spark commented on SPARK-4912: - User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-3431) Parallelize Scala/Java test execution

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270324#comment-14270324 ] Nicholas Chammas commented on SPARK-3431: - Generic update: * For those not

[jira] [Comment Edited] (SPARK-4636) Cluster By Distribute By output different with Hive

[ https://issues.apache.org/jira/browse/SPARK-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270466#comment-14270466 ] Cheng Hao edited comment on SPARK-4636 at 1/9/15 2:56 AM: -- The

[jira] [Created] (SPARK-5161) Parallelize Python test execution

Nicholas Chammas created SPARK-5161: --- Summary: Parallelize Python test execution Key: SPARK-5161 URL: https://issues.apache.org/jira/browse/SPARK-5161 Project: Spark Issue Type:

[jira] [Commented] (SPARK-5119) java.lang.ArrayIndexOutOfBoundsException on trying to train decision tree model

[ https://issues.apache.org/jira/browse/SPARK-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270316#comment-14270316 ] Kai Sasaki commented on SPARK-5119: --- I think impurity implemented MLlib cannot keep

[jira] [Updated] (SPARK-5122) Remove Shark from spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5122: - Affects Version/s: 1.0.0 Remove Shark from spark-ec2 --- Key:

[jira] [Created] (SPARK-5163) Load properties from configuration file for example spark-defaults.conf when creating SparkConf object

YanTang Zhai created SPARK-5163: --- Summary: Load properties from configuration file for example spark-defaults.conf when creating SparkConf object Key: SPARK-5163 URL:

[jira] [Commented] (SPARK-5163) Load properties from configuration file for example spark-defaults.conf when creating SparkConf object

[ https://issues.apache.org/jira/browse/SPARK-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270491#comment-14270491 ] Apache Spark commented on SPARK-5163: - User 'YanTangZhai' has created a pull request

[jira] [Updated] (SPARK-3431) Parallelize Scala/Java test execution

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3431: Summary: Parallelize Scala/Java test execution (was: Parallelize execution of tests)

[jira] [Resolved] (SPARK-4048) Enhance and extend hadoop-provided profile

[ https://issues.apache.org/jira/browse/SPARK-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4048. Resolution: Fixed Fix Version/s: 1.3.0 Enhance and extend hadoop-provided profile

[jira] [Commented] (SPARK-2387) Remove the stage barrier for better resource utilization

[ https://issues.apache.org/jira/browse/SPARK-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270444#comment-14270444 ] Lianhui Wang commented on SPARK-2387: - [~xuefuz] [~sandyr] [~lirui] yes, i think

[jira] [Commented] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270254#comment-14270254 ] Apache Spark commented on SPARK-3490: - User 'andrewor14' has created a pull request

[jira] [Commented] (SPARK-1882) Support dynamic memory sharing in Mesos

[ https://issues.apache.org/jira/browse/SPARK-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270553#comment-14270553 ] Jongyoul Lee commented on SPARK-1882: - [~aash] I have a question. Do you think

[jira] [Created] (SPARK-5164) YARN | Spark job submits from windows machine to a linux YARN cluster fail

Aniket Bhatnagar created SPARK-5164: --- Summary: YARN | Spark job submits from windows machine to a linux YARN cluster fail Key: SPARK-5164 URL: https://issues.apache.org/jira/browse/SPARK-5164

[jira] [Commented] (SPARK-5162) Python yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270543#comment-14270543 ] Harry Brundage commented on SPARK-5162: --- [~sandyr] are you familiar with why the

[jira] [Updated] (SPARK-5164) YARN | Spark job submits from windows machine to a linux YARN cluster fail

[ https://issues.apache.org/jira/browse/SPARK-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Bhatnagar updated SPARK-5164: Description: While submitting spark jobs from a windows machine to a linux YARN cluster,

[jira] [Updated] (SPARK-5164) YARN | Spark job submits from windows machine to a linux YARN cluster fail

[ https://issues.apache.org/jira/browse/SPARK-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Bhatnagar updated SPARK-5164: Description: While submitting spark jobs from a windows machine to a linux YARN cluster,

[jira] [Created] (SPARK-5168) Make SQLConf a field rather than mixin in SQLContext

Reynold Xin created SPARK-5168: -- Summary: Make SQLConf a field rather than mixin in SQLContext Key: SPARK-5168 URL: https://issues.apache.org/jira/browse/SPARK-5168 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-5152) Let metrics.properties file take an hdfs:// path

[ https://issues.apache.org/jira/browse/SPARK-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270616#comment-14270616 ] Patrick Wendell edited comment on SPARK-5152 at 1/9/15 6:19 AM:

[jira] [Commented] (SPARK-5153) flaky test of Reliable Kafka input stream with multiple topics

[ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270655#comment-14270655 ] Saisai Shao commented on SPARK-5153: Hi [~CodingCat], thanks a lot for your reporting,

[jira] [Commented] (SPARK-2621) Update task InputMetrics incrementally

[ https://issues.apache.org/jira/browse/SPARK-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270714#comment-14270714 ] Rui Li commented on SPARK-2621: --- Hey [~sandyr], it seems after this change we require the

[jira] [Closed] (SPARK-5000) Alias support string literal in spark sql

[ https://issues.apache.org/jira/browse/SPARK-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei closed SPARK-5000. -- Resolution: Fixed Alias support string literal in spark sql -

[jira] [Commented] (SPARK-5000) Alias support string literal in spark sql

[ https://issues.apache.org/jira/browse/SPARK-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270718#comment-14270718 ] wangfei commented on SPARK-5000: backticks can do this, so close this one. Alias support

[jira] [Created] (SPARK-5167) Move Row into sql package and make it usable for Java

Reynold Xin created SPARK-5167: -- Summary: Move Row into sql package and make it usable for Java Key: SPARK-5167 URL: https://issues.apache.org/jira/browse/SPARK-5167 Project: Spark Issue Type:

[jira] [Commented] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270574#comment-14270574 ] Saisai Shao commented on SPARK-5147: Hi Max, I think this is a left problem for the

[jira] [Commented] (SPARK-5152) Let metrics.properties file take an hdfs:// path

[ https://issues.apache.org/jira/browse/SPARK-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270616#comment-14270616 ] Patrick Wendell commented on SPARK-5152: Should we be loading the metrics

[jira] [Commented] (SPARK-1882) Support dynamic memory sharing in Mesos

[ https://issues.apache.org/jira/browse/SPARK-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270621#comment-14270621 ] Jongyoul Lee commented on SPARK-1882: - [~aash] First of all, It looks like that It's

[jira] [Commented] (SPARK-4989) wrong application configuration cause cluster down in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270630#comment-14270630 ] Apache Spark commented on SPARK-4989: - User 'liyezhang556520' has created a pull

[jira] [Commented] (SPARK-5164) YARN | Spark job submits from windows machine to a linux YARN cluster fail

[ https://issues.apache.org/jira/browse/SPARK-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270533#comment-14270533 ] Aniket Bhatnagar commented on SPARK-5164: - First issue can be fixed by using

[jira] [Created] (SPARK-5166) Stabilize Spark SQL APIs

Reynold Xin created SPARK-5166: -- Summary: Stabilize Spark SQL APIs Key: SPARK-5166 URL: https://issues.apache.org/jira/browse/SPARK-5166 Project: Spark Issue Type: Task Components:

[jira] [Updated] (SPARK-5097) Adding data frame APIs to SchemaRDD

[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5097: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-5166 Adding data frame APIs to

[jira] [Updated] (SPARK-5123) Stabilize Spark SQL data type API

[ https://issues.apache.org/jira/browse/SPARK-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5123: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5166 Stabilize Spark SQL data type API

[jira] [Commented] (SPARK-5165) Add support for rollup and cube in sqlcontext

[ https://issues.apache.org/jira/browse/SPARK-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270542#comment-14270542 ] Apache Spark commented on SPARK-5165: - User 'scwf' has created a pull request for this

[jira] [Created] (SPARK-5165) Add support for rollup and cube in sqlcontext

wangfei created SPARK-5165: -- Summary: Add support for rollup and cube in sqlcontext Key: SPARK-5165 URL: https://issues.apache.org/jira/browse/SPARK-5165 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-5123) Stabilize Spark SQL data type API

[ https://issues.apache.org/jira/browse/SPARK-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5123: --- Summary: Stabilize Spark SQL data type API (was: Expose only one version of the data type APIs (i.e.

[jira] [Updated] (SPARK-5166) Stabilize Spark SQL APIs

[ https://issues.apache.org/jira/browse/SPARK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5166: --- Description: Before we take Spark SQL out of alpha, we need to audit the APIs and stabilize them.

[jira] [Updated] (SPARK-5123) Stabilize Spark SQL data type API

[ https://issues.apache.org/jira/browse/SPARK-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5123: --- Description: Having two versions of the data type APIs (one for Java, one for Scala) requires

[jira] [Issue Comment Deleted] (SPARK-5123) Stabilize Spark SQL data type API

[ https://issues.apache.org/jira/browse/SPARK-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5123: --- Comment: was deleted (was: User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-1882) Support dynamic memory sharing in Mesos

[ https://issues.apache.org/jira/browse/SPARK-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270559#comment-14270559 ] Andrew Ash commented on SPARK-1882: --- Yes -- in the simplest case where you have one

[jira] [Updated] (SPARK-2620) case class cannot be used as key for reduce

[ https://issues.apache.org/jira/browse/SPARK-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2620: --- Assignee: Tobias Schlatter case class cannot be used as key for reduce

[jira] [Commented] (SPARK-1143) ClusterSchedulerSuite (soon to be TaskSchedulerImplSuite) does not actually test the ClusterScheduler/TaskSchedulerImpl

[ https://issues.apache.org/jira/browse/SPARK-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270612#comment-14270612 ] Apache Spark commented on SPARK-1143: - User 'kayousterhout' has created a pull request

[jira] [Commented] (SPARK-5136) Improve documentation around setting up Spark IntelliJ project

[ https://issues.apache.org/jira/browse/SPARK-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270626#comment-14270626 ] Patrick Wendell commented on SPARK-5136: I've updated it to be in the new

[jira] [Commented] (SPARK-5136) Improve documentation around setting up Spark IntelliJ project

[ https://issues.apache.org/jira/browse/SPARK-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270624#comment-14270624 ] Patrick Wendell commented on SPARK-5136: Hey Guys, I wrote that on the wiki quite

[jira] [Commented] (SPARK-4989) wrong application configuration cause cluster down in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270678#comment-14270678 ] Apache Spark commented on SPARK-4989: - User 'liyezhang556520' has created a pull

[jira] [Comment Edited] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270574#comment-14270574 ] Saisai Shao edited comment on SPARK-5147 at 1/9/15 7:29 AM: Hi

[jira] [Commented] (SPARK-5168) Make SQLConf a field rather than mixin in SQLContext

[ https://issues.apache.org/jira/browse/SPARK-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270575#comment-14270575 ] Apache Spark commented on SPARK-5168: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-5141) CaseInsensitiveMap throws java.io.NotSerializableException

[ https://issues.apache.org/jira/browse/SPARK-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270635#comment-14270635 ] Gankun Luo commented on SPARK-5141: --- Resolved CaseInsensitiveMap throws

[jira] [Commented] (SPARK-4989) wrong application configuration cause cluster down in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270634#comment-14270634 ] Apache Spark commented on SPARK-4989: - User 'liyezhang556520' has created a pull

[jira] [Commented] (SPARK-5141) CaseInsensitiveMap throws java.io.NotSerializableException

[ https://issues.apache.org/jira/browse/SPARK-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270636#comment-14270636 ] Gankun Luo commented on SPARK-5141: --- Resolved CaseInsensitiveMap throws

[jira] [Commented] (SPARK-4122) Add library to write data back to Kafka

[ https://issues.apache.org/jira/browse/SPARK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270373#comment-14270373 ] Hari Shreedharan commented on SPARK-4122: - The current design doc talks only about

[jira] [Commented] (SPARK-4865) Include temporary tables in SHOW TABLES

[ https://issues.apache.org/jira/browse/SPARK-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270394#comment-14270394 ] Bill Bejeck commented on SPARK-4865: Since I've worked on the related task SPARK-3299,

[jira] [Commented] (SPARK-5007) Try random port when startServiceOnPort to reduce the chance of port collision

[ https://issues.apache.org/jira/browse/SPARK-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270421#comment-14270421 ] YanTang Zhai commented on SPARK-5007: - [~rxin] Oh, I see. Thank you very much. Try

[jira] [Commented] (SPARK-4955) Executor does not get killed after configured interval.

[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270434#comment-14270434 ] Apache Spark commented on SPARK-4955: - User 'lianhuiwang' has created a pull request

[jira] [Comment Edited] (SPARK-2387) Remove the stage barrier for better resource utilization

[ https://issues.apache.org/jira/browse/SPARK-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270444#comment-14270444 ] Lianhui Wang edited comment on SPARK-2387 at 1/9/15 2:44 AM: -

[jira] [Closed] (SPARK-4973) Local directory in the driver of client-mode continues remaining even if application finished when external shuffle is enabled

[ https://issues.apache.org/jira/browse/SPARK-4973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4973. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Target Version/s: 1.3.0,

[jira] [Commented] (SPARK-5157) Configure more JVM options properly when we use ConcMarkSweepGC for AM.

[ https://issues.apache.org/jira/browse/SPARK-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270128#comment-14270128 ] Apache Spark commented on SPARK-5157: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-4983) Tag EC2 instances in the same call that launches them

[ https://issues.apache.org/jira/browse/SPARK-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270084#comment-14270084 ] Gen TANG commented on SPARK-4983: - By boto, we can only tag instance after it launched, to

[jira] [Commented] (SPARK-1630) PythonRDDs don't handle nulls gracefully

[ https://issues.apache.org/jira/browse/SPARK-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270097#comment-14270097 ] Davies Liu commented on SPARK-1630: --- We hit this issue with Kafka Python API, it will be

[jira] [Commented] (SPARK-5061) SQLContext: overload createParquetFile

[ https://issues.apache.org/jira/browse/SPARK-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270102#comment-14270102 ] Apache Spark commented on SPARK-5061: - User 'alexbaretta' has created a pull request

[jira] [Commented] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270224#comment-14270224 ] Apache Spark commented on SPARK-3490: - User 'andrewor14' has created a pull request

[jira] [Created] (SPARK-5162) Python yarn-cluster mode

Dana Klassen created SPARK-5162: --- Summary: Python yarn-cluster mode Key: SPARK-5162 URL: https://issues.apache.org/jira/browse/SPARK-5162 Project: Spark Issue Type: New Feature

[jira] [Comment Edited] (SPARK-2387) Remove the stage barrier for better resource utilization

[ https://issues.apache.org/jira/browse/SPARK-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270444#comment-14270444 ] Lianhui Wang edited comment on SPARK-2387 at 1/9/15 2:44 AM: -

[jira] [Commented] (SPARK-4983) Tag EC2 instances in the same call that launches them

[ https://issues.apache.org/jira/browse/SPARK-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270147#comment-14270147 ] Nicholas Chammas commented on SPARK-4983: - Yeah, I took a quick look at the boto

[jira] [Created] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

Andrew Ray created SPARK-5159: - Summary: Thrift server does not respect hive.server2.enable.doAs=true Key: SPARK-5159 URL: https://issues.apache.org/jira/browse/SPARK-5159 Project: Spark Issue

[jira] [Resolved] (SPARK-4891) Add exponential, log normal, and gamma distributions to data generator to PySpark's MLlib

[ https://issues.apache.org/jira/browse/SPARK-4891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4891. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3955

[jira] [Commented] (SPARK-5123) Expose only one version of the data type APIs (i.e. remove the Java-specific API)

[ https://issues.apache.org/jira/browse/SPARK-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270188#comment-14270188 ] Apache Spark commented on SPARK-5123: - User 'rxin' has created a pull request for this

[jira] [Closed] (SPARK-2100) Allow users to disable Jetty Spark UI in local mode

[ https://issues.apache.org/jira/browse/SPARK-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2100. Resolution: Duplicate Allow users to disable Jetty Spark UI in local mode

[jira] [Commented] (SPARK-2316) StorageStatusListener should avoid O(blocks) operations

[ https://issues.apache.org/jira/browse/SPARK-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269243#comment-14269243 ] Paul Wolfe commented on SPARK-2316: --- Any workaround ideas for users who can't yet

[jira] [Commented] (SPARK-5137) subtract does not take the spark.default.parallelism into account

[ https://issues.apache.org/jira/browse/SPARK-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269263#comment-14269263 ] Al M commented on SPARK-5137: - That's right. {code}a{code} has 11 partitions and

[jira] [Comment Edited] (SPARK-5137) subtract does not take the spark.default.parallelism into account

[ https://issues.apache.org/jira/browse/SPARK-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269263#comment-14269263 ] Al M edited comment on SPARK-5137 at 1/8/15 12:30 PM: -- That's right.

[jira] [Comment Edited] (SPARK-5137) subtract does not take the spark.default.parallelism into account

[ https://issues.apache.org/jira/browse/SPARK-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269263#comment-14269263 ] Al M edited comment on SPARK-5137 at 1/8/15 12:30 PM: -- That's right.

[jira] [Closed] (SPARK-5137) subtract does not take the spark.default.parallelism into account

[ https://issues.apache.org/jira/browse/SPARK-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Al M closed SPARK-5137. --- Resolution: Not a Problem subtract does not take the spark.default.parallelism into account

[jira] [Commented] (SPARK-4955) Executor does not get killed after configured interval.

[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269275#comment-14269275 ] Lianhui Wang commented on SPARK-4955: - yes, because YarnSchedulerActor cannot connect

[jira] [Updated] (SPARK-5100) Spark Thrift server monitor page

[ https://issues.apache.org/jira/browse/SPARK-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-5100: --- Attachment: (was: Spark Thrift-server monitor page.pdf) Spark Thrift server monitor page

[jira] [Updated] (SPARK-5100) Spark Thrift server monitor page

[ https://issues.apache.org/jira/browse/SPARK-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-5100: --- Attachment: Spark Thrift-server monitor page.pdf design doc Spark Thrift server monitor page

[jira] [Commented] (SPARK-4406) SVD should check for k 1

[ https://issues.apache.org/jira/browse/SPARK-4406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268979#comment-14268979 ] Apache Spark commented on SPARK-4406: - User 'MechCoder' has created a pull request for

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268440#comment-14268440 ] Gerard Maas edited comment on SPARK-4940 at 1/8/15 9:21 AM: Hi

[jira] [Commented] (SPARK-5141) CaseInsensitiveMap throws java.io.NotSerializableException

[ https://issues.apache.org/jira/browse/SPARK-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268962#comment-14268962 ] Apache Spark commented on SPARK-5141: - User 'luogankun' has created a pull request for

[jira] [Commented] (SPARK-5137) subtract does not take the spark.default.parallelism into account

[ https://issues.apache.org/jira/browse/SPARK-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268969#comment-14268969 ] Al M commented on SPARK-5137: - Yes I do mean subtractByKey. Sorry for not being clear. I'm

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269038#comment-14269038 ] Gerard Maas edited comment on SPARK-4940 at 1/8/15 9:28 AM: I

[jira] [Commented] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269038#comment-14269038 ] Gerard Maas commented on SPARK-4940: I forgot to mention that in the previous example,

[jira] [Commented] (SPARK-5100) Spark Thrift server monitor page

[ https://issues.apache.org/jira/browse/SPARK-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269002#comment-14269002 ] Apache Spark commented on SPARK-5100: - User 'tianyi' has created a pull request for

[jira] [Updated] (SPARK-4940) Support more evenly distributing cores for Mesos mode

[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gerard Maas updated SPARK-4940: --- Attachment: mesos-config-difference-3nodes-vs-2nodes.png Difference of job performance due to

[jira] [Created] (SPARK-5143) spark-network-yarn 2.11 depends on spark-network-shuffle 2.10

Aniket Bhatnagar created SPARK-5143: --- Summary: spark-network-yarn 2.11 depends on spark-network-shuffle 2.10 Key: SPARK-5143 URL: https://issues.apache.org/jira/browse/SPARK-5143 Project: Spark

[jira] [Commented] (SPARK-4963) SchemaRDD.sample may return wrong results

[ https://issues.apache.org/jira/browse/SPARK-4963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269136#comment-14269136 ] Yanbo Liang commented on SPARK-4963: Can anyone verify and merge this patch? It's a

[jira] [Created] (SPARK-5144) spark-yarn module should be published

Aniket Bhatnagar created SPARK-5144: --- Summary: spark-yarn module should be published Key: SPARK-5144 URL: https://issues.apache.org/jira/browse/SPARK-5144 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4159) Maven build doesn't run JUnit test suites

[ https://issues.apache.org/jira/browse/SPARK-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269149#comment-14269149 ] Sean Owen commented on SPARK-4159: -- [~sandyr] Ah right. Now you have two sets of

[jira] [Commented] (SPARK-3452) Maven build should skip publishing artifacts people shouldn't depend on

[ https://issues.apache.org/jira/browse/SPARK-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269145#comment-14269145 ] Aniket Bhatnagar commented on SPARK-3452: - I have opened another defect -

[jira] [Commented] (SPARK-5137) subtract does not take the spark.default.parallelism into account

[ https://issues.apache.org/jira/browse/SPARK-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269168#comment-14269168 ] Sean Owen commented on SPARK-5137: -- When you run {{a.subtractByKey(b))}} I assume that

  1   2   >