[jira] [Commented] (SPARK-2393) Simple cost estimation and auto selection of broadcast join

[ https://issues.apache.org/jira/browse/SPARK-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054598#comment-14054598 ] Michael Armbrust commented on SPARK-2393: -

[jira] [Updated] (SPARK-2393) Simple cost estimation and auto selection of broadcast join

[ https://issues.apache.org/jira/browse/SPARK-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2393: Assignee: Zongheng Yang (was: Michael Armbrust) Simple cost estimation and auto

[jira] [Updated] (SPARK-2097) UDF Support

[ https://issues.apache.org/jira/browse/SPARK-2097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2097: Priority: Blocker (was: Critical) UDF Support --- Key:

[jira] [Updated] (SPARK-2391) LIMIT queries ship a whole partition of data for in-memory tables

[ https://issues.apache.org/jira/browse/SPARK-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2391: --- Summary: LIMIT queries ship a whole partition of data for in-memory tables (was: LIMIT queries ship

[jira] [Resolved] (SPARK-2391) LIMIT queries ship a whole partition of data

[ https://issues.apache.org/jira/browse/SPARK-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2391. Resolution: Fixed Fix Version/s: 1.1.0 LIMIT queries ship a whole partition of data

[jira] [Updated] (SPARK-2290) Do not send SPARK_HOME from workers to executors

[ https://issues.apache.org/jira/browse/SPARK-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2290: --- Issue Type: Improvement (was: Bug) Do not send SPARK_HOME from workers to executors

[jira] [Updated] (SPARK-2290) Do not send SPARK_HOME from workers to executors

[ https://issues.apache.org/jira/browse/SPARK-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2290: --- Priority: Major (was: Minor) Do not send SPARK_HOME from workers to executors

[jira] [Updated] (SPARK-2290) Do not send SPARK_HOME from workers to executors

[ https://issues.apache.org/jira/browse/SPARK-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2290: --- Assignee: Patrick Wendell Do not send SPARK_HOME from workers to executors

[jira] [Updated] (SPARK-2401) AdaBoost.MH, a multi-class multi-label classifier

[ https://issues.apache.org/jira/browse/SPARK-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Bai updated SPARK-2401: Description: Multi-class multi-label classifiers are very useful in web page profiling, audience

[jira] [Updated] (SPARK-2290) Do not send SPARK_HOME from workers to executors

[ https://issues.apache.org/jira/browse/SPARK-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2290: --- Summary: Do not send SPARK_HOME from workers to executors (was: Worker should directly use

[jira] [Commented] (SPARK-2290) Do not send SPARK_HOME from workers to executors

[ https://issues.apache.org/jira/browse/SPARK-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054633#comment-14054633 ] Patrick Wendell commented on SPARK-2290: I updated the description here. It is

[jira] [Created] (SPARK-2402) DiskBlockObjectWriter should update the initial position when reusing this object

Saisai Shao created SPARK-2402: -- Summary: DiskBlockObjectWriter should update the initial position when reusing this object Key: SPARK-2402 URL: https://issues.apache.org/jira/browse/SPARK-2402 Project:

[jira] [Commented] (SPARK-2402) DiskBlockObjectWriter should update the initial position when reusing this object

[ https://issues.apache.org/jira/browse/SPARK-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054686#comment-14054686 ] Saisai Shao commented on SPARK-2402: PR submitted,

[jira] [Comment Edited] (SPARK-2201) Improve FlumeInputDStream's stability and make it scalable

[ https://issues.apache.org/jira/browse/SPARK-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054442#comment-14054442 ] sunshangchun edited comment on SPARK-2201 at 7/8/14 11:12 AM: --

[jira] [Commented] (SPARK-2387) Remove the stage barrier for better resource utilization

[ https://issues.apache.org/jira/browse/SPARK-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054862#comment-14054862 ] Rui Li commented on SPARK-2387: --- PR created at https://github.com/apache/spark/pull/1328

[jira] [Commented] (SPARK-2306) BoundedPriorityQueue is private and not registered with Kryo

[ https://issues.apache.org/jira/browse/SPARK-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054885#comment-14054885 ] Daniel Darabos commented on SPARK-2306: --- Thanks! Until 1.1 is out, here's a

[jira] [Created] (SPARK-2403) Spark stuck when class is not registered with Kryo

Daniel Darabos created SPARK-2403: - Summary: Spark stuck when class is not registered with Kryo Key: SPARK-2403 URL: https://issues.apache.org/jira/browse/SPARK-2403 Project: Spark Issue

[jira] [Commented] (SPARK-2403) Spark stuck when class is not registered with Kryo

[ https://issues.apache.org/jira/browse/SPARK-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054907#comment-14054907 ] Daniel Darabos commented on SPARK-2403: --- I think DAGSchedulerActorSupervisor is

[jira] [Comment Edited] (SPARK-2403) Spark stuck when class is not registered with Kryo

[ https://issues.apache.org/jira/browse/SPARK-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054907#comment-14054907 ] Daniel Darabos edited comment on SPARK-2403 at 7/8/14 12:59 PM:

[jira] [Created] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

Nan Zhu created SPARK-2404: -- Summary: spark-submit and spark-class may overwrite the already defined SPARK_HOME Key: SPARK-2404 URL: https://issues.apache.org/jira/browse/SPARK-2404 Project: Spark

[jira] [Updated] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

[ https://issues.apache.org/jira/browse/SPARK-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-2404: --- Description: in spark-class and spark-submit the SPARK_HOME is set to the present working directory,

[jira] [Commented] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

[ https://issues.apache.org/jira/browse/SPARK-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055165#comment-14055165 ] Nan Zhu commented on SPARK-2404: PR https://github.com/apache/spark/pull/1331

[jira] [Created] (SPARK-2405) Multiple instances of an InMemoryRelation in a single plan results in recaching

Michael Armbrust created SPARK-2405: --- Summary: Multiple instances of an InMemoryRelation in a single plan results in recaching Key: SPARK-2405 URL: https://issues.apache.org/jira/browse/SPARK-2405

[jira] [Updated] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

[ https://issues.apache.org/jira/browse/SPARK-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-2404: --- Description: in spark-class and spark-submit the SPARK_HOME is set to the present working directory,

[jira] [Resolved] (SPARK-2403) Spark stuck when class is not registered with Kryo

[ https://issues.apache.org/jira/browse/SPARK-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-2403. --- Resolution: Fixed Fix Version/s: 1.0.2 1.1.0 Spark stuck when

[jira] [Commented] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055207#comment-14055207 ] Masayoshi TSUZUKI commented on SPARK-2017: -- I wonder if the browser can handle

[jira] [Commented] (SPARK-2063) Creating a SchemaRDD via sql() does not correctly resolve nested types

[ https://issues.apache.org/jira/browse/SPARK-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055193#comment-14055193 ] Yin Huai commented on SPARK-2063: - Tried two queries. The following one works {code} val

[jira] [Created] (SPARK-2406) Partitioned Parquet Support

Michael Armbrust created SPARK-2406: --- Summary: Partitioned Parquet Support Key: SPARK-2406 URL: https://issues.apache.org/jira/browse/SPARK-2406 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-2395) Optimize common like expressions

[ https://issues.apache.org/jira/browse/SPARK-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2395. Resolution: Fixed Fix Version/s: 1.1.0 Optimize common like expressions

[jira] [Created] (SPARK-2407) Implement SQL SUBSTR() directly in Catalyst

William Benton created SPARK-2407: - Summary: Implement SQL SUBSTR() directly in Catalyst Key: SPARK-2407 URL: https://issues.apache.org/jira/browse/SPARK-2407 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2407) Implement SQL SUBSTR() directly in Catalyst

[ https://issues.apache.org/jira/browse/SPARK-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055232#comment-14055232 ] William Benton commented on SPARK-2407: --- I have this on a branch and will submit a

[jira] [Commented] (SPARK-786) Clean up old work directories in standalone worker

[ https://issues.apache.org/jira/browse/SPARK-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055270#comment-14055270 ] Anurag Tangri commented on SPARK-786: - Knock-knock :) Can somebody assign this ticket

[jira] [Resolved] (SPARK-2400) config spark.yarn.max.executor.failures is not explained accurately

[ https://issues.apache.org/jira/browse/SPARK-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2400. -- Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Chen Chao Target

[jira] [Updated] (SPARK-2393) Simple cost estimation and auto selection of broadcast join

[ https://issues.apache.org/jira/browse/SPARK-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zongheng Yang updated SPARK-2393: - Description: Spark SQL should support the common optimization known as cost estimations. For

[jira] [Created] (SPARK-2408) RDD.map(func) dependencies issue after checkpoint count

Daniel Fry created SPARK-2408: - Summary: RDD.map(func) dependencies issue after checkpoint count Key: SPARK-2408 URL: https://issues.apache.org/jira/browse/SPARK-2408 Project: Spark Issue Type:

[jira] [Created] (SPARK-2409) Make SQLConf thread safe

Reynold Xin created SPARK-2409: -- Summary: Make SQLConf thread safe Key: SPARK-2409 URL: https://issues.apache.org/jira/browse/SPARK-2409 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-2407) Implement SQL SUBSTR() directly in Catalyst

[ https://issues.apache.org/jira/browse/SPARK-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2407: --- Assignee: William Benton Implement SQL SUBSTR() directly in Catalyst

[jira] [Updated] (SPARK-2407) Implement SQL SUBSTR() directly in Catalyst

[ https://issues.apache.org/jira/browse/SPARK-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2407: --- Target Version/s: 1.1.0 Implement SQL SUBSTR() directly in Catalyst

[jira] [Updated] (SPARK-2393) Simple cost estimation and auto selection of broadcast join

[ https://issues.apache.org/jira/browse/SPARK-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2393: --- Issue Type: New Feature (was: Bug) Simple cost estimation and auto selection of broadcast join

[jira] [Commented] (SPARK-2407) Implement SQL SUBSTR() directly in Catalyst

[ https://issues.apache.org/jira/browse/SPARK-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055399#comment-14055399 ] Reynold Xin commented on SPARK-2407: Great - I'm assigning the ticket to you Will.

[jira] [Commented] (SPARK-2408) RDD.map(func) dependencies issue after checkpoint count

[ https://issues.apache.org/jira/browse/SPARK-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055408#comment-14055408 ] Sean Owen commented on SPARK-2408: -- This one confuses me too. It happens without the

[jira] [Commented] (SPARK-2226) HAVING should be able to contain aggregate expressions that don't appear in the aggregation list.

[ https://issues.apache.org/jira/browse/SPARK-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055455#comment-14055455 ] Michael Armbrust commented on SPARK-2226: - Regarding how to implement this: I

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055460#comment-14055460 ] Tathagata Das commented on SPARK-2377: -- Thanks for making this JIRA. There has been

[jira] [Resolved] (SPARK-2160) error of Decision tree algorithm in Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2160. -- Resolution: Duplicate error of Decision tree algorithm in Spark MLlib

[jira] [Commented] (SPARK-2226) HAVING should be able to contain aggregate expressions that don't appear in the aggregation list.

[ https://issues.apache.org/jira/browse/SPARK-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055499#comment-14055499 ] William Benton commented on SPARK-2226: --- Thanks, [~marmbrus], this makes sense!

[jira] [Created] (SPARK-2410) Thrift/JDBC Server

Michael Armbrust created SPARK-2410: --- Summary: Thrift/JDBC Server Key: SPARK-2410 URL: https://issues.apache.org/jira/browse/SPARK-2410 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-2411) Standalone Master - direct users to turn on event logs

Andrew Or created SPARK-2411: Summary: Standalone Master - direct users to turn on event logs Key: SPARK-2411 URL: https://issues.apache.org/jira/browse/SPARK-2411 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-2362) newFilesOnly = true FileInputDStream processes existing files in a directory

[ https://issues.apache.org/jira/browse/SPARK-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2362: - Affects Version/s: 0.9.0 0.9.1 newFilesOnly = true FileInputDStream

[jira] [Updated] (SPARK-2362) newFilesOnly = true FileInputDStream processes existing files in a directory

[ https://issues.apache.org/jira/browse/SPARK-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2362: - Fix Version/s: 1.0.2 newFilesOnly = true FileInputDStream processes existing files in a

[jira] [Resolved] (SPARK-2362) newFilesOnly = true FileInputDStream processes existing files in a directory

[ https://issues.apache.org/jira/browse/SPARK-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-2362. -- Resolution: Fixed newFilesOnly = true FileInputDStream processes existing files in a

[jira] [Commented] (SPARK-2340) Resolve paths properly for event logging / history server

[ https://issues.apache.org/jira/browse/SPARK-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055579#comment-14055579 ] Andrew Or commented on SPARK-2340: -- https://github.com/apache/spark/pull/1280 Resolve

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055637#comment-14055637 ] Nishkam Ravi commented on SPARK-2398: - Thanks [~gq], looks related. I can see the

[jira] [Commented] (SPARK-786) Clean up old work directories in standalone worker

[ https://issues.apache.org/jira/browse/SPARK-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055638#comment-14055638 ] Andrew Or commented on SPARK-786: - I believe this was already fixed in

[jira] [Updated] (SPARK-2360) CSV import to SchemaRDDs

[ https://issues.apache.org/jira/browse/SPARK-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2360: Assignee: Hossein Falaki CSV import to SchemaRDDs

[jira] [Updated] (SPARK-2411) Standalone Master - direct users to turn on event logs

[ https://issues.apache.org/jira/browse/SPARK-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2411: - Attachment: Screen Shot 2014-07-08 at 4.23.51 PM.png Standalone Master - direct users to turn on event

[jira] [Resolved] (SPARK-2392) Executors should not start their own HTTP servers

[ https://issues.apache.org/jira/browse/SPARK-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2392. Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Andrew Or Executors should not

[jira] [Created] (SPARK-2412) CoalescedRDD throws exception with certain pref locs

Aaron Davidson created SPARK-2412: - Summary: CoalescedRDD throws exception with certain pref locs Key: SPARK-2412 URL: https://issues.apache.org/jira/browse/SPARK-2412 Project: Spark Issue

[jira] [Created] (SPARK-2413) Upgrade junit_xml_listener to 0.5.1

DB Tsai created SPARK-2413: -- Summary: Upgrade junit_xml_listener to 0.5.1 Key: SPARK-2413 URL: https://issues.apache.org/jira/browse/SPARK-2413 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-2413) Upgrade junit_xml_listener to 0.5.1

[ https://issues.apache.org/jira/browse/SPARK-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2413. Resolution: Fixed Fix Version/s: 1.1.0 Assignee: DB Tsai Upgrade

[jira] [Commented] (SPARK-2298) Show stage attempt in UI

[ https://issues.apache.org/jira/browse/SPARK-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055720#comment-14055720 ] Reynold Xin commented on SPARK-2298: It does because the two attempts will be two

[jira] [Issue Comment Deleted] (SPARK-2308) Add KMeans MiniBatch clustering algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doris Xin updated SPARK-2308: - Comment: was deleted (was: Hey guys, Sorry to crash the party. I don't think small clusters are

[jira] [Resolved] (SPARK-2384) Add tooltips for shuffle write and scheduler delay in UI

[ https://issues.apache.org/jira/browse/SPARK-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2384. Resolution: Fixed Fix Version/s: 1.1.0 Add tooltips for shuffle write and scheduler delay