[jira] [Resolved] (SPARK-2093) NullPropagation should use exact type value.

[ https://issues.apache.org/jira/browse/SPARK-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2093. - Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 NullPropagation

[jira] [Updated] (SPARK-2093) NullPropagation should use exact type value.

[ https://issues.apache.org/jira/browse/SPARK-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2093: Assignee: Takuya Ueshin NullPropagation should use exact type value.

[jira] [Assigned] (SPARK-2014) Make PySpark store RDDs in MEMORY_ONLY_SER with compression by default

[ https://issues.apache.org/jira/browse/SPARK-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-2014: -- Assignee: Prashant Sharma Make PySpark store RDDs in MEMORY_ONLY_SER with compression

[jira] [Resolved] (SPARK-1968) SQL commands for caching tables

[ https://issues.apache.org/jira/browse/SPARK-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1968. - Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 SQL commands for

[jira] [Updated] (SPARK-2107) FilterPushdownSuite imports Junit and leads to compilation error

[ https://issues.apache.org/jira/browse/SPARK-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2107: Component/s: SQL FilterPushdownSuite imports Junit and leads to compilation error

[jira] [Resolved] (SPARK-2107) FilterPushdownSuite imports Junit and leads to compilation error

[ https://issues.apache.org/jira/browse/SPARK-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2107. - Resolution: Fixed FilterPushdownSuite imports Junit and leads to compilation error

[jira] [Updated] (SPARK-2107) FilterPushdownSuite imports Junit and leads to compilation error

[ https://issues.apache.org/jira/browse/SPARK-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2107: --- Fix Version/s: 1.1.0 1.0.1 FilterPushdownSuite imports Junit and leads to

[jira] [Resolved] (SPARK-2091) pyspark/mllib is not compatible with numpy-1.4

[ https://issues.apache.org/jira/browse/SPARK-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2091. -- Resolution: Fixed Fix Version/s: 1.0.1 pyspark/mllib is not compatible with numpy-1.4

[jira] [Created] (SPARK-2109) Setting SPARK_MEM for bin/pyspark does not work.

Prashant Sharma created SPARK-2109: -- Summary: Setting SPARK_MEM for bin/pyspark does not work. Key: SPARK-2109 URL: https://issues.apache.org/jira/browse/SPARK-2109 Project: Spark Issue

[jira] [Updated] (SPARK-2109) Setting SPARK_MEM for bin/pyspark does not work.

[ https://issues.apache.org/jira/browse/SPARK-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-2109: --- Description: prashant@sc:~/work/spark$ SPARK_MEM=10G bin/pyspark Python 2.7.6 (default,

[jira] [Updated] (SPARK-2109) Setting SPARK_MEM for bin/pyspark does not work.

[ https://issues.apache.org/jira/browse/SPARK-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-2109: --- Fix Version/s: 1.1.0 1.0.1 Setting SPARK_MEM for bin/pyspark does not

[jira] [Updated] (SPARK-2109) Setting SPARK_MEM for bin/pyspark does not work.

[ https://issues.apache.org/jira/browse/SPARK-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-2109: --- Affects Version/s: 1.0.0 Setting SPARK_MEM for bin/pyspark does not work.

[jira] [Commented] (SPARK-2109) Setting SPARK_MEM for bin/pyspark does not work.

[ https://issues.apache.org/jira/browse/SPARK-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027600#comment-14027600 ] Prashant Sharma commented on SPARK-2109: https://github.com/apache/spark/pull/1050

[jira] [Assigned] (SPARK-2109) Setting SPARK_MEM for bin/pyspark does not work.

[ https://issues.apache.org/jira/browse/SPARK-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-2109: -- Assignee: Prashant Sharma Setting SPARK_MEM for bin/pyspark does not work.

[jira] [Created] (SPARK-2110) Misleading help displayed for interactive mode pyspark --help

Prashant Sharma created SPARK-2110: -- Summary: Misleading help displayed for interactive mode pyspark --help Key: SPARK-2110 URL: https://issues.apache.org/jira/browse/SPARK-2110 Project: Spark

[jira] [Assigned] (SPARK-2032) Add an RDD.samplePartitions method for partition-level sampling

[ https://issues.apache.org/jira/browse/SPARK-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-2032: -- Assignee: Prashant Sharma Add an RDD.samplePartitions method for partition-level

[jira] [Assigned] (SPARK-2021) External hashing in PySpark

[ https://issues.apache.org/jira/browse/SPARK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-2021: -- Assignee: Prashant Sharma External hashing in PySpark ---

[jira] [Resolved] (SPARK-1639) Some tidying of Spark on YARN code

[ https://issues.apache.org/jira/browse/SPARK-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1639. -- Resolution: Fixed Fix Version/s: 1.1.0 Some tidying of Spark on YARN code

[jira] [Commented] (SPARK-2018) Big-Endian (IBM Power7) Spark Serialization issue

[ https://issues.apache.org/jira/browse/SPARK-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027753#comment-14027753 ] Gireesh Punathil commented on SPARK-2018: - I face this same issue for a different

[jira] [Commented] (SPARK-2018) Big-Endian (IBM Power7) Spark Serialization issue

[ https://issues.apache.org/jira/browse/SPARK-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027808#comment-14027808 ] Mridul Muralidharan commented on SPARK-2018: Ah ! This is an interesting bug.

[jira] [Commented] (SPARK-2018) Big-Endian (IBM Power7) Spark Serialization issue

[ https://issues.apache.org/jira/browse/SPARK-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027819#comment-14027819 ] Gireesh Punathil commented on SPARK-2018: - Mridul, Thanks for responding. 1. As

[jira] [Created] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

Thomas Graves created SPARK-2111: Summary: pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1 Key: SPARK-2111 URL: https://issues.apache.org/jira/browse/SPARK-2111 Project: Spark Issue Type:

[jira] [Closed] (SPARK-1699) Python relative should be independence from the core, becomes subprojects

[ https://issues.apache.org/jira/browse/SPARK-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1699. -- Resolution: Unresolved Python relative should be independence from the core, becomes subprojects

[jira] [Commented] (SPARK-2061) Deprecate `splits` in JavaRDDLike and add `partitions`

[ https://issues.apache.org/jira/browse/SPARK-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027978#comment-14027978 ] Anant Daksh Asthana commented on SPARK-2061: Could I be added as the Assignee

[jira] [Created] (SPARK-2112) ParquetTypesConverter should not create its own conf

Michael Armbrust created SPARK-2112: --- Summary: ParquetTypesConverter should not create its own conf Key: SPARK-2112 URL: https://issues.apache.org/jira/browse/SPARK-2112 Project: Spark

[jira] [Resolved] (SPARK-2070) Package private methods are not excluded correctly

[ https://issues.apache.org/jira/browse/SPARK-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2070. Resolution: Fixed Fixed by: https://github.com/apache/spark/pull/1021 Package private

[jira] [Resolved] (SPARK-2071) Package private classes that are deleted from an older version of Spark trigger errors

[ https://issues.apache.org/jira/browse/SPARK-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2071. Resolution: Fixed Fixed by: https://github.com/apache/spark/pull/1021 Package private

[jira] [Resolved] (SPARK-2108) Mark SparkContext methods that return block information as developer API's

[ https://issues.apache.org/jira/browse/SPARK-2108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2108. Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 Issue resolved by

[jira] [Created] (SPARK-2113) awaitTermination() after stop() will hang in Spark Stremaing

Patrick Wendell created SPARK-2113: -- Summary: awaitTermination() after stop() will hang in Spark Stremaing Key: SPARK-2113 URL: https://issues.apache.org/jira/browse/SPARK-2113 Project: Spark

[jira] [Resolved] (SPARK-2113) awaitTermination() after stop() will hang in Spark Stremaing

[ https://issues.apache.org/jira/browse/SPARK-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2113. Resolution: Fixed Fixed by: https://github.com/apache/spark/pull/1001 awaitTermination()

[jira] [Updated] (SPARK-2061) Deprecate `splits` in JavaRDDLike and add `partitions`

[ https://issues.apache.org/jira/browse/SPARK-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2061: --- Assignee: Anant Daksh Asthana (was: Patrick Wendell) Deprecate `splits` in JavaRDDLike and

[jira] [Assigned] (SPARK-2061) Deprecate `splits` in JavaRDDLike and add `partitions`

[ https://issues.apache.org/jira/browse/SPARK-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reassigned SPARK-2061: -- Assignee: Patrick Wendell Deprecate `splits` in JavaRDDLike and add `partitions`

[jira] [Commented] (SPARK-2100) Allow users to disable Jetty Spark UI in local mode

[ https://issues.apache.org/jira/browse/SPARK-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028133#comment-14028133 ] Patrick Wendell commented on SPARK-2100: Could you elaborate a bit more on the

[jira] [Commented] (SPARK-2061) Deprecate `splits` in JavaRDDLike and add `partitions`

[ https://issues.apache.org/jira/browse/SPARK-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028128#comment-14028128 ] Patrick Wendell commented on SPARK-2061: Go for it! Deprecate `splits` in

[jira] [Created] (SPARK-2114) Aggregations on raw data

Sandy Ryza created SPARK-2114: - Summary: Aggregations on raw data Key: SPARK-2114 URL: https://issues.apache.org/jira/browse/SPARK-2114 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-2106) Unify the HiveContext

[ https://issues.apache.org/jira/browse/SPARK-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028233#comment-14028233 ] Michael Armbrust commented on SPARK-2106: - Regarding the first point, I actually

[jira] [Comment Edited] (SPARK-2081) Undefine output() from the abstract class Command and implement it in concrete subclasses

[ https://issues.apache.org/jira/browse/SPARK-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028245#comment-14028245 ] Cheng Lian edited comment on SPARK-2081 at 6/11/14 7:12 PM: To

[jira] [Updated] (SPARK-2081) Undefine output() from the abstract class Command and implement it in concrete subclasses

[ https://issues.apache.org/jira/browse/SPARK-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2081: Assignee: Cheng Lian (was: Zongheng Yang) Undefine output() from the abstract class

[jira] [Commented] (SPARK-2053) Add Catalyst expression for CASE WHEN

[ https://issues.apache.org/jira/browse/SPARK-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028541#comment-14028541 ] Zongheng Yang commented on SPARK-2053: -- Submitted pull request:

[jira] [Created] (SPARK-2115) Stage kill link is too close to stage details link

Nicholas Chammas created SPARK-2115: --- Summary: Stage kill link is too close to stage details link Key: SPARK-2115 URL: https://issues.apache.org/jira/browse/SPARK-2115 Project: Spark Issue

[jira] [Updated] (SPARK-2115) Stage kill link is too close to stage details link

[ https://issues.apache.org/jira/browse/SPARK-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2115: Attachment: Accident-prone kill link.png Safe and user-friendly kill

[jira] [Updated] (SPARK-2116) Load spark-defaults.conf from directory specified by SPARK_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Albert Chu updated SPARK-2116: -- Attachment: SPARK-2116.patch Load spark-defaults.conf from directory specified by SPARK_CONF_DIR

[jira] [Created] (SPARK-2116) Load spark-defaults.conf from directory specified by SPARK_CONF_DIR

Albert Chu created SPARK-2116: - Summary: Load spark-defaults.conf from directory specified by SPARK_CONF_DIR Key: SPARK-2116 URL: https://issues.apache.org/jira/browse/SPARK-2116 Project: Spark

[jira] [Created] (SPARK-2117) UTF8 Characters Break PySpark

Michael Armbrust created SPARK-2117: --- Summary: UTF8 Characters Break PySpark Key: SPARK-2117 URL: https://issues.apache.org/jira/browse/SPARK-2117 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2114) Aggregations on raw data

[ https://issues.apache.org/jira/browse/SPARK-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-2114: -- Description: For groupByKey and join transformations, Spark tasks on the reduce side deserialize

[jira] [Updated] (SPARK-2114) groupByKey and joins on raw data

[ https://issues.apache.org/jira/browse/SPARK-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-2114: -- Summary: groupByKey and joins on raw data (was: Aggregations on raw data) groupByKey and joins on

[jira] [Resolved] (SPARK-2117) UTF8 Characters Break PySpark

[ https://issues.apache.org/jira/browse/SPARK-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2117. - Resolution: Won't Fix i realized this was actually an issue with the CSV reader. UTF8

[jira] [Created] (SPARK-2118) If tools jar is not present, MIMA build should exit with an exception

Patrick Wendell created SPARK-2118: -- Summary: If tools jar is not present, MIMA build should exit with an exception Key: SPARK-2118 URL: https://issues.apache.org/jira/browse/SPARK-2118 Project:

[jira] [Updated] (SPARK-2118) If tools jar is not present, MIMA build should exit with an exception

[ https://issues.apache.org/jira/browse/SPARK-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2118: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-2069 If tools jar is not

[jira] [Commented] (SPARK-2099) Report metrics for running tasks

[ https://issues.apache.org/jira/browse/SPARK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028681#comment-14028681 ] Sandy Ryza commented on SPARK-2099: --- https://github.com/apache/spark/pull/1056 Report

[jira] [Closed] (SPARK-1672) Support separate partitioners (and numbers of partitions) for users and products

[ https://issues.apache.org/jira/browse/SPARK-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-1672. Resolution: Implemented Assignee: Tor Myklebust PR:

[jira] [Commented] (SPARK-2106) Unify the HiveContext

[ https://issues.apache.org/jira/browse/SPARK-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028699#comment-14028699 ] Cheng Hao commented on SPARK-2106: -- Oh, I see your point. Actually I was suggesting is we

[jira] [Commented] (SPARK-2106) Unify the HiveContext

[ https://issues.apache.org/jira/browse/SPARK-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028705#comment-14028705 ] Cheng Hao commented on SPARK-2106: -- BTW, call hql(...).collect() is a good way to

[jira] [Commented] (SPARK-2106) Unify the HiveContext

[ https://issues.apache.org/jira/browse/SPARK-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028709#comment-14028709 ] Michael Armbrust commented on SPARK-2106: - SchemaRDD has a method `queryExecution`

[jira] [Created] (SPARK-2119) Reading Parquet InputSplits dominates query execution time when reading off S3

Michael Armbrust created SPARK-2119: --- Summary: Reading Parquet InputSplits dominates query execution time when reading off S3 Key: SPARK-2119 URL: https://issues.apache.org/jira/browse/SPARK-2119

[jira] [Created] (SPARK-2120) Not fully cached when there is enough memory

Shuo Xiang created SPARK-2120: - Summary: Not fully cached when there is enough memory Key: SPARK-2120 URL: https://issues.apache.org/jira/browse/SPARK-2120 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2121) Not fully cached when there is enough memory

Shuo Xiang created SPARK-2121: - Summary: Not fully cached when there is enough memory Key: SPARK-2121 URL: https://issues.apache.org/jira/browse/SPARK-2121 Project: Spark Issue Type: Bug

[jira] [Closed] (SPARK-2120) Not fully cached when there is enough memory

[ https://issues.apache.org/jira/browse/SPARK-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang closed SPARK-2120. - Resolution: Duplicate Not fully cached when there is enough memory

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028715#comment-14028715 ] Madhu Siddalingaiah commented on SPARK-983: --- [Aaron

[jira] [Comment Edited] (SPARK-983) Support external sorting for RDD#sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028715#comment-14028715 ] Madhu Siddalingaiah edited comment on SPARK-983 at 6/12/14 1:52 AM:

[jira] [Created] (SPARK-2122) Move aggregation into shuffle implementation

Saisai Shao created SPARK-2122: -- Summary: Move aggregation into shuffle implementation Key: SPARK-2122 URL: https://issues.apache.org/jira/browse/SPARK-2122 Project: Spark Issue Type:

[jira] [Created] (SPARK-2123) Basic pluggable interface for shuffle

Matei Zaharia created SPARK-2123: Summary: Basic pluggable interface for shuffle Key: SPARK-2123 URL: https://issues.apache.org/jira/browse/SPARK-2123 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-2125) Add sorting flag to ShuffleManager, and implement it in HashShuffleManager

Matei Zaharia created SPARK-2125: Summary: Add sorting flag to ShuffleManager, and implement it in HashShuffleManager Key: SPARK-2125 URL: https://issues.apache.org/jira/browse/SPARK-2125 Project:

[jira] [Created] (SPARK-2124) Move aggregation into ShuffleManager implementations

Matei Zaharia created SPARK-2124: Summary: Move aggregation into ShuffleManager implementations Key: SPARK-2124 URL: https://issues.apache.org/jira/browse/SPARK-2124 Project: Spark Issue

[jira] [Updated] (SPARK-2124) Move aggregation into ShuffleManager implementations

[ https://issues.apache.org/jira/browse/SPARK-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2124: - Assignee: Saisai Shao Move aggregation into ShuffleManager implementations

[jira] [Resolved] (SPARK-2123) Basic pluggable interface for shuffle

[ https://issues.apache.org/jira/browse/SPARK-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2123. -- Resolution: Fixed Resolved in https://github.com/apache/spark/pull/1009 Basic pluggable

[jira] [Created] (SPARK-2126) Move MapOutputTracker behind ShuffleManager interface

Matei Zaharia created SPARK-2126: Summary: Move MapOutputTracker behind ShuffleManager interface Key: SPARK-2126 URL: https://issues.apache.org/jira/browse/SPARK-2126 Project: Spark Issue

[jira] [Commented] (SPARK-2125) Add sorting flag to ShuffleManager, and implement it in HashShuffleManager

[ https://issues.apache.org/jira/browse/SPARK-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028833#comment-14028833 ] Saisai Shao commented on SPARK-2125: Hi Matei, for moving sort into hash shuffle

[jira] [Created] (SPARK-2127) Use application specific folders to dump metrics via CsvSink

Rahul Singhal created SPARK-2127: Summary: Use application specific folders to dump metrics via CsvSink Key: SPARK-2127 URL: https://issues.apache.org/jira/browse/SPARK-2127 Project: Spark

[jira] [Commented] (SPARK-2127) Use application specific folders to dump metrics via CsvSink

[ https://issues.apache.org/jira/browse/SPARK-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028854#comment-14028854 ] Rahul Singhal commented on SPARK-2127: -- A PR is in the works. Use application