[jira] [Created] (SPARK-3000) drop old blocks to disk in parallel when memory is not large enough for caching new blocks

Zhang, Liye created SPARK-3000: -- Summary: drop old blocks to disk in parallel when memory is not large enough for caching new blocks Key: SPARK-3000 URL: https://issues.apache.org/jira/browse/SPARK-3000

[jira] [Updated] (SPARK-3000) drop old blocks to disk in parallel when memory is not large enough for caching new blocks

[ https://issues.apache.org/jira/browse/SPARK-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang, Liye updated SPARK-3000: --- Description: In spark, rdd can be cached in memory for later use, and the cached memory size

[jira] [Comment Edited] (SPARK-2372) Grouped Optimization/Learning

[ https://issues.apache.org/jira/browse/SPARK-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094668#comment-14094668 ] Kyle Ellrott edited comment on SPARK-2372 at 8/13/14 6:06 AM: --

[jira] [Updated] (SPARK-3000) drop old blocks to disk in parallel when memory is not large enough for caching new blocks

[ https://issues.apache.org/jira/browse/SPARK-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang, Liye updated SPARK-3000: --- Remaining Estimate: (was: 168h) Original Estimate: (was: 168h) drop old blocks to disk

[jira] [Updated] (SPARK-2998) scala.collection.mutable.HashSet cannot be cast to scala.collection.mutable.BitSet

[ https://issues.apache.org/jira/browse/SPARK-2998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengyanhong updated SPARK-2998: --- Description: run a HiveQL via yarn-cluster, got error as below: {quote} 14/08/13 11:10:01 INFO

[jira] [Created] (SPARK-3001) Improve Spearman's correlation

Xiangrui Meng created SPARK-3001: Summary: Improve Spearman's correlation Key: SPARK-3001 URL: https://issues.apache.org/jira/browse/SPARK-3001 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3001) Improve Spearman's correlation

[ https://issues.apache.org/jira/browse/SPARK-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095207#comment-14095207 ] Apache Spark commented on SPARK-3001: - User 'mengxr' has created a pull request for

[jira] [Created] (SPARK-3002) Reuse Netty clients

Reynold Xin created SPARK-3002: -- Summary: Reuse Netty clients Key: SPARK-3002 URL: https://issues.apache.org/jira/browse/SPARK-3002 Project: Spark Issue Type: Sub-task Reporter:

[jira] [Updated] (SPARK-3002) Reuse Netty clients

[ https://issues.apache.org/jira/browse/SPARK-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3002: --- Description: To create a client manager that reuses clients (and connections). Can also use

[jira] [Resolved] (SPARK-2993) colStats in Statistics as wrapper around MultivariateStatisticalSummary in Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-2993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2993. -- Resolution: Implemented Fix Version/s: 1.1.0 Target Version/s: 1.1.0

[jira] [Created] (SPARK-3003) FailedStage could not be cancelled by DAGScheduler when cancelJob or cancelStage

YanTang Zhai created SPARK-3003: --- Summary: FailedStage could not be cancelled by DAGScheduler when cancelJob or cancelStage Key: SPARK-3003 URL: https://issues.apache.org/jira/browse/SPARK-3003

[jira] [Commented] (SPARK-2890) Spark SQL should allow SELECT with duplicated columns

[ https://issues.apache.org/jira/browse/SPARK-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095224#comment-14095224 ] Jianshi Huang commented on SPARK-2890: -- I think the fault is on my side. I should've

[jira] [Closed] (SPARK-2890) Spark SQL should allow SELECT with duplicated columns

[ https://issues.apache.org/jira/browse/SPARK-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang closed SPARK-2890. Resolution: Invalid Spark SQL should allow SELECT with duplicated columns

[jira] [Created] (SPARK-3004) HiveThriftServer2 throws exception when the result set contains NULL

Cheng Lian created SPARK-3004: - Summary: HiveThriftServer2 throws exception when the result set contains NULL Key: SPARK-3004 URL: https://issues.apache.org/jira/browse/SPARK-3004 Project: Spark

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095232#comment-14095232 ] Debasish Das commented on SPARK-2426: - Hi Xiangrui, The branch is ready for an

[jira] [Updated] (SPARK-2973) Add a way to show tables without executing a job

[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2973: Target Version/s: 1.2.0 Add a way to show tables without executing a job

[jira] [Commented] (SPARK-2973) Add a way to show tables without executing a job

[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095241#comment-14095241 ] Michael Armbrust commented on SPARK-2973: - We can just override executeCollect()

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095247#comment-14095247 ] Mridul Muralidharan commented on SPARK-2089: Since I am not maintaining the

[jira] [Updated] (SPARK-2969) Make ScalaReflection be able to handle ArrayType.containsNull and MapType.valueContainsNull.

[ https://issues.apache.org/jira/browse/SPARK-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-2969: - Summary: Make ScalaReflection be able to handle ArrayType.containsNull and

[jira] [Created] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

Xu Zhongxing created SPARK-3005: --- Summary: Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask() Key: SPARK-3005 URL:

[jira] [Created] (SPARK-3006) Failed to execute spark-shell in Windows OS

Masayoshi TSUZUKI created SPARK-3006: Summary: Failed to execute spark-shell in Windows OS Key: SPARK-3006 URL: https://issues.apache.org/jira/browse/SPARK-3006 Project: Spark Issue

[jira] [Commented] (SPARK-3006) Failed to execute spark-shell in Windows OS

[ https://issues.apache.org/jira/browse/SPARK-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095284#comment-14095284 ] Masayoshi TSUZUKI commented on SPARK-3006: -- This is because the option {{--class

[jira] [Commented] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095285#comment-14095285 ] Xu Zhongxing commented on SPARK-3005: - A related question: why does fined-grain mode

[jira] [Created] (SPARK-3007) Add Dynamic Partition support to Spark Sql hive

baishuo created SPARK-3007: -- Summary: Add Dynamic Partition support to Spark Sql hive Key: SPARK-3007 URL: https://issues.apache.org/jira/browse/SPARK-3007 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3006) Failed to execute spark-shell in Windows OS

[ https://issues.apache.org/jira/browse/SPARK-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095286#comment-14095286 ] Apache Spark commented on SPARK-3006: - User 'tsudukim' has created a pull request for

[jira] [Commented] (SPARK-3007) Add Dynamic Partition support to Spark Sql hive

[ https://issues.apache.org/jira/browse/SPARK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095300#comment-14095300 ] baishuo commented on SPARK-3007: after modify the code, I can run the hiveql with dynamic

[jira] [Comment Edited] (SPARK-3007) Add Dynamic Partition support to Spark Sql hive

[ https://issues.apache.org/jira/browse/SPARK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095300#comment-14095300 ] baishuo edited comment on SPARK-3007 at 8/13/14 9:08 AM: - after

[jira] [Comment Edited] (SPARK-3007) Add Dynamic Partition support to Spark Sql hive

[ https://issues.apache.org/jira/browse/SPARK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095300#comment-14095300 ] baishuo edited comment on SPARK-3007 at 8/13/14 9:10 AM: - after

[jira] [Commented] (SPARK-3004) HiveThriftServer2 throws exception when the result set contains NULL

[ https://issues.apache.org/jira/browse/SPARK-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095308#comment-14095308 ] Apache Spark commented on SPARK-3004: - User 'liancheng' has created a pull request for

[jira] [Issue Comment Deleted] (SPARK-2204) Scheduler for Mesos in fine-grained mode launches tasks on wrong executors

[ https://issues.apache.org/jira/browse/SPARK-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Zhongxing updated SPARK-2204: Comment: was deleted (was: I encountered this issue again when I use Spark 1.0.2, Mesos 0.18.1,

[jira] [Created] (SPARK-3008) PySpark fails due to zipimport not able to load the assembly jar (/usr/bin/python: No module named pyspark)

Jai Kumar Singh created SPARK-3008: -- Summary: PySpark fails due to zipimport not able to load the assembly jar (/usr/bin/python: No module named pyspark) Key: SPARK-3008 URL:

[jira] [Created] (SPARK-3009) ApplicationInfo doesn't get initialised after deserialisation during recovery

Jacek Lewandowski created SPARK-3009: Summary: ApplicationInfo doesn't get initialised after deserialisation during recovery Key: SPARK-3009 URL: https://issues.apache.org/jira/browse/SPARK-3009

[jira] [Commented] (SPARK-3003) FailedStage could not be cancelled by DAGScheduler when cancelJob or cancelStage

[ https://issues.apache.org/jira/browse/SPARK-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095474#comment-14095474 ] Apache Spark commented on SPARK-3003: - User 'YanTangZhai' has created a pull request

[jira] [Commented] (SPARK-3009) ApplicationInfo doesn't get initialised after deserialisation during recovery

[ https://issues.apache.org/jira/browse/SPARK-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095479#comment-14095479 ] Apache Spark commented on SPARK-3009: - User 'jacek-lewandowski' has created a pull

[jira] [Commented] (SPARK-3009) ApplicationInfo doesn't get initialised after deserialisation during recovery

[ https://issues.apache.org/jira/browse/SPARK-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095483#comment-14095483 ] Jacek Lewandowski commented on SPARK-3009: -- [~andrewor14] could you review it

[jira] [Updated] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Debasish Das updated SPARK-2426: Description: Current ALS supports least squares and nonnegative least squares. I presented ADMM

[jira] [Created] (SPARK-3010) fix redundant conditional

wangfei created SPARK-3010: -- Summary: fix redundant conditional Key: SPARK-3010 URL: https://issues.apache.org/jira/browse/SPARK-3010 Project: Spark Issue Type: Improvement Components:

[jira] [Comment Edited] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095232#comment-14095232 ] Debasish Das edited comment on SPARK-2426 at 8/13/14 3:31 PM: --

[jira] [Commented] (SPARK-3010) fix redundant conditional

[ https://issues.apache.org/jira/browse/SPARK-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095587#comment-14095587 ] Apache Spark commented on SPARK-3010: - User 'scwf' has created a pull request for this

[jira] [Created] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

Joseph Su created SPARK-3011: Summary: _temporary directory should be filtered out by sqlContext.parquetFile Key: SPARK-3011 URL: https://issues.apache.org/jira/browse/SPARK-3011 Project: Spark

[jira] [Commented] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

[ https://issues.apache.org/jira/browse/SPARK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095636#comment-14095636 ] Sean Owen commented on SPARK-3011: -- Duplicate, or very closely related:

[jira] [Commented] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

[ https://issues.apache.org/jira/browse/SPARK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095641#comment-14095641 ] Apache Spark commented on SPARK-3011: - User 'joesu' has created a pull request for

[jira] [Commented] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

[ https://issues.apache.org/jira/browse/SPARK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095640#comment-14095640 ] Joseph Su commented on SPARK-3011: -- Pull request is here:

[jira] [Created] (SPARK-3012) Standardized Distance Functions between two Vectors for MLlib

Yu Ishikawa created SPARK-3012: -- Summary: Standardized Distance Functions between two Vectors for MLlib Key: SPARK-3012 URL: https://issues.apache.org/jira/browse/SPARK-3012 Project: Spark

[jira] [Created] (SPARK-3013) Doctest of inferSchema in Spark SQL Python API fails

Cheng Lian created SPARK-3013: - Summary: Doctest of inferSchema in Spark SQL Python API fails Key: SPARK-3013 URL: https://issues.apache.org/jira/browse/SPARK-3013 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095721#comment-14095721 ] Thomas Graves commented on SPARK-2140: -- ah it seems things have changed. Its now

[jira] [Updated] (SPARK-3013) Doctest of inferSchema in Spark SQL Python API fails

[ https://issues.apache.org/jira/browse/SPARK-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3013: Assignee: Davies Liu Doctest of inferSchema in Spark SQL Python API fails

[jira] [Commented] (SPARK-1442) Add Window function support

[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095861#comment-14095861 ] Adam Nowak commented on SPARK-1442: --- Does the Spark SQLContext support windowing

[jira] [Commented] (SPARK-2846) Spark SQL hive implementation bypass StorageHandler which breaks any customized StorageHandler

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095879#comment-14095879 ] Alex Liu commented on SPARK-2846: - pull @ https://github.com/apache/spark/pull/1927

[jira] [Updated] (SPARK-2969) Make ScalaReflection be able to handle ArrayType.containsNull and MapType.valueContainsNull.

[ https://issues.apache.org/jira/browse/SPARK-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2969: Priority: Critical (was: Major) Make ScalaReflection be able to handle

[jira] [Commented] (SPARK-2846) Spark SQL hive implementation bypass StorageHandler which breaks any customized StorageHandler

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095925#comment-14095925 ] Apache Spark commented on SPARK-2846: - User 'alexliu68' has created a pull request for

[jira] [Commented] (SPARK-3013) Doctest of inferSchema in Spark SQL Python API fails

[ https://issues.apache.org/jira/browse/SPARK-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095942#comment-14095942 ] Apache Spark commented on SPARK-3013: - User 'davies' has created a pull request for

[jira] [Updated] (SPARK-2846) Add configureInputJobPropertiesForStorageHandler to initialization of job conf

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-2846: Summary: Add configureInputJobPropertiesForStorageHandler to initialization of job conf (was: Spark SQL

[jira] [Updated] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1391: --- Issue Type: Improvement (was: Bug) BlockManager cannot transfer blocks larger than 2G in size

[jira] [Updated] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1391: --- Assignee: (was: Min Zhou) BlockManager cannot transfer blocks larger than 2G in size

[jira] [Updated] (SPARK-1297) Upgrade HBase dependency to 0.98.0

[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1297: --- Assignee: Ted Yu Upgrade HBase dependency to 0.98.0 --

[jira] [Created] (SPARK-3014) Log a more informative message when yarn-cluster app fails because SparkContext wasn't initialized

Sandy Ryza created SPARK-3014: - Summary: Log a more informative message when yarn-cluster app fails because SparkContext wasn't initialized Key: SPARK-3014 URL: https://issues.apache.org/jira/browse/SPARK-3014

[jira] [Updated] (SPARK-3014) Log a more informative messages in a couple failure scenarios

[ https://issues.apache.org/jira/browse/SPARK-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-3014: -- Summary: Log a more informative messages in a couple failure scenarios (was: Log a more informative

[jira] [Updated] (SPARK-3014) Log a more informative messages in a couple failure scenarios

[ https://issues.apache.org/jira/browse/SPARK-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-3014: -- Description: This is what shows up currently when the user code fails to initialize a SparkContext

[jira] [Created] (SPARK-3015) Removing broadcast in quick successions causes Akka timeout

Andrew Or created SPARK-3015: Summary: Removing broadcast in quick successions causes Akka timeout Key: SPARK-3015 URL: https://issues.apache.org/jira/browse/SPARK-3015 Project: Spark Issue

[jira] [Resolved] (SPARK-2983) improve performance of sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2983. -- Resolution: Fixed Fix Version/s: 1.1.0 improve performance of sortByKey()

[jira] [Created] (SPARK-3016) Client should be able to put blocks in addition to fetch blocks

Reynold Xin created SPARK-3016: -- Summary: Client should be able to put blocks in addition to fetch blocks Key: SPARK-3016 URL: https://issues.apache.org/jira/browse/SPARK-3016 Project: Spark

[jira] [Created] (SPARK-3017) Implement unit/integration tests for connection failures

Reynold Xin created SPARK-3017: -- Summary: Implement unit/integration tests for connection failures Key: SPARK-3017 URL: https://issues.apache.org/jira/browse/SPARK-3017 Project: Spark Issue

[jira] [Commented] (SPARK-3015) Removing broadcast in quick successions causes Akka timeout

[ https://issues.apache.org/jira/browse/SPARK-3015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096251#comment-14096251 ] Apache Spark commented on SPARK-3015: - User 'andrewor14' has created a pull request

[jira] [Updated] (SPARK-3018) Release all BlockFetcherIterator upon task completion/failure

[ https://issues.apache.org/jira/browse/SPARK-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3018: --- Description: BlockFetcherIterator retains ReferenceCountedBuffers returned by client.fetchBlocks.

[jira] [Commented] (SPARK-2907) Use mutable.HashMap to represent Model in Word2Vec

[ https://issues.apache.org/jira/browse/SPARK-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096301#comment-14096301 ] Apache Spark commented on SPARK-2907: - User 'Ishiihara' has created a pull request for

[jira] [Created] (SPARK-3020) Print completed indices rather than tasks in web UI

Patrick Wendell created SPARK-3020: -- Summary: Print completed indices rather than tasks in web UI Key: SPARK-3020 URL: https://issues.apache.org/jira/browse/SPARK-3020 Project: Spark Issue

[jira] [Resolved] (SPARK-2817) add show create table support

[ https://issues.apache.org/jira/browse/SPARK-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2817. - Resolution: Fixed Fix Version/s: 1.1.0 add show create table support

[jira] [Resolved] (SPARK-3004) HiveThriftServer2 throws exception when the result set contains NULL

[ https://issues.apache.org/jira/browse/SPARK-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3004. - Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Cheng Lian

[jira] [Resolved] (SPARK-2963) The description about building to use HiveServer and CLI is incomplete

[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2963. - Resolution: Fixed Fix Version/s: 1.1.0 The description about building to use

[jira] [Resolved] (SPARK-3013) Doctest of inferSchema in Spark SQL Python API fails

[ https://issues.apache.org/jira/browse/SPARK-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3013. - Resolution: Fixed Fix Version/s: 1.1.0 Doctest of inferSchema in Spark SQL

[jira] [Created] (SPARK-3021) Job remains in Active Stages after failing

Michael Armbrust created SPARK-3021: --- Summary: Job remains in Active Stages after failing Key: SPARK-3021 URL: https://issues.apache.org/jira/browse/SPARK-3021 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-2994) Support for Hive UDFs that take arrays of structs as arguments

[ https://issues.apache.org/jira/browse/SPARK-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2994. - Resolution: Fixed Fix Version/s: 1.1.0 Support for Hive UDFs that take arrays of

[jira] [Resolved] (SPARK-2935) Failure with push down of conjunctive parquet predicates

[ https://issues.apache.org/jira/browse/SPARK-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2935. - Resolution: Fixed Fix Version/s: 1.1.0 Failure with push down of conjunctive

[jira] [Resolved] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2970. - Resolution: Fixed Fix Version/s: 1.1.0 spark-sql script ends with IOException

[jira] [Resolved] (SPARK-3020) Print completed indices rather than tasks in web UI

[ https://issues.apache.org/jira/browse/SPARK-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3020. Resolution: Fixed Fix Version/s: 1.1.0 Print completed indices rather than tasks in web UI

[jira] [Assigned] (SPARK-2625) Fix ShuffleReadMetrics for NettyBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-2625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-2625: -- Assignee: Reynold Xin Fix ShuffleReadMetrics for NettyBlockFetcherIterator

[jira] [Updated] (SPARK-2625) Fix ShuffleReadMetrics for NettyBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-2625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2625: --- Component/s: Spark Core Fix ShuffleReadMetrics for NettyBlockFetcherIterator

[jira] [Updated] (SPARK-3022) FindBinsForLevel in decision tree should call findBin only once for each feature

[ https://issues.apache.org/jira/browse/SPARK-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiping Li updated SPARK-3022: - Description: `findbinsForLevel` is applied to every `LabeledPoint` to find bins for all nodes at a given

[jira] [Commented] (SPARK-3022) FindBinsForLevel in decision tree should call findBin only once for each feature

[ https://issues.apache.org/jira/browse/SPARK-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096526#comment-14096526 ] Qiping Li commented on SPARK-3022: -- What's more, there's no need to store `feature2bins`

[jira] [Updated] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] OuyangJin updated SPARK-3005: - Attachment: SPARK-3005_1.diff a quick fix for fine grained killTask Spark with Mesos fine-grained

[jira] [Created] (SPARK-3024) CLI interface to Driver

Jeff Hammerbacher created SPARK-3024: Summary: CLI interface to Driver Key: SPARK-3024 URL: https://issues.apache.org/jira/browse/SPARK-3024 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3023) SIGINT to driver with yarn-client should release containers on the cluster

Jeff Hammerbacher created SPARK-3023: Summary: SIGINT to driver with yarn-client should release containers on the cluster Key: SPARK-3023 URL: https://issues.apache.org/jira/browse/SPARK-3023

[jira] [Updated] (SPARK-3023) SIGINT to driver with yarn-client should release containers on the cluster

[ https://issues.apache.org/jira/browse/SPARK-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Hammerbacher updated SPARK-3023: - Issue Type: Improvement (was: Bug) SIGINT to driver with yarn-client should release

[jira] [Commented] (SPARK-3024) CLI interface to Driver

[ https://issues.apache.org/jira/browse/SPARK-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096541#comment-14096541 ] Patrick Wendell commented on SPARK-3024: Hey Jeff - mind giving a bit more color

[jira] [Commented] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096539#comment-14096539 ] Xu Zhongxing commented on SPARK-3005: - Could adding an empty killTask method to

[jira] [Commented] (SPARK-3024) CLI interface to Driver

[ https://issues.apache.org/jira/browse/SPARK-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096547#comment-14096547 ] Jeff Hammerbacher commented on SPARK-3024: -- It would be nice to be able to list

[jira] [Created] (SPARK-3025) Allow JDBC clients to set a fair scheduler pool

Patrick Wendell created SPARK-3025: -- Summary: Allow JDBC clients to set a fair scheduler pool Key: SPARK-3025 URL: https://issues.apache.org/jira/browse/SPARK-3025 Project: Spark Issue

[jira] [Updated] (SPARK-3026) Provide a good error message if JDBC server is used but Spark is not compiled with -Pthriftserver

[ https://issues.apache.org/jira/browse/SPARK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3026: --- Priority: Critical (was: Major) Provide a good error message if JDBC server is used but

[jira] [Created] (SPARK-3026) Provide a good error message if JDBC server is used but Spark is not compiled with -Pthriftserver

Patrick Wendell created SPARK-3026: -- Summary: Provide a good error message if JDBC server is used but Spark is not compiled with -Pthriftserver Key: SPARK-3026 URL:

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3019: --- Attachment: PluggableBlockTransferServiceProposalforSpark.pdf Design Doc - draft 1 Pluggable block

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3019: --- Description: The attached design doc proposes a standard interface for block transferring, which

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3019: --- Attachment: (was: PluggableBlockTransferServiceProposalforSpark.pdf) Pluggable block transfer

[jira] [Updated] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3005: --- Description: I am using Spark, Mesos, spark-cassandra-connector to do some work on a cassandra

[jira] [Updated] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2356: --- Description: I'm trying to run some transformation on Spark, it works fine on cluster (YARN, linux

[jira] [Commented] (SPARK-3025) Allow JDBC clients to set a fair scheduler pool

[ https://issues.apache.org/jira/browse/SPARK-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096580#comment-14096580 ] Apache Spark commented on SPARK-3025: - User 'pwendell' has created a pull request for

[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096601#comment-14096601 ] Guoqiang Li commented on SPARK-2356: This should be problems caused by not set

[jira] [Updated] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-2926: --- Attachment: Spark Shuffle Test Report.pdf Add MR-style (merge-sort) SortShuffleReader for

[jira] [Comment Edited] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096539#comment-14096539 ] Xu Zhongxing edited comment on SPARK-3005 at 8/14/14 5:57 AM: --