[jira] [Commented] (SPARK-7888) Be able to disable intercept in Linear Regression in ML package

[ https://issues.apache.org/jira/browse/SPARK-7888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590713#comment-14590713 ] DB Tsai commented on SPARK-7888: Yeah, we don't re-center but still scaling to unit

[jira] [Resolved] (SPARK-8306) AddJar command needs to set the new class loader to the HiveConf inside executionHive.state.

[ https://issues.apache.org/jira/browse/SPARK-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8306. - Resolution: Fixed Fix Version/s: 1.5.0 1.4.1 AddJar command

[jira] [Updated] (SPARK-8392) RDDOperationGraph: getting cached nodes is slow

[ https://issues.apache.org/jira/browse/SPARK-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8392: - Summary: RDDOperationGraph: getting cached nodes is slow (was: the process is hang on when getting

[jira] [Updated] (SPARK-8392) RDDOperationGraph: getting cached nodes is slow

[ https://issues.apache.org/jira/browse/SPARK-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8392: - Affects Version/s: 1.4.0 RDDOperationGraph: getting cached nodes is slow

[jira] [Comment Edited] (SPARK-8406) Race condition when writing Parquet files

[ https://issues.apache.org/jira/browse/SPARK-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589592#comment-14589592 ] Cheng Lian edited comment on SPARK-8406 at 6/17/15 6:07 PM: An

[jira] [Updated] (SPARK-7075) Project Tungsten: Improving Physical Execution and Memory Management

[ https://issues.apache.org/jira/browse/SPARK-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7075: --- Description: Based on our observation, majority of Spark workloads are not bottlenecked by I/O or

[jira] [Created] (SPARK-8413) DirectParquetOutputCommitter doesn't clean up the file on task failure

Mingyu Kim created SPARK-8413: - Summary: DirectParquetOutputCommitter doesn't clean up the file on task failure Key: SPARK-8413 URL: https://issues.apache.org/jira/browse/SPARK-8413 Project: Spark

[jira] [Commented] (SPARK-8365) pyspark does not retain --packages or --jars passed on the command line as of 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590327#comment-14590327 ] Andrew Or commented on SPARK-8365: -- I'm still investigating ATM, but I don't believe so.

[jira] [Resolved] (SPARK-7020) Restrict module testing based on commit contents

[ https://issues.apache.org/jira/browse/SPARK-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-7020. --- Resolution: Fixed Fix Version/s: 1.5.0 Assignee: Brennon York I'm going to resolve

[jira] [Updated] (SPARK-5647) Output metrics do not show up for older hadoop versions ( 2.5)

[ https://issues.apache.org/jira/browse/SPARK-5647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5647: --- Target Version/s: (was: 1.4.0) Output metrics do not show up for older hadoop versions (

[jira] [Resolved] (SPARK-4227) Document external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4227. Resolution: Not A Problem Resolving since I think the conclusion is that this works.

[jira] [Updated] (SPARK-8369) Support dependency jar and files on HDFS in standalone cluster mode

[ https://issues.apache.org/jira/browse/SPARK-8369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-8369: --- Target Version/s: (was: 1.3.1, 1.4.0) Support dependency jar and files on HDFS in

[jira] [Updated] (SPARK-7355) FlakyTest - o.a.s.DriverSuite

[ https://issues.apache.org/jira/browse/SPARK-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7355: --- Target Version/s: 1.4.1 (was: 1.4.0) FlakyTest - o.a.s.DriverSuite

[jira] [Updated] (SPARK-6026) Eliminate the bypassMergeThreshold parameter and associated hash-ish shuffle within the Sort shuffle code

[ https://issues.apache.org/jira/browse/SPARK-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6026: --- Target Version/s: (was: 1.4.0) Eliminate the bypassMergeThreshold parameter and associated

[jira] [Updated] (SPARK-8010) Implict promote Numeric type to String type in HiveTypeCoercion

[ https://issues.apache.org/jira/browse/SPARK-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8010: Shepherd: Yin Huai (was: Michael Armbrust) Implict promote Numeric type to String type in

[jira] [Updated] (SPARK-7021) JUnit output for Python tests

[ https://issues.apache.org/jira/browse/SPARK-7021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7021: --- Target Version/s: 1.5.0 (was: 1.4.0) JUnit output for Python tests

[jira] [Updated] (SPARK-7016) Refactor dev/run-tests(-jenkins) from Bash to Python

[ https://issues.apache.org/jira/browse/SPARK-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7016: --- Target Version/s: 1.5.0 (was: 1.4.0) Refactor dev/run-tests(-jenkins) from Bash to Python

[jira] [Updated] (SPARK-5915) Spillable should check every N bytes rather than every 32 elements

[ https://issues.apache.org/jira/browse/SPARK-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5915: --- Target Version/s: (was: 1.4.0) Spillable should check every N bytes rather than every 32

[jira] [Resolved] (SPARK-3044) Create RSS feed for Spark News

[ https://issues.apache.org/jira/browse/SPARK-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3044. --- Resolution: Later I'm going to close this as Later since it's a super low priority and would take a

[jira] [Commented] (SPARK-8356) Reconcile callUDF and callUdf

[ https://issues.apache.org/jira/browse/SPARK-8356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590478#comment-14590478 ] Benjamin Fradet commented on SPARK-8356: [~marmbrus] Are we sure {{callUDF}} is

[jira] [Commented] (SPARK-595) Document local-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590495#comment-14590495 ] Josh Rosen commented on SPARK-595: -- Hey Justin, do you want to submit a PR for this? I'll

[jira] [Updated] (SPARK-8365) pyspark does not retain --packages or --jars passed on the command line as of 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8365: Assignee: Andrew Or pyspark does not retain --packages or --jars passed on the command

[jira] [Updated] (SPARK-8365) pyspark does not retain --packages or --jars passed on the command line as of 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8365: Component/s: SQL pyspark does not retain --packages or --jars passed on the command line

[jira] [Updated] (SPARK-8167) Tasks that fail due to YARN preemption can cause job failure

[ https://issues.apache.org/jira/browse/SPARK-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8167: - Priority: Blocker (was: Critical) Tasks that fail due to YARN preemption can cause job failure

[jira] [Updated] (SPARK-8167) Tasks that fail due to YARN preemption can cause job failure

[ https://issues.apache.org/jira/browse/SPARK-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8167: - Target Version/s: 1.5.0 Tasks that fail due to YARN preemption can cause job failure

[jira] [Updated] (SPARK-7859) Collect_SET behaves different under different version of JDK

[ https://issues.apache.org/jira/browse/SPARK-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-7859: Target Version/s: 1.5.0 Shepherd: Yin Huai Assignee: Cheng Hao

[jira] [Updated] (SPARK-7026) LeftSemiJoin can not work when it has both equal condition and not equal condition.

[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-7026: Shepherd: Michael Armbrust LeftSemiJoin can not work when it has both equal condition and

[jira] [Reopened] (SPARK-7026) LeftSemiJoin can not work when it has both equal condition and not equal condition.

[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reopened SPARK-7026: - Assignee: Adrian Wang LeftSemiJoin can not work when it has both equal condition and

[jira] [Created] (SPARK-8416) Thread dump page should highlight Spark executor threads

Josh Rosen created SPARK-8416: - Summary: Thread dump page should highlight Spark executor threads Key: SPARK-8416 URL: https://issues.apache.org/jira/browse/SPARK-8416 Project: Spark Issue Type:

[jira] [Commented] (SPARK-8356) Reconcile callUDF and callUdf

[ https://issues.apache.org/jira/browse/SPARK-8356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590519#comment-14590519 ] Michael Armbrust commented on SPARK-8356: - Its always better to have smaller more

[jira] [Commented] (SPARK-8356) Reconcile callUDF and callUdf

[ https://issues.apache.org/jira/browse/SPARK-8356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590521#comment-14590521 ] Benjamin Fradet commented on SPARK-8356: Ok, thanks a lot for your pointers.

[jira] [Closed] (SPARK-8395) spark-submit documentation is incorrect

[ https://issues.apache.org/jira/browse/SPARK-8395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8395. Resolution: Fixed Fix Version/s: 1.5.0 1.4.1 Assignee: Sean

[jira] [Closed] (SPARK-8161) externalBlockStoreInitialized is never set to be true

[ https://issues.apache.org/jira/browse/SPARK-8161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8161. Resolution: Fixed Fix Version/s: 1.5.0 1.4.1 Assignee:

[jira] [Updated] (SPARK-8161) externalBlockStoreInitialized is never set to be true

[ https://issues.apache.org/jira/browse/SPARK-8161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8161: - Affects Version/s: (was: 1.5.0) 1.4.0 externalBlockStoreInitialized is never

[jira] [Updated] (SPARK-6740) SQL operator and condition precedence is not honoured

[ https://issues.apache.org/jira/browse/SPARK-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6740: Target Version/s: 1.5.0 Shepherd: Michael Armbrust SQL operator and condition

[jira] [Updated] (SPARK-8406) Race condition when writing Parquet files

[ https://issues.apache.org/jira/browse/SPARK-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-8406: -- Description: To support appending, the Parquet data source tries to find out the max part number of

[jira] [Updated] (SPARK-8366) When task fails and append a new one, the ExecutorAllocationManager can't sense the new tasks

[ https://issues.apache.org/jira/browse/SPARK-8366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8366: - Affects Version/s: 1.4.0 When task fails and append a new one, the ExecutorAllocationManager can't

[jira] [Resolved] (SPARK-8404) Use thread-safe collections to make KafkaStreamSuite tests more reliable

[ https://issues.apache.org/jira/browse/SPARK-8404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-8404. -- Resolution: Fixed Assignee: Shixiong Zhu Target Version/s: 1.4.1, 1.5.0

[jira] [Commented] (SPARK-8368) ClassNotFoundException in closure for map

[ https://issues.apache.org/jira/browse/SPARK-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590636#comment-14590636 ] Yin Huai commented on SPARK-8368: - I have reproduced it. I am investigating it now.

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590680#comment-14590680 ] Zhan Zhang commented on SPARK-7009: --- [~airhorns] Please refer to

[jira] [Updated] (SPARK-6740) SQL operator and condition precedence is not honoured

[ https://issues.apache.org/jira/browse/SPARK-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6740: Assignee: Santiago M. Mola SQL operator and condition precedence is not honoured

[jira] [Commented] (SPARK-7334) Implement RandomProjection for Dimensionality Reduction

[ https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590711#comment-14590711 ] Yu Ishikawa commented on SPARK-7334: [~sebalf], I'm very sorry for the delay of my

[jira] [Commented] (SPARK-8390) Update DirectKafkaWordCount examples to show how offset ranges can be used

[ https://issues.apache.org/jira/browse/SPARK-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590718#comment-14590718 ] Apache Spark commented on SPARK-8390: - User 'koeninger' has created a pull request for

[jira] [Assigned] (SPARK-8390) Update DirectKafkaWordCount examples to show how offset ranges can be used

[ https://issues.apache.org/jira/browse/SPARK-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8390: --- Assignee: Apache Spark (was: Cody Koeninger) Update DirectKafkaWordCount examples to show

[jira] [Commented] (SPARK-8402) DP means clustering

[ https://issues.apache.org/jira/browse/SPARK-8402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591009#comment-14591009 ] Joseph K. Bradley commented on SPARK-8402: -- Feel free to go ahead and work on it.

[jira] [Updated] (SPARK-8402) DP means clustering

[ https://issues.apache.org/jira/browse/SPARK-8402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8402: - Target Version/s: (was: 1.5.0) DP means clustering

[jira] [Updated] (SPARK-8287) Filters not pushed with substitution through aggregation

[ https://issues.apache.org/jira/browse/SPARK-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8287: Issue Type: Improvement (was: Bug) Filters not pushed with substitution through

[jira] [Created] (SPARK-8422) Introduce a module abstraction inside of dev/run-tests

Josh Rosen created SPARK-8422: - Summary: Introduce a module abstraction inside of dev/run-tests Key: SPARK-8422 URL: https://issues.apache.org/jira/browse/SPARK-8422 Project: Spark Issue Type:

[jira] [Created] (SPARK-8423) More informative DecisionTreeModel.toDebugString

Justin Yip created SPARK-8423: - Summary: More informative DecisionTreeModel.toDebugString Key: SPARK-8423 URL: https://issues.apache.org/jira/browse/SPARK-8423 Project: Spark Issue Type:

[jira] [Commented] (SPARK-7888) Be able to disable intercept in Linear Regression in ML package

[ https://issues.apache.org/jira/browse/SPARK-7888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591119#comment-14591119 ] holdenk commented on SPARK-7888: Cool, I did a quick prototype as well but mine doesn't

[jira] [Commented] (SPARK-8368) ClassNotFoundException in closure for map

[ https://issues.apache.org/jira/browse/SPARK-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591154#comment-14591154 ] Yin Huai commented on SPARK-8368: - [~zwChan] I have found the cause. Will have a fix soon.

[jira] [Commented] (SPARK-8402) DP means clustering

[ https://issues.apache.org/jira/browse/SPARK-8402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591197#comment-14591197 ] Meethu Mathew commented on SPARK-8402: -- Could you please assign the ticket to me?

[jira] [Resolved] (SPARK-7605) Python API for ElementwiseProduct

[ https://issues.apache.org/jira/browse/SPARK-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-7605. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6346

[jira] [Assigned] (SPARK-8425) Add blacklist mechanism for task scheduling

[ https://issues.apache.org/jira/browse/SPARK-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8425: --- Assignee: Apache Spark Add blacklist mechanism for task scheduling

[jira] [Commented] (SPARK-8425) Add blacklist mechanism for task scheduling

[ https://issues.apache.org/jira/browse/SPARK-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591305#comment-14591305 ] Apache Spark commented on SPARK-8425: - User 'jerryshao' has created a pull request for

[jira] [Assigned] (SPARK-8425) Add blacklist mechanism for task scheduling

[ https://issues.apache.org/jira/browse/SPARK-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8425: --- Assignee: (was: Apache Spark) Add blacklist mechanism for task scheduling

[jira] [Updated] (SPARK-8428) TimSort Comparison method violates its general contract with CLUSTER BY

[ https://issues.apache.org/jira/browse/SPARK-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan McCarthy updated SPARK-8428: --- Description: Running an SQL query that has a sub query and multiple left joins fails when

[jira] [Closed] (SPARK-8392) RDDOperationGraph: getting cached nodes is slow

[ https://issues.apache.org/jira/browse/SPARK-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8392. Resolution: Fixed Fix Version/s: 1.5.0 1.4.1 Target Version/s: 1.4.1,

[jira] [Updated] (SPARK-8381) reuse typeConvert when convert Seq[Row] to catalyst type

[ https://issues.apache.org/jira/browse/SPARK-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-8381: -- Issue Type: Improvement (was: Bug) reuse typeConvert when convert Seq[Row] to catalyst type

[jira] [Resolved] (SPARK-8381) reuse typeConvert when convert Seq[Row] to catalyst type

[ https://issues.apache.org/jira/browse/SPARK-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-8381. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6831

[jira] [Updated] (SPARK-8381) reuse typeConvert when convert Seq[Row] to catalyst type

[ https://issues.apache.org/jira/browse/SPARK-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-8381: -- Assignee: Lianhui Wang reuse typeConvert when convert Seq[Row] to catalyst type

[jira] [Updated] (SPARK-6785) DateUtils can not handle date before 1970/01/01 correctly

[ https://issues.apache.org/jira/browse/SPARK-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6785: Assignee: Christian Kadner DateUtils can not handle date before 1970/01/01 correctly

[jira] [Created] (SPARK-8420) Inconsistent behavior with Dataframe Timestamp between 1.3.1 and 1.4.0

Justin Yip created SPARK-8420: - Summary: Inconsistent behavior with Dataframe Timestamp between 1.3.1 and 1.4.0 Key: SPARK-8420 URL: https://issues.apache.org/jira/browse/SPARK-8420 Project: Spark

[jira] [Commented] (SPARK-8420) Inconsistent behavior with Dataframe Timestamp between 1.3.1 and 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591051#comment-14591051 ] Justin Yip commented on SPARK-8420: --- [~yhuai] Inconsistent behavior with Dataframe

[jira] [Commented] (SPARK-8393) JavaStreamingContext#awaitTermination() throws non-declared InterruptedException

[ https://issues.apache.org/jira/browse/SPARK-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591130#comment-14591130 ] Tathagata Das commented on SPARK-8393: -- I think the best way is to catch all

[jira] [Updated] (SPARK-8420) Inconsistent behavior with Dataframe Timestamp between 1.3.1 and 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8420: Target Version/s: 1.4.1, 1.5.0 (was: 1.4.1) Inconsistent behavior with Dataframe Timestamp between 1.3.1

[jira] [Created] (SPARK-8424) Add blacklist mechanism for task scheduler and Yarn container allocation

Saisai Shao created SPARK-8424: -- Summary: Add blacklist mechanism for task scheduler and Yarn container allocation Key: SPARK-8424 URL: https://issues.apache.org/jira/browse/SPARK-8424 Project: Spark

[jira] [Updated] (SPARK-8301) Improve UTF8String substring/startsWith/endsWith/contains performance

[ https://issues.apache.org/jira/browse/SPARK-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8301: Assignee: (was: Michael Armbrust) Improve UTF8String

[jira] [Updated] (SPARK-8421) Spark SQL DATE_ADD function - Spark 1.3.1 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-8421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan McCarthy updated SPARK-8421: --- Description: Running with a parquet backed table in hive ‘dim_promo_date_curr_p' which has

[jira] [Updated] (SPARK-8301) Improve UTF8String substring/startsWith/endsWith/contains performance

[ https://issues.apache.org/jira/browse/SPARK-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8301: Shepherd: Davies Liu (was: Reynold Xin) Improve UTF8String

[jira] [Assigned] (SPARK-8422) Introduce a module abstraction inside of dev/run-tests

[ https://issues.apache.org/jira/browse/SPARK-8422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8422: --- Assignee: Josh Rosen (was: Apache Spark) Introduce a module abstraction inside of

[jira] [Commented] (SPARK-8422) Introduce a module abstraction inside of dev/run-tests

[ https://issues.apache.org/jira/browse/SPARK-8422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591084#comment-14591084 ] Apache Spark commented on SPARK-8422: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-8368) ClassNotFoundException in closure for map

[ https://issues.apache.org/jira/browse/SPARK-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591129#comment-14591129 ] CHEN Zhiwei commented on SPARK-8368: Great! I am not familiar to the class loader,

[jira] [Commented] (SPARK-8389) Expose KafkaRDDs offsetRange in Java and Python

[ https://issues.apache.org/jira/browse/SPARK-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591152#comment-14591152 ] Saisai Shao commented on SPARK-8389: I think for python API, it is not easy to add

[jira] [Commented] (SPARK-8368) ClassNotFoundException in closure for map

[ https://issues.apache.org/jira/browse/SPARK-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591171#comment-14591171 ] Yin Huai commented on SPARK-8368: - The cause of this problem is

[jira] [Created] (SPARK-8426) Add blacklist mechanism for YARN container allocation

Saisai Shao created SPARK-8426: -- Summary: Add blacklist mechanism for YARN container allocation Key: SPARK-8426 URL: https://issues.apache.org/jira/browse/SPARK-8426 Project: Spark Issue Type:

[jira] [Commented] (SPARK-8389) Expose KafkaRDDs offsetRange in Java and Python

[ https://issues.apache.org/jira/browse/SPARK-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591170#comment-14591170 ] Cody Koeninger commented on SPARK-8389: --- Static type doesn't really matter since

[jira] [Created] (SPARK-8425) Add blacklist mechanism for task scheduling

Saisai Shao created SPARK-8425: -- Summary: Add blacklist mechanism for task scheduling Key: SPARK-8425 URL: https://issues.apache.org/jira/browse/SPARK-8425 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-6009) IllegalArgumentException thrown by TimSort when SQL ORDER BY RAND ()

[ https://issues.apache.org/jira/browse/SPARK-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591245#comment-14591245 ] Nathan McCarthy edited comment on SPARK-6009 at 6/18/15 4:48 AM:

[jira] [Commented] (SPARK-6009) IllegalArgumentException thrown by TimSort when SQL ORDER BY RAND ()

[ https://issues.apache.org/jira/browse/SPARK-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591245#comment-14591245 ] Nathan McCarthy commented on SPARK-6009: Doesnt seem to just affect ORDER BY

[jira] [Updated] (SPARK-8345) Add an SQL node as a feature transformer

[ https://issues.apache.org/jira/browse/SPARK-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8345: - Summary: Add an SQL node as a feature transformer (was: Add an SQL node as a feature

[jira] [Commented] (SPARK-8410) Hive VersionsSuite RuntimeException

[ https://issues.apache.org/jira/browse/SPARK-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591030#comment-14591030 ] Michael Armbrust commented on SPARK-8410: - It looks like either a transient error,

[jira] [Updated] (SPARK-8420) Inconsistent behavior with Dataframe Timestamp between 1.3.1 and 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8420: Description: I am trying out 1.4.0 and notice there are some differences in behavior

[jira] [Updated] (SPARK-8420) Inconsistent behavior with Dataframe Timestamp between 1.3.1 and 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8420: Target Version/s: 1.4.1 (was: 1.4.0) Inconsistent behavior with Dataframe Timestamp

[jira] [Assigned] (SPARK-8422) Introduce a module abstraction inside of dev/run-tests

[ https://issues.apache.org/jira/browse/SPARK-8422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8422: --- Assignee: Apache Spark (was: Josh Rosen) Introduce a module abstraction inside of

[jira] [Commented] (SPARK-7889) Jobs progress of apps on complete page of HistoryServer shows uncompleted

[ https://issues.apache.org/jira/browse/SPARK-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591141#comment-14591141 ] meiyoula commented on SPARK-7889: - [~ste...@apache.org] Can you realize your proposal with

[jira] [Commented] (SPARK-8424) Add blacklist mechanism for task scheduler and Yarn container allocation

[ https://issues.apache.org/jira/browse/SPARK-8424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591173#comment-14591173 ] Saisai Shao commented on SPARK-8424: Here is the design doc in Google doc, any comment

[jira] [Commented] (SPARK-8389) Expose KafkaRDDs offsetRange in Java and Python

[ https://issues.apache.org/jira/browse/SPARK-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591211#comment-14591211 ] Saisai Shao commented on SPARK-8389: I'm not saying its impossible to do it, from my

[jira] [Created] (SPARK-8428) TimSort Comparison method violates its general contract with CLUSTER BY

Nathan McCarthy created SPARK-8428: -- Summary: TimSort Comparison method violates its general contract with CLUSTER BY Key: SPARK-8428 URL: https://issues.apache.org/jira/browse/SPARK-8428 Project:

[jira] [Created] (SPARK-8427) Incorrect ACL checking for partitioned table in Spark SQL-1.4

Karthik Subramanian created SPARK-8427: -- Summary: Incorrect ACL checking for partitioned table in Spark SQL-1.4 Key: SPARK-8427 URL: https://issues.apache.org/jira/browse/SPARK-8427 Project:

[jira] [Comment Edited] (SPARK-6813) SparkR style guide

[ https://issues.apache.org/jira/browse/SPARK-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590862#comment-14590862 ] Yu Ishikawa edited comment on SPARK-6813 at 6/18/15 12:41 AM: --

[jira] [Updated] (SPARK-8278) Remove deprecated JsonRDD functionality in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8278: Priority: Critical (was: Minor) Remove deprecated JsonRDD functionality in Spark SQL

[jira] [Updated] (SPARK-8278) Remove deprecated JsonRDD functionality in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8278: Target Version/s: 1.6.0 Remove deprecated JsonRDD functionality in Spark SQL

[jira] [Updated] (SPARK-8301) Improve UTF8String substring/startsWith/endsWith/contains performance

[ https://issues.apache.org/jira/browse/SPARK-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8301: Assignee: Michael Armbrust Improve UTF8String substring/startsWith/endsWith/contains

[jira] [Created] (SPARK-8421) Spark SQL DATE_ADD function - Spark 1.3.1 1.4.0

Nathan McCarthy created SPARK-8421: -- Summary: Spark SQL DATE_ADD function - Spark 1.3.1 1.4.0 Key: SPARK-8421 URL: https://issues.apache.org/jira/browse/SPARK-8421 Project: Spark Issue

[jira] [Commented] (SPARK-8390) Update DirectKafkaWordCount examples to show how offset ranges can be used

[ https://issues.apache.org/jira/browse/SPARK-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591131#comment-14591131 ] Tathagata Das commented on SPARK-8390: -- I think its best to incorporate some of the

[jira] [Commented] (SPARK-8373) When an RDD has no partition, Python sum will throw Can not reduce() empty RDD

[ https://issues.apache.org/jira/browse/SPARK-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591138#comment-14591138 ] Apache Spark commented on SPARK-8373: - User 'zsxwing' has created a pull request for

[jira] [Updated] (SPARK-8368) ClassNotFoundException in closure for map

[ https://issues.apache.org/jira/browse/SPARK-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8368: Component/s: (was: Spark Core) SQL ClassNotFoundException in closure for map

[jira] [Commented] (SPARK-7943) saveAsTable in DataFrameWriter can only add table to DataBase “default”

[ https://issues.apache.org/jira/browse/SPARK-7943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591232#comment-14591232 ] Apache Spark commented on SPARK-7943: - User 'baishuo' has created a pull request for

[jira] [Updated] (SPARK-8427) Incorrect ACL checking for partitioned table in Spark SQL-1.4

[ https://issues.apache.org/jira/browse/SPARK-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Subramanian updated SPARK-8427: --- Environment: CentOS 6 OS X 10.9.5, Hive-0.13.1, Spark-1.4, Hadoop 2.6.0 (was:

<    1   2   3   4   >