[jira] [Commented] (SPARK-9865) Flaky SparkR test: test_sparkSQL.R: sample on a DataFrame

2015-10-22 Thread Weiqiang Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969536#comment-14969536 ] Weiqiang Zhuang commented on SPARK-9865: Just FYI, I also got this test failure a couple of time.

[jira] [Updated] (SPARK-10562) .partitionBy() creates the metastore partition columns in all lowercase, but persists the data path as MixedCase resulting in an error when the data is later attempted t

2015-10-22 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10562: - Assignee: Wenchen Fan > .partitionBy() creates the metastore partition columns in all lowercase, but >

[jira] [Resolved] (SPARK-11232) NettyRpcEndpointRef.send should not be interrupted

2015-10-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11232. Resolution: Fixed Assignee: Shixiong Zhu > NettyRpcEndpointRef.send should not be

[jira] [Updated] (SPARK-11264) ./bin/spark-class can't find assembly jars with certain GREP_OPTIONS set

2015-10-22 Thread Jeffrey Naisbitt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Naisbitt updated SPARK-11264: - Component/s: Spark Shell > ./bin/spark-class can't find assembly jars with certain

[jira] [Created] (SPARK-11260) Add 'with' API

2015-10-22 Thread Weiqiang Zhuang (JIRA)
Weiqiang Zhuang created SPARK-11260: --- Summary: Add 'with' API Key: SPARK-11260 URL: https://issues.apache.org/jira/browse/SPARK-11260 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-11245) Upgrade twitter4j to version 4.x

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11245: Assignee: (was: Apache Spark) > Upgrade twitter4j to version 4.x >

[jira] [Assigned] (SPARK-11245) Upgrade twitter4j to version 4.x

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11245: Assignee: Apache Spark > Upgrade twitter4j to version 4.x >

[jira] [Commented] (SPARK-11260) Add 'with' API

2015-10-22 Thread Weiqiang Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969528#comment-14969528 ] Weiqiang Zhuang commented on SPARK-11260: - I will be working on this. > Add 'with' API >

[jira] [Created] (SPARK-11261) Provide a more flexible alternative to Jdbc RDD

2015-10-22 Thread Richard Marscher (JIRA)
Richard Marscher created SPARK-11261: Summary: Provide a more flexible alternative to Jdbc RDD Key: SPARK-11261 URL: https://issues.apache.org/jira/browse/SPARK-11261 Project: Spark

[jira] [Resolved] (SPARK-11163) Remove unnecessary addPendingTask calls in TaskSetManager.executorLost

2015-10-22 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-11163. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9154

[jira] [Commented] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-10-22 Thread kevin yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968711#comment-14968711 ] kevin yu commented on SPARK-5966: - Hello Tathagata & Andrew: I have code a possible fix, and the error

[jira] [Commented] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968739#comment-14968739 ] Apache Spark commented on SPARK-5966: - User 'kevinyu98' has created a pull request for this issue:

[jira] [Created] (SPARK-11258) Remove quadratic runtime complexity for converting a Spark DataFrame into an R data.frame

2015-10-22 Thread Frank Rosner (JIRA)
Frank Rosner created SPARK-11258: Summary: Remove quadratic runtime complexity for converting a Spark DataFrame into an R data.frame Key: SPARK-11258 URL: https://issues.apache.org/jira/browse/SPARK-11258

[jira] [Commented] (SPARK-9523) Receiver for Spark Streaming does not naturally support kryo serializer

2015-10-22 Thread Yuhang Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968628#comment-14968628 ] Yuhang Chen commented on SPARK-9523: So you mean closures also support kryo? But I never add any kryo

[jira] [Created] (SPARK-11255) R Test build should run on R 3.1.1

2015-10-22 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11255: Summary: R Test build should run on R 3.1.1 Key: SPARK-11255 URL: https://issues.apache.org/jira/browse/SPARK-11255 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11245) Upgrade twitter4j to version 4.x

2015-10-22 Thread pronix (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968744#comment-14968744 ] pronix commented on SPARK-11245: https://github.com/apache/spark/pull/9221 > Upgrade twitter4j to

[jira] [Commented] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968608#comment-14968608 ] Sun Rui commented on SPARK-11231: - [~shivaram] This feature can be done on R side even though it is

[jira] [Commented] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968605#comment-14968605 ] Sun Rui commented on SPARK-11231: - [~mpollock], we are implement R-like merge in sparkR

[jira] [Commented] (SPARK-11181) Spark Yarn : Spark reducing total executors count even when Dynamic Allocation is disabled.

2015-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968648#comment-14968648 ] Saisai Shao commented on SPARK-11181: - OK, get it. > Spark Yarn : Spark reducing total executors

[jira] [Reopened] (SPARK-11214) Join with Unicode-String results wrong empty

2015-10-22 Thread Hans Fischer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hans Fischer reopened SPARK-11214: -- I have updated to Spark 1.5.1 and still get an empty result set. How could this be? > Join with

[jira] [Created] (SPARK-11257) Spark dataframe negate filter conditions

2015-10-22 Thread Lokesh Kumar (JIRA)
Lokesh Kumar created SPARK-11257: Summary: Spark dataframe negate filter conditions Key: SPARK-11257 URL: https://issues.apache.org/jira/browse/SPARK-11257 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11258) Remove quadratic runtime complexity for converting a Spark DataFrame into an R data.frame

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968814#comment-14968814 ] Apache Spark commented on SPARK-11258: -- User 'FRosner' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11258) Remove quadratic runtime complexity for converting a Spark DataFrame into an R data.frame

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11258: Assignee: (was: Apache Spark) > Remove quadratic runtime complexity for converting a

[jira] [Assigned] (SPARK-11258) Remove quadratic runtime complexity for converting a Spark DataFrame into an R data.frame

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11258: Assignee: Apache Spark > Remove quadratic runtime complexity for converting a Spark

[jira] [Commented] (SPARK-11181) Spark Yarn : Spark reducing total executors count even when Dynamic Allocation is disabled.

2015-10-22 Thread prakhar jauhari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968655#comment-14968655 ] prakhar jauhari commented on SPARK-11181: - Ya. i'll do it for branch 1.3 :) > Spark Yarn : Spark

[jira] [Assigned] (SPARK-9319) Add support for setting column names, types

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9319: --- Assignee: Apache Spark > Add support for setting column names, types >

[jira] [Assigned] (SPARK-9319) Add support for setting column names, types

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9319: --- Assignee: (was: Apache Spark) > Add support for setting column names, types >

[jira] [Commented] (SPARK-9319) Add support for setting column names, types

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968657#comment-14968657 ] Apache Spark commented on SPARK-9319: - User 'felixcheung' has created a pull request for this issue:

[jira] [Commented] (SPARK-9595) Adding API to SparkConf for kryo serializers registration

2015-10-22 Thread Yuhang Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968615#comment-14968615 ] Yuhang Chen commented on SPARK-9595: Sorry, I replied to the wrong person, the question is meant for

[jira] [Created] (SPARK-11254) Thriftserver on kerberos secured cluster in YARN mode

2015-10-22 Thread wpxidian (JIRA)
wpxidian created SPARK-11254: Summary: Thriftserver on kerberos secured cluster in YARN mode Key: SPARK-11254 URL: https://issues.apache.org/jira/browse/SPARK-11254 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

2015-10-22 Thread Romi Kuntsman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968662#comment-14968662 ] Romi Kuntsman commented on SPARK-11229: --- [~marmbrus] it's reproducible in 1.5.1 as [~xwu0226]

[jira] [Closed] (SPARK-11190) SparkR support for cassandra collection types.

2015-10-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-11190. --- Resolution: Won't Fix Close it as the feature is already in the master branch, and will be available in

[jira] [Commented] (SPARK-11181) Spark Yarn : Spark reducing total executors count even when Dynamic Allocation is disabled.

2015-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968649#comment-14968649 ] Saisai Shao commented on SPARK-11181: - I think you'd better backport to branch 1.3, not for tag 1.3.1

[jira] [Assigned] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5966: --- Assignee: Apache Spark (was: Andrew Or) > Spark-submit deploy-mode incorrectly affecting

[jira] [Assigned] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5966: --- Assignee: Andrew Or (was: Apache Spark) > Spark-submit deploy-mode incorrectly affecting

[jira] [Updated] (SPARK-11258) Remove quadratic runtime complexity for converting a Spark DataFrame into an R data.frame

2015-10-22 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Rosner updated SPARK-11258: - Description: h4. Introduction We tried to collect a DataFrame with > 1 million rows and a few

[jira] [Commented] (SPARK-11195) Exception thrown on executor throws ClassNotFound on driver

2015-10-22 Thread Hurshal Patel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969535#comment-14969535 ] Hurshal Patel commented on SPARK-11195: --- the only difference between my repro and yours is that I

[jira] [Created] (SPARK-11262) Unit test for gradient, loss layers, memory management for multilayer perceptron

2015-10-22 Thread Alexander Ulanov (JIRA)
Alexander Ulanov created SPARK-11262: Summary: Unit test for gradient, loss layers, memory management for multilayer perceptron Key: SPARK-11262 URL: https://issues.apache.org/jira/browse/SPARK-11262

[jira] [Commented] (SPARK-11245) Upgrade twitter4j to version 4.x

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969527#comment-14969527 ] Apache Spark commented on SPARK-11245: -- User 'pronix' has created a pull request for this issue:

[jira] [Created] (SPARK-11263) lintr Throws Warnings on Commented Code in Documentation

2015-10-22 Thread Sen Fang (JIRA)
Sen Fang created SPARK-11263: Summary: lintr Throws Warnings on Commented Code in Documentation Key: SPARK-11263 URL: https://issues.apache.org/jira/browse/SPARK-11263 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-11078) Ensure spilling tests are actually spilling

2015-10-22 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-11078. Resolution: Fixed Fix Version/s: 1.6.0 > Ensure spilling tests are actually spilling >

[jira] [Updated] (SPARK-11264) ./bin/spark-class can't find assembly jars with certain GREP_OPTIONS set

2015-10-22 Thread Jeffrey Naisbitt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Naisbitt updated SPARK-11264: - Description: Some GREP_OPTIONS will modify the output of the grep commands that are

[jira] [Comment Edited] (SPARK-11195) Exception thrown on executor throws ClassNotFound on driver

2015-10-22 Thread Hurshal Patel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969535#comment-14969535 ] Hurshal Patel edited comment on SPARK-11195 at 10/22/15 5:47 PM: - this is

[jira] [Commented] (SPARK-11264) ./bin/spark-class can't find assembly jars with certain GREP_OPTIONS set

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969745#comment-14969745 ] Apache Spark commented on SPARK-11264: -- User 'naisbitt' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11264) ./bin/spark-class can't find assembly jars with certain GREP_OPTIONS set

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11264: Assignee: (was: Apache Spark) > ./bin/spark-class can't find assembly jars with

[jira] [Created] (SPARK-11264) ./bin/spark-class can't find assembly jars with certain GREP_OPTIONS set

2015-10-22 Thread Jeffrey Naisbitt (JIRA)
Jeffrey Naisbitt created SPARK-11264: Summary: ./bin/spark-class can't find assembly jars with certain GREP_OPTIONS set Key: SPARK-11264 URL: https://issues.apache.org/jira/browse/SPARK-11264

[jira] [Assigned] (SPARK-11264) ./bin/spark-class can't find assembly jars with certain GREP_OPTIONS set

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11264: Assignee: Apache Spark > ./bin/spark-class can't find assembly jars with certain

[jira] [Commented] (SPARK-11261) Provide a more flexible alternative to Jdbc RDD

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969595#comment-14969595 ] Apache Spark commented on SPARK-11261: -- User 'rmarsch' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11261) Provide a more flexible alternative to Jdbc RDD

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11261: Assignee: Apache Spark > Provide a more flexible alternative to Jdbc RDD >

[jira] [Assigned] (SPARK-11261) Provide a more flexible alternative to Jdbc RDD

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11261: Assignee: (was: Apache Spark) > Provide a more flexible alternative to Jdbc RDD >

[jira] [Commented] (SPARK-11262) Unit test for gradient, loss layers, memory management for multilayer perceptron

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969637#comment-14969637 ] Apache Spark commented on SPARK-11262: -- User 'avulanov' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11262) Unit test for gradient, loss layers, memory management for multilayer perceptron

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11262: Assignee: Apache Spark > Unit test for gradient, loss layers, memory management for

[jira] [Assigned] (SPARK-11262) Unit test for gradient, loss layers, memory management for multilayer perceptron

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11262: Assignee: (was: Apache Spark) > Unit test for gradient, loss layers, memory

[jira] [Created] (SPARK-11265) YarnClient cant get tokens to talk to Hive in a secure cluster

2015-10-22 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-11265: -- Summary: YarnClient cant get tokens to talk to Hive in a secure cluster Key: SPARK-11265 URL: https://issues.apache.org/jira/browse/SPARK-11265 Project: Spark

[jira] [Resolved] (SPARK-11200) NettyRpcEnv endless message "cannot send ${message} because RpcEnv is closed"

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11200. --- Resolution: Cannot Reproduce > NettyRpcEnv endless message "cannot send ${message} because RpcEnv is

[jira] [Updated] (SPARK-11242) In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented incorrectly

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11242: -- Priority: Trivial (was: Minor) > In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented

[jira] [Resolved] (SPARK-11162) Allow enabling debug logging from the command line

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11162. --- Resolution: Not A Problem > Allow enabling debug logging from the command line >

[jira] [Resolved] (SPARK-11181) Spark Yarn : Spark reducing total executors count even when Dynamic Allocation is disabled.

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11181. --- Resolution: Duplicate I pretty strongly doubt there will be another 1.3.x release [~prakhar088] so I

[jira] [Closed] (SPARK-11257) Spark dataframe negate filter conditions

2015-10-22 Thread Lokesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Kumar closed SPARK-11257. Resolution: Fixed Came to know that '!' unary op can be replaced with NOT and its working fine.

[jira] [Updated] (SPARK-11179) Push filters through aggregate if filters are subset of 'group by' expressions

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11179: -- Assignee: Nitin Goyal > Push filters through aggregate if filters are subset of 'group by' expressions

[jira] [Updated] (SPARK-10463) remove PromotePrecision during optimization

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10463: -- Assignee: Adrian Wang > remove PromotePrecision during optimization >

[jira] [Commented] (SPARK-11102) Uninformative exception when specifing non-exist input for JSON data source

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969019#comment-14969019 ] Apache Spark commented on SPARK-11102: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Resolved] (SPARK-11227) Spark1.5+ HDFS HA mode throw java.net.UnknownHostException: nameservice1

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11227. --- Resolution: Not A Problem > Spark1.5+ HDFS HA mode throw java.net.UnknownHostException: nameservice1

[jira] [Closed] (SPARK-11213) Documentation for remote spark Submit for R Scripts from 1.5 on CDH 5.4

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-11213. - > Documentation for remote spark Submit for R Scripts from 1.5 on CDH 5.4 >

[jira] [Updated] (SPARK-11248) Spark hivethriftserver is using the wrong user to while getting HDFS permissions

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11248: -- Component/s: SQL > Spark hivethriftserver is using the wrong user to while getting HDFS > permissions

[jira] [Updated] (SPARK-11245) Upgrade twitter4j to version 4.x

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11245: -- Fix Version/s: (was: 1.6.0) [~luciano resende] don't set Fix version or Target version. > Upgrade

[jira] [Updated] (SPARK-11252) ExternalShuffleClient should release connection after it had completed to fetch blocks from yarn's NameManager

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11252: -- Component/s: YARN Shuffle > ExternalShuffleClient should release connection after it

[jira] [Closed] (SPARK-11257) Spark dataframe negate filter conditions

2015-10-22 Thread Lokesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Kumar closed SPARK-11257. Resolution: Not A Problem There was no fix. See earlier comments. > Spark dataframe negate filter

[jira] [Commented] (SPARK-11227) Spark1.5+ HDFS HA mode throw java.net.UnknownHostException: nameservice1

2015-10-22 Thread Yuri Saito (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969002#comment-14969002 ] Yuri Saito commented on SPARK-11227: [~ste...@apache.org] But, same environments, spark1.4.0 run with

[jira] [Commented] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2015-10-22 Thread HAN YE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969011#comment-14969011 ] HAN YE commented on SPARK-5594: --- I don't know what your codes are.If your codes run one StreamingContext

[jira] [Resolved] (SPARK-11121) Incorrect TaskLocation type

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11121. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9096

[jira] [Updated] (SPARK-11121) Incorrect TaskLocation type

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11121: -- Assignee: zhichao-li > Incorrect TaskLocation type > --- > >

[jira] [Updated] (SPARK-11208) Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11208: -- Assignee: Artem Aliev > Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config

[jira] [Reopened] (SPARK-11257) Spark dataframe negate filter conditions

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-11257: --- [~lokeshdotp] don't resolve these as "Fixed" since there was not a problem that was fixed by a

[jira] [Updated] (SPARK-11216) add encoder/decoder for external row

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11216: -- Assignee: Wenchen Fan > add encoder/decoder for external row > >

[jira] [Updated] (SPARK-11257) Spark dataframe negate filter conditions

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11257: -- Target Version/s: (was: 1.5.1) Fix Version/s: (was: 1.5.0) [~lokeshdotp] also do not set

[jira] [Resolved] (SPARK-11257) Spark dataframe negate filter conditions

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11257. --- Resolution: Not A Problem > Spark dataframe negate filter conditions >

[jira] [Commented] (SPARK-11257) Spark dataframe negate filter conditions

2015-10-22 Thread Lokesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968976#comment-14968976 ] Lokesh Kumar commented on SPARK-11257: -- Sorry using jira for the first time. > Spark dataframe

[jira] [Reopened] (SPARK-11257) Spark dataframe negate filter conditions

2015-10-22 Thread Lokesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Kumar reopened SPARK-11257: -- > Spark dataframe negate filter conditions > > >

[jira] [Updated] (SPARK-11247) Back to master always try to use fqdn node name

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11247: -- Fix Version/s: (was: 1.6.0) > Back to master always try to use fqdn node name >

[jira] [Resolved] (SPARK-11247) Back to master always try to use fqdn node name

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11247. --- Resolution: Not A Problem > Back to master always try to use fqdn node name >

[jira] [Resolved] (SPARK-8048) Explicit partitionning of an RDD with 0 partition will yield empty outer join

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8048. -- Resolution: Duplicate > Explicit partitionning of an RDD with 0 partition will yield empty outer join >

[jira] [Updated] (SPARK-11250) Generate different alias for columns with same name during join

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11250: -- Component/s: SQL > Generate different alias for columns with same name during join >

[jira] [Resolved] (SPARK-11213) Documentation for remote spark Submit for R Scripts from 1.5 on CDH 5.4

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11213. --- Resolution: Invalid [~ankit30988] do *not* reopen an issue unless something has materially changed

[jira] [Comment Edited] (SPARK-11213) Documentation for remote spark Submit for R Scripts from 1.5 on CDH 5.4

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968969#comment-14968969 ] Sean Owen edited comment on SPARK-11213 at 10/22/15 11:15 AM: -- [~AnkitKhare]

[jira] [Resolved] (SPARK-9735) Auto infer partition schema of HadoopFsRelation should should respected the user specified one

2015-10-22 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-9735. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8026

[jira] [Reopened] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

2015-10-22 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reopened SPARK-11229: -- > NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0 >

[jira] [Closed] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

2015-10-22 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust closed SPARK-11229. Resolution: Fixed Fix Version/s: 1.6.0 > NPE in JoinedRow.isNullAt when

[jira] [Resolved] (SPARK-11116) Initial API Draft

2015-10-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6. - Resolution: Fixed Fix Version/s: 1.6.0 > Initial API Draft > - > >

[jira] [Created] (SPARK-11269) Java API support & test cases

2015-10-22 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-11269: --- Summary: Java API support & test cases Key: SPARK-11269 URL: https://issues.apache.org/jira/browse/SPARK-11269 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-11267) NettyRpcEnv and sparkDriver services report the same port in the logs

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969923#comment-14969923 ] Sean Owen commented on SPARK-11267: --- Huh that does look weird. [~zsxwing] does this somehow have to do

[jira] [Updated] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11265: --- Summary: YarnClient can't get tokens to talk to Hive in a secure cluster (was: YarnClient

[jira] [Created] (SPARK-11268) Non-daemon startup scripts

2015-10-22 Thread Simon Hafner (JIRA)
Simon Hafner created SPARK-11268: Summary: Non-daemon startup scripts Key: SPARK-11268 URL: https://issues.apache.org/jira/browse/SPARK-11268 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-7021) JUnit output for Python tests

2015-10-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-7021. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8323

[jira] [Updated] (SPARK-10812) Spark Hadoop Util does not support stopping a non-yarn Spark Context & starting a Yarn spark context.

2015-10-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-10812: --- Fix Version/s: 1.5.2 Backported to branch-1.5 (clean merge) to fix SPARK-11201. > Spark

[jira] [Assigned] (SPARK-10024) Python API RF and GBT related params clear up

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10024: Assignee: Apache Spark > Python API RF and GBT related params clear up >

[jira] [Commented] (SPARK-10024) Python API RF and GBT related params clear up

2015-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969945#comment-14969945 ] Apache Spark commented on SPARK-10024: -- User 'vectorijk' has created a pull request for this issue:

[jira] [Commented] (SPARK-11265) YarnClient cant get tokens to talk to Hive in a secure cluster

2015-10-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969818#comment-14969818 ] Steve Loughran commented on SPARK-11265: Initial report from Chester Chen {noformat} This is

[jira] [Created] (SPARK-11266) Peak memory tests swallow failures

2015-10-22 Thread Andrew Or (JIRA)
Andrew Or created SPARK-11266: - Summary: Peak memory tests swallow failures Key: SPARK-11266 URL: https://issues.apache.org/jira/browse/SPARK-11266 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-11242) In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented incorrectly

2015-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11242. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9201

  1   2   >