[jira] [Updated] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-21 Thread Matt Pollock (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Pollock updated SPARK-11231: - Description: In the case where the key column of two data frames are named the same thing, join

[jira] [Updated] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-21 Thread Matt Pollock (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Pollock updated SPARK-11231: - Description: In the case where the key column of two data frames are named the same thing, join

[jira] [Created] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-21 Thread Matt Pollock (JIRA)
Matt Pollock created SPARK-11231: Summary: join returns schema with duplicated and ambiguous join columns Key: SPARK-11231 URL: https://issues.apache.org/jira/browse/SPARK-11231 Project: Spark

[jira] [Assigned] (SPARK-11253) reset all accumulators in physical operators before execute an action

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11253: Assignee: (was: Apache Spark) > reset all accumulators in physical operators before

[jira] [Commented] (SPARK-11190) SparkR support for cassandra collection types.

2015-10-21 Thread Bilind Hajer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968596#comment-14968596 ] Bilind Hajer commented on SPARK-11190: -- > df.recipes <- read.df(sqlContext, source = >

[jira] [Commented] (SPARK-11253) reset all accumulators in physical operators before execute an action

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968484#comment-14968484 ] Apache Spark commented on SPARK-11253: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11253) reset all accumulators in physical operators before execute an action

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11253: Assignee: Apache Spark > reset all accumulators in physical operators before execute an

[jira] [Assigned] (SPARK-8658) AttributeReference equals method only compare name, exprId and dataType

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8658: --- Assignee: Apache Spark > AttributeReference equals method only compare name, exprId and

[jira] [Commented] (SPARK-8658) AttributeReference equals method only compare name, exprId and dataType

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968553#comment-14968553 ] Apache Spark commented on SPARK-8658: - User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-8658) AttributeReference equals method only compare name, exprId and dataType

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8658: --- Assignee: (was: Apache Spark) > AttributeReference equals method only compare name,

[jira] [Commented] (SPARK-11243) Output UnsafeRow in InMemoryTableScan

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968576#comment-14968576 ] Apache Spark commented on SPARK-11243: -- User 'davies' has created a pull request for this issue:

[jira] [Commented] (SPARK-7428) DataFrame.join() could create a new df with duplicate column name

2015-10-21 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968575#comment-14968575 ] Xiao Li commented on SPARK-7428: IMO, it is impossible to fix the problem if we use dataFrames. Spark SQL

[jira] [Commented] (SPARK-11247) Back to master always try to use fqdn node name

2015-10-21 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968517#comment-14968517 ] Carson Wang commented on SPARK-11247: - You can set SPARK_MASTER_IP to your master's IP address in

[jira] [Commented] (SPARK-10754) table and column name are case sensitive when json Dataframe was registered as tempTable using JavaSparkContext.

2015-10-21 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968570#comment-14968570 ] Huaxin Gao commented on SPARK-10754: Hi Scott, I tried a temp table with a struct column but can't

[jira] [Created] (SPARK-11232) NettyRpcEndpointRef.send should not be interrupted

2015-10-21 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-11232: Summary: NettyRpcEndpointRef.send should not be interrupted Key: SPARK-11232 URL: https://issues.apache.org/jira/browse/SPARK-11232 Project: Spark Issue

[jira] [Commented] (SPARK-11232) NettyRpcEndpointRef.send should not be interrupted

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967432#comment-14967432 ] Apache Spark commented on SPARK-11232: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11232) NettyRpcEndpointRef.send should not be interrupted

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11232: Assignee: (was: Apache Spark) > NettyRpcEndpointRef.send should not be interrupted >

[jira] [Assigned] (SPARK-11232) NettyRpcEndpointRef.send should not be interrupted

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11232: Assignee: Apache Spark > NettyRpcEndpointRef.send should not be interrupted >

[jira] [Commented] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-21 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967364#comment-14967364 ] Shivaram Venkataraman commented on SPARK-11231: --- [~Narine] [~sunrui] Is this covered by

[jira] [Commented] (SPARK-11087) spark.sql.orc.filterPushdown does not work, No ORC pushdown predicate

2015-10-21 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967456#comment-14967456 ] Zhan Zhang commented on SPARK-11087: [~patcharee] I tried again, used the step you provided, and

[jira] [Updated] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-21 Thread Matt Pollock (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Pollock updated SPARK-11231: - Description: In the case where the key column of two data frames are named the same thing, join

[jira] [Commented] (SPARK-10754) table and column name are case sensitive when json Dataframe was registered as tempTable using JavaSparkContext.

2015-10-21 Thread Scott Lyons (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967387#comment-14967387 ] Scott Lyons commented on SPARK-10754: - I continue to have this issue, and can confirm that case

[jira] [Updated] (SPARK-11232) NettyRpcEndpointRef.send should not be interrupted

2015-10-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-11232: - Description: The current NettyRpcEndpointRef.send can be interrupted because it uses

[jira] [Commented] (SPARK-11021) SparkSQL cli throws exception when using with Hive 0.12 metastore in spark-1.5.0 version

2015-10-21 Thread Xiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967577#comment-14967577 ] Xiu commented on SPARK-11021: - I put hive 0.12 jar paths inside spark-defaults.conf, and am able to create

[jira] [Created] (SPARK-11234) What's cooking classification

2015-10-21 Thread Xusen Yin (JIRA)
Xusen Yin created SPARK-11234: - Summary: What's cooking classification Key: SPARK-11234 URL: https://issues.apache.org/jira/browse/SPARK-11234 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-6488) Support addition/multiplication in PySpark's BlockMatrix

2015-10-21 Thread Mike Dusenberry (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940102#comment-14940102 ] Mike Dusenberry edited comment on SPARK-6488 at 10/21/15 6:16 PM: -- FYI: I

[jira] [Created] (SPARK-11236) Upgrade Tachyon dependency to 0.8.0

2015-10-21 Thread Calvin Jia (JIRA)
Calvin Jia created SPARK-11236: -- Summary: Upgrade Tachyon dependency to 0.8.0 Key: SPARK-11236 URL: https://issues.apache.org/jira/browse/SPARK-11236 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-11233) cosh is not registered in function registry

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11233: Assignee: Apache Spark > cosh is not registered in function registry >

[jira] [Commented] (SPARK-11233) cosh is not registered in function registry

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967530#comment-14967530 ] Apache Spark commented on SPARK-11233: -- User 'shagunsodhani' has created a pull request for this

[jira] [Commented] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967564#comment-14967564 ] Felix Cheung commented on SPARK-11231: -- This should be addressed in the new merge implementation in

[jira] [Commented] (SPARK-9595) Adding API to SparkConf for kryo serializers registration

2015-10-21 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967563#comment-14967563 ] holdenk commented on SPARK-9595: I'm not sure I get the question - do you have a bit of code which shows

[jira] [Comment Edited] (SPARK-10181) HiveContext is not used with keytab principal but with user principal/unix username

2015-10-21 Thread Yu Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967643#comment-14967643 ] Yu Gao edited comment on SPARK-10181 at 10/21/15 6:49 PM: -- I reproduced this

[jira] [Created] (SPARK-11240) PMML export for SVM models in ML pipeline

2015-10-21 Thread holdenk (JIRA)
holdenk created SPARK-11240: --- Summary: PMML export for SVM models in ML pipeline Key: SPARK-11240 URL: https://issues.apache.org/jira/browse/SPARK-11240 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-11239) PMML export for ML linear regression

2015-10-21 Thread holdenk (JIRA)
holdenk created SPARK-11239: --- Summary: PMML export for ML linear regression Key: SPARK-11239 URL: https://issues.apache.org/jira/browse/SPARK-11239 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-11087) spark.sql.orc.filterPushdown does not work, No ORC pushdown predicate

2015-10-21 Thread patcharee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967661#comment-14967661 ] patcharee commented on SPARK-11087: --- Hi [~zzhan] What version of hive and orc file you are using? Can

[jira] [Commented] (SPARK-11144) Add SparkLauncher for Spark Streaming, Spark SQL, etc

2015-10-21 Thread Yuhang Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967488#comment-14967488 ] Yuhang Chen commented on SPARK-11144: - Thanks for your advice and sorry for the inconvenience. > Add

[jira] [Assigned] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8835: --- Assignee: (was: Apache Spark) > Provide pluggable Congestion Strategies to deal with

[jira] [Commented] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967542#comment-14967542 ] Apache Spark commented on SPARK-8835: - User 'huitseeker' has created a pull request for this issue:

[jira] [Commented] (SPARK-11171) PMML for Pipelines API

2015-10-21 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967558#comment-14967558 ] holdenk commented on SPARK-11171: - Skimming the mailing list it seems that people mostly just want more

[jira] [Resolved] (SPARK-11216) add encoder/decoder for external row

2015-10-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-11216. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9184

[jira] [Commented] (SPARK-10181) HiveContext is not used with keytab principal but with user principal/unix username

2015-10-21 Thread Yu Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967643#comment-14967643 ] Yu Gao commented on SPARK-10181: I reproduced this issue. Some hive metastore client threads try to read

[jira] [Created] (SPARK-11238) SparkR: Documentation change for merge function

2015-10-21 Thread Narine Kokhlikyan (JIRA)
Narine Kokhlikyan created SPARK-11238: - Summary: SparkR: Documentation change for merge function Key: SPARK-11238 URL: https://issues.apache.org/jira/browse/SPARK-11238 Project: Spark

[jira] [Created] (SPARK-11237) PMML export for ML KMeans

2015-10-21 Thread holdenk (JIRA)
holdenk created SPARK-11237: --- Summary: PMML export for ML KMeans Key: SPARK-11237 URL: https://issues.apache.org/jira/browse/SPARK-11237 Project: Spark Issue Type: Sub-task Components:

[jira] [Commented] (SPARK-9595) Adding API to SparkConf for kryo serializers registration

2015-10-21 Thread Yuhang Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967503#comment-14967503 ] Yuhang Chen commented on SPARK-9595: So you mean closures also support kryo? But I never add any kryo

[jira] [Created] (SPARK-11233) cosh is not registered in function registry

2015-10-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-11233: --- Summary: cosh is not registered in function registry Key: SPARK-11233 URL: https://issues.apache.org/jira/browse/SPARK-11233 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load

2015-10-21 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] François Garillot updated SPARK-8835: - Description: Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398]

[jira] [Assigned] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8835: --- Assignee: Apache Spark > Provide pluggable Congestion Strategies to deal with Streaming load

[jira] [Resolved] (SPARK-10534) ORDER BY clause allows only columns that are present in SELECT statement

2015-10-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10534. -- Resolution: Fixed Fix Version/s: 1.5.2 1.6.0 Issue resolved by pull request

[jira] [Updated] (SPARK-10534) ORDER BY clause allows only columns that are present in SELECT statement

2015-10-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10534: - Assignee: Dilip Biswal > ORDER BY clause allows only columns that are present in SELECT statement >

[jira] [Commented] (SPARK-11171) PMML for Pipelines API

2015-10-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967608#comment-14967608 ] Joseph K. Bradley commented on SPARK-11171: --- Sounds great, thanks! > PMML for Pipelines API >

[jira] [Created] (SPARK-11241) Add a common trait for PMML exportable ML pipeline models

2015-10-21 Thread holdenk (JIRA)
holdenk created SPARK-11241: --- Summary: Add a common trait for PMML exportable ML pipeline models Key: SPARK-11241 URL: https://issues.apache.org/jira/browse/SPARK-11241 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-11037) Cleanup Option usage in JdbcUtils

2015-10-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-11037. - Resolution: Fixed Assignee: Pravin Gadakh Fix Version/s: 1.6.0 > Cleanup Option

[jira] [Issue Comment Deleted] (SPARK-11233) cosh is not registered in function registry

2015-10-21 Thread Shagun Sodhani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shagun Sodhani updated SPARK-11233: --- Comment: was deleted (was: Submitted a PR: https://github.com/apache/spark/pull/9199) >

[jira] [Commented] (SPARK-11233) cosh is not registered in function registry

2015-10-21 Thread Shagun Sodhani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967536#comment-14967536 ] Shagun Sodhani commented on SPARK-11233: Submitted a PR:

[jira] [Created] (SPARK-11235) Support streaming data using network library

2015-10-21 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-11235: -- Summary: Support streaming data using network library Key: SPARK-11235 URL: https://issues.apache.org/jira/browse/SPARK-11235 Project: Spark Issue Type:

[jira] [Created] (SPARK-11242) In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented incorrectly

2015-10-21 Thread Xiu (JIRA)
Xiu created SPARK-11242: --- Summary: In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented incorrectly Key: SPARK-11242 URL: https://issues.apache.org/jira/browse/SPARK-11242 Project: Spark

[jira] [Assigned] (SPARK-10857) SQL injection bug in JdbcDialect.getTableExistsQuery()

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10857: Assignee: Apache Spark > SQL injection bug in JdbcDialect.getTableExistsQuery() >

[jira] [Commented] (SPARK-10857) SQL injection bug in JdbcDialect.getTableExistsQuery()

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967690#comment-14967690 ] Apache Spark commented on SPARK-10857: -- User 'rick-ibm' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10857) SQL injection bug in JdbcDialect.getTableExistsQuery()

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10857: Assignee: (was: Apache Spark) > SQL injection bug in

[jira] [Assigned] (SPARK-11243) Output UnsafeRow in InMemoryTableScan

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11243: Assignee: Apache Spark > Output UnsafeRow in InMemoryTableScan >

[jira] [Commented] (SPARK-11243) Output UnsafeRow in InMemoryTableScan

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967697#comment-14967697 ] Apache Spark commented on SPARK-11243: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11243) Output UnsafeRow in InMemoryTableScan

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11243: Assignee: (was: Apache Spark) > Output UnsafeRow in InMemoryTableScan >

[jira] [Created] (SPARK-11244) sparkR.stop doesn't clean up .sparkRSQLsc in environment

2015-10-21 Thread Sen Fang (JIRA)
Sen Fang created SPARK-11244: Summary: sparkR.stop doesn't clean up .sparkRSQLsc in environment Key: SPARK-11244 URL: https://issues.apache.org/jira/browse/SPARK-11244 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11244) sparkR.stop doesn't clean up .sparkRSQLsc in environment

2015-10-21 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967778#comment-14967778 ] Shivaram Venkataraman commented on SPARK-11244: --- Good catch -- Could you send a PR for this

[jira] [Comment Edited] (SPARK-10857) SQL injection bug in JdbcDialect.getTableExistsQuery()

2015-10-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967827#comment-14967827 ] Reynold Xin edited comment on SPARK-10857 at 10/21/15 8:31 PM: --- We want to

[jira] [Commented] (SPARK-10857) SQL injection bug in JdbcDialect.getTableExistsQuery()

2015-10-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967827#comment-14967827 ] Reynold Xin commented on SPARK-10857: - We want to support a query within that. Would this rule that

[jira] [Updated] (SPARK-10743) keep the name of expression if possible when do cast

2015-10-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-10743: - Assignee: Wenchen Fan > keep the name of expression if possible when do cast >

[jira] [Assigned] (SPARK-11236) Upgrade Tachyon dependency to 0.8.0

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11236: Assignee: Apache Spark > Upgrade Tachyon dependency to 0.8.0 >

[jira] [Commented] (SPARK-11236) Upgrade Tachyon dependency to 0.8.0

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967728#comment-14967728 ] Apache Spark commented on SPARK-11236: -- User 'calvinjia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11236) Upgrade Tachyon dependency to 0.8.0

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11236: Assignee: (was: Apache Spark) > Upgrade Tachyon dependency to 0.8.0 >

[jira] [Commented] (SPARK-11087) spark.sql.orc.filterPushdown does not work, No ORC pushdown predicate

2015-10-21 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1496#comment-1496 ] Zhan Zhang commented on SPARK-11087: [~patcharee] I use the embeded hive metastore without any

[jira] [Commented] (SPARK-11238) SparkR: Documentation change for merge function

2015-10-21 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967807#comment-14967807 ] Shivaram Venkataraman commented on SPARK-11238: --- Also on this we should mark this as a

[jira] [Commented] (SPARK-11242) In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented incorrectly

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967673#comment-14967673 ] Apache Spark commented on SPARK-11242: -- User 'xguo27' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11242) In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented incorrectly

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11242: Assignee: Apache Spark > In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented

[jira] [Assigned] (SPARK-11242) In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is documented incorrectly

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11242: Assignee: (was: Apache Spark) > In conf/spark-env.sh.template SPARK_DRIVER_MEMORY is

[jira] [Assigned] (SPARK-11244) sparkR.stop doesn't clean up .sparkRSQLsc in environment

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11244: Assignee: (was: Apache Spark) > sparkR.stop doesn't clean up .sparkRSQLsc in

[jira] [Commented] (SPARK-11244) sparkR.stop doesn't clean up .sparkRSQLsc in environment

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967806#comment-14967806 ] Apache Spark commented on SPARK-11244: -- User 'saurfang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11244) sparkR.stop doesn't clean up .sparkRSQLsc in environment

2015-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11244: Assignee: Apache Spark > sparkR.stop doesn't clean up .sparkRSQLsc in environment >

[jira] [Commented] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

2015-10-21 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967694#comment-14967694 ] Xin Wu commented on SPARK-11229: Here is what I tried: 1. start spark-shell with command

[jira] [Commented] (SPARK-10977) SQL injection bugs in JdbcUtils and DataFrameWriter

2015-10-21 Thread Rick Hillegas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967695#comment-14967695 ] Rick Hillegas commented on SPARK-10977: --- I have created a pull request for SPARK-10857 which also

[jira] [Created] (SPARK-11243) Output UnsafeRow in InMemoryTableScan

2015-10-21 Thread Davies Liu (JIRA)
Davies Liu created SPARK-11243: -- Summary: Output UnsafeRow in InMemoryTableScan Key: SPARK-11243 URL: https://issues.apache.org/jira/browse/SPARK-11243 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-11238) SparkR: Documentation change for merge function

2015-10-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11238: Labels: releasenotes (was: ) > SparkR: Documentation change for merge function >

[jira] [Commented] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

2015-10-21 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967811#comment-14967811 ] Xin Wu commented on SPARK-11229: I am able to recreate it on 1.5.1 {code} scala> data02.show 15/10/21

[jira] [Resolved] (SPARK-10743) keep the name of expression if possible when do cast

2015-10-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-10743. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8859

[jira] [Resolved] (SPARK-11197) Run SQL query on files directly without create a table

2015-10-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-11197. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9173

[jira] [Resolved] (SPARK-9740) first/last aggregate NULL behavior

2015-10-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9740. - Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8113

[jira] [Created] (SPARK-11245) Upgrade twitter4j to version 4.x

2015-10-21 Thread Luciano Resende (JIRA)
Luciano Resende created SPARK-11245: --- Summary: Upgrade twitter4j to version 4.x Key: SPARK-11245 URL: https://issues.apache.org/jira/browse/SPARK-11245 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11171) PMML for Pipelines API

2015-10-21 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967956#comment-14967956 ] holdenk commented on SPARK-11171: - So from the implementation point of view, we could keep extending the

[jira] [Resolved] (SPARK-8654) Analysis exception when using "NULL IN (...)": invalid cast

2015-10-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8654. - Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9036

[jira] [Resolved] (SPARK-9210) checkValidAggregateExpression() throws exceptions with bad error messages

2015-10-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9210. - Resolution: Fixed Assignee: Yin Huai Fix Version/s: 1.6.0 >

[jira] [Resolved] (SPARK-11208) Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config

2015-10-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-11208. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9178

[jira] [Resolved] (SPARK-11233) cosh is not registered in function registry

2015-10-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-11233. - Resolution: Fixed Assignee: Shagun Sodhani Fix Version/s: 1.6.0

[jira] [Commented] (SPARK-11190) SparkR support for cassandra collection types.

2015-10-21 Thread Bilind Hajer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967979#comment-14967979 ] Bilind Hajer commented on SPARK-11190: -- Ok, got master spark version built, I am no longer getting

[jira] [Commented] (SPARK-8658) AttributeReference equals method only compare name, exprId and dataType

2015-10-21 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967898#comment-14967898 ] Xiao Li commented on SPARK-8658: Hi, Michael, Thank you! It sounds a trivial work. Let me try it. Send a

[jira] [Commented] (SPARK-11190) SparkR support for cassandra collection types.

2015-10-21 Thread Bilind Hajer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967903#comment-14967903 ] Bilind Hajer commented on SPARK-11190: -- I built spark using mvn, on the master version on github

[jira] [Created] (SPARK-11246) [1.5] Table cache for Parquet broken in 1.5

2015-10-21 Thread David Ross (JIRA)
David Ross created SPARK-11246: -- Summary: [1.5] Table cache for Parquet broken in 1.5 Key: SPARK-11246 URL: https://issues.apache.org/jira/browse/SPARK-11246 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11190) SparkR support for cassandra collection types.

2015-10-21 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967907#comment-14967907 ] Shivaram Venkataraman commented on SPARK-11190: --- You'll need to add the -Psparkr flag as

[jira] [Commented] (SPARK-10857) SQL injection bug in JdbcDialect.getTableExistsQuery()

2015-10-21 Thread Rick Hillegas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967930#comment-14967930 ] Rick Hillegas commented on SPARK-10857: --- Hi Reynold, Yes, this would fail if the table were a

[jira] [Created] (SPARK-11247) Back to master always try to use fqdn node name

2015-10-21 Thread Luciano Resende (JIRA)
Luciano Resende created SPARK-11247: --- Summary: Back to master always try to use fqdn node name Key: SPARK-11247 URL: https://issues.apache.org/jira/browse/SPARK-11247 Project: Spark Issue

[jira] [Commented] (SPARK-11190) SparkR support for cassandra collection types.

2015-10-21 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968085#comment-14968085 ] Shivaram Venkataraman commented on SPARK-11190: --- Environments can be used like a hash map

  1   2   3   >