[jira] [Updated] (SPARK-14954) Add PARTITIONED BY and CLUSTERED BY clause for data source CTAS syntax

2016-04-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14954: --- Summary: Add PARTITIONED BY and CLUSTERED BY clause for data source CTAS syntax (was: Add PARTITION

[jira] [Commented] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for data source CTAS syntax

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261547#comment-15261547 ] Apache Spark commented on SPARK-14954: -- User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-14346) SHOW CREATE TABLE command (Native)

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261544#comment-15261544 ] Apache Spark commented on SPARK-14346: -- User 'liancheng' has created a pull request for this issue:

[jira] [Resolved] (SPARK-14961) Support LongToUnsafeRowMap larger than 1G

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14961. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12740

[jira] [Commented] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261489#comment-15261489 ] Yanbo Liang commented on SPARK-14831: - +1 > Make ML APIs in SparkR consistent >

[jira] [Commented] (SPARK-14906) Move VectorUDT and MatrixUDT in PySpark to new ML package

2016-04-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261487#comment-15261487 ] Yanbo Liang commented on SPARK-14906: - OK, thanks for kindly remind. > Move VectorUDT and MatrixUDT

[jira] [Assigned] (SPARK-14972) Improve performance of JSON schema inference's inferField step

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14972: Assignee: Apache Spark (was: Josh Rosen) > Improve performance of JSON schema

[jira] [Assigned] (SPARK-14972) Improve performance of JSON schema inference's inferField step

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14972: Assignee: Josh Rosen (was: Apache Spark) > Improve performance of JSON schema

[jira] [Commented] (SPARK-14972) Improve performance of JSON schema inference's inferField step

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261485#comment-15261485 ] Apache Spark commented on SPARK-14972: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-14972) Improve performance of JSON schema inference's inferField step

2016-04-27 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-14972: -- Summary: Improve performance of JSON schema inference's inferField step Key: SPARK-14972 URL: https://issues.apache.org/jira/browse/SPARK-14972 Project: Spark

[jira] [Commented] (SPARK-14781) Support subquery in nested predicates

2016-04-27 Thread Frederick Reiss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261474#comment-15261474 ] Frederick Reiss commented on SPARK-14781: - Sure, I'd be happy to put something together to cover

[jira] [Commented] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-04-27 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261471#comment-15261471 ] Narine Kokhlikyan commented on SPARK-12922: --- Thank you for quick responses [~shivaram] and

[jira] [Assigned] (SPARK-14971) PySpark ML Params setter code clean up

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14971: Assignee: Apache Spark > PySpark ML Params setter code clean up >

[jira] [Assigned] (SPARK-14971) PySpark ML Params setter code clean up

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14971: Assignee: (was: Apache Spark) > PySpark ML Params setter code clean up >

[jira] [Commented] (SPARK-14971) PySpark ML Params setter code clean up

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261469#comment-15261469 ] Apache Spark commented on SPARK-14971: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Updated] (SPARK-14971) PySpark ML Params setter code clean up

2016-04-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-14971: Description: PySpark ML Params setter code clean up. For examples, {{setInputCol}} can be

[jira] [Updated] (SPARK-14971) PySpark ML Params setter code clean up

2016-04-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-14971: Description: PySpark ML Params setter code clean up. >From {code:none} self._set(inputCol=value)

[jira] [Created] (SPARK-14971) PySpark ML Params setter code clean up

2016-04-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-14971: --- Summary: PySpark ML Params setter code clean up Key: SPARK-14971 URL: https://issues.apache.org/jira/browse/SPARK-14971 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-14955) JDBCRelation should report an IllegalArgumentException if stride equals 0

2016-04-27 Thread Yang Juan hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261459#comment-15261459 ] Yang Juan hu commented on SPARK-14955: -- Got, thanks. > JDBCRelation should report an

[jira] [Commented] (SPARK-14955) JDBCRelation should report an IllegalArgumentException if stride equals 0

2016-04-27 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261457#comment-15261457 ] Bo Meng commented on SPARK-14955: - Only after committer gets PR merged, then this JIRA will be

[jira] [Commented] (SPARK-14955) JDBCRelation should report an IllegalArgumentException if stride equals 0

2016-04-27 Thread Yang Juan hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261435#comment-15261435 ] Yang Juan hu commented on SPARK-14955: -- Meng Bo, the pull looks fine for me. Can I close this issue

[jira] [Assigned] (SPARK-14970) DataSource enumerates all files in FileCatalog to infer schema even if there is user specified schema

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14970: Assignee: Apache Spark (was: Tathagata Das) > DataSource enumerates all files in

[jira] [Assigned] (SPARK-14970) DataSource enumerates all files in FileCatalog to infer schema even if there is user specified schema

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14970: Assignee: Tathagata Das (was: Apache Spark) > DataSource enumerates all files in

[jira] [Commented] (SPARK-14970) DataSource enumerates all files in FileCatalog to infer schema even if there is user specified schema

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261430#comment-15261430 ] Apache Spark commented on SPARK-14970: -- User 'tdas' has created a pull request for this issue:

[jira] [Created] (SPARK-14970) DataSource enumerates all files in FileCatalog to infer schema even if there is user specified schema

2016-04-27 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-14970: - Summary: DataSource enumerates all files in FileCatalog to infer schema even if there is user specified schema Key: SPARK-14970 URL:

[jira] [Commented] (SPARK-12143) When column type is binary, select occurs ClassCastExcption in Beeline.

2016-04-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261366#comment-15261366 ] Hyukjin Kwon commented on SPARK-12143: -- [~srowen] Can I close this? This was resolved by my PR

[jira] [Assigned] (SPARK-14969) Remove unnecessary compute function in LogisticGradient

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14969: Assignee: Apache Spark > Remove unnecessary compute function in LogisticGradient >

[jira] [Assigned] (SPARK-14969) Remove unnecessary compute function in LogisticGradient

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14969: Assignee: (was: Apache Spark) > Remove unnecessary compute function in

[jira] [Reopened] (SPARK-14969) Remove unnecessary compute function in LogisticGradient

2016-04-27 Thread ding (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ding reopened SPARK-14969: -- > Remove unnecessary compute function in LogisticGradient >

[jira] [Closed] (SPARK-14908) Provide support HDFS-located resources for "spark.executor.extraClasspath" on YARN

2016-04-27 Thread Dubkov Mikhail (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dubkov Mikhail closed SPARK-14908. -- Resolution: Won't Fix Find details on GitHub's pull request conversation. > Provide support

[jira] [Commented] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-04-27 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261339#comment-15261339 ] Sun Rui commented on SPARK-12922: - [~Narine] does AppendColumns logical operator

[jira] [Updated] (SPARK-14935) DistributedSuite "local-cluster format" shouldn't actually launch clusters

2016-04-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14935: --- Assignee: Xin Ren > DistributedSuite "local-cluster format" shouldn't actually launch clusters >

[jira] [Resolved] (SPARK-14969) Remove unnecessary compute function in LogisticGradient

2016-04-27 Thread ding (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ding resolved SPARK-14969. -- Resolution: Fixed > Remove unnecessary compute function in LogisticGradient >

[jira] [Assigned] (SPARK-14969) Remove unnecessary compute function in LogisticGradient

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14969: Assignee: (was: Apache Spark) > Remove unnecessary compute function in

[jira] [Assigned] (SPARK-14969) Remove unnecessary compute function in LogisticGradient

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14969: Assignee: Apache Spark > Remove unnecessary compute function in LogisticGradient >

[jira] [Commented] (SPARK-14969) Remove unnecessary compute function in LogisticGradient

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261325#comment-15261325 ] Apache Spark commented on SPARK-14969: -- User 'dding3' has created a pull request for this issue:

[jira] [Created] (SPARK-14969) Remove unnecessary compute function in LogisticGradient

2016-04-27 Thread ding (JIRA)
ding created SPARK-14969: Summary: Remove unnecessary compute function in LogisticGradient Key: SPARK-14969 URL: https://issues.apache.org/jira/browse/SPARK-14969 Project: Spark Issue Type:

[jira] [Updated] (SPARK-14934) Slow JsonHadoopFsRelationSuite test: "SPARK-8406: Avoids name collision while writing files"

2016-04-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14934: --- Fix Version/s: 2.0.0 > Slow JsonHadoopFsRelationSuite test: "SPARK-8406: Avoids name collision while

[jira] [Resolved] (SPARK-14934) Slow JsonHadoopFsRelationSuite test: "SPARK-8406: Avoids name collision while writing files"

2016-04-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-14934. Resolution: Fixed After SPARK-14966 this now takes about 6 seconds, so I'm going to declare

[jira] [Assigned] (SPARK-14934) Slow JsonHadoopFsRelationSuite test: "SPARK-8406: Avoids name collision while writing files"

2016-04-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14934: -- Assignee: Josh Rosen > Slow JsonHadoopFsRelationSuite test: "SPARK-8406: Avoids name

[jira] [Resolved] (SPARK-14966) SizeEstimator should ignore classes in the scala.reflect package

2016-04-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14966. - Resolution: Fixed Fix Version/s: 2.0.0 > SizeEstimator should ignore classes in the

[jira] [Updated] (SPARK-14938) Use Datasets.as to improve internal implementation

2016-04-27 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-14938: - Description: As discussed in [https://github.com/apache/spark/pull/11915], we can use

[jira] [Updated] (SPARK-14938) Use Datasets.as to improve internal implementation

2016-04-27 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-14938: - Summary: Use Datasets.as to improve internal implementation (was: Use Datasets to improve

[jira] [Commented] (SPARK-14915) Tasks that fail due to CommitDeniedException (a side-effect of speculation) can cause job to never complete

2016-04-27 Thread Jason Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261252#comment-15261252 ] Jason Moore commented on SPARK-14915: - That's exactly my current thinking too. But even if keep

[jira] [Commented] (SPARK-14591) Remove org.apache.spark.sql.catalyst.parser.DataTypeParser

2016-04-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261246#comment-15261246 ] Yin Huai commented on SPARK-14591: -- Thanks! So, keywords in the list of

[jira] [Commented] (SPARK-14781) Support subquery in nested predicates

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261230#comment-15261230 ] Davies Liu commented on SPARK-14781: [~freiss] SemiPlus is not introduced yet. Even the subquery in

[jira] [Updated] (SPARK-14785) Support correlated scalar subquery

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14785: --- Description: For example: {code} SELECT a from t where b > (select avg(c) from t2 where t.id =

[jira] [Commented] (SPARK-13023) Check for presence of 'root' module after computing test_modules, not changed_modules

2016-04-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261193#comment-15261193 ] Yin Huai commented on SPARK-13023: -- https://github.com/apache/spark/pull/12743 has been merged to branch

[jira] [Updated] (SPARK-13023) Check for presence of 'root' module after computing test_modules, not changed_modules

2016-04-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-13023: - Fix Version/s: 1.6.2 > Check for presence of 'root' module after computing test_modules, not >

[jira] [Updated] (SPARK-14968) TPC-DS query 1 resolved attribute(s) missing

2016-04-27 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-14968: --- Summary: TPC-DS query 1 resolved attribute(s) missing (was: TPC-DS query 1 fails to generate plan)

[jira] [Updated] (SPARK-14968) TPC-DS query 1 fails to generate plan

2016-04-27 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-14968: --- Description: This is a regression from a week ago. Failed to generate plan for query 1 in TPCDS

[jira] [Updated] (SPARK-14968) TPC-DS query 1 fails to generate plan

2016-04-27 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-14968: --- Affects Version/s: (was: 1.6.1) 2.0.0 > TPC-DS query 1 fails to generate

[jira] [Updated] (SPARK-14968) TPC-DS query 1 fails to generate plan

2016-04-27 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-14968: --- Priority: Critical (was: Major) > TPC-DS query 1 fails to generate plan >

[jira] [Updated] (SPARK-14968) TPC-DS query 1 fails to generate plan

2016-04-27 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-14968: --- Description: This is a regression from a week ago. Failed to generate plan for query 1 in TPCDS

[jira] [Created] (SPARK-14968) TPC-DS query 1 fails to generate plan

2016-04-27 Thread JESSE CHEN (JIRA)
JESSE CHEN created SPARK-14968: -- Summary: TPC-DS query 1 fails to generate plan Key: SPARK-14968 URL: https://issues.apache.org/jira/browse/SPARK-14968 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-14945) Python SparkSession API

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261184#comment-15261184 ] Apache Spark commented on SPARK-14945: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14945) Python SparkSession API

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14945: Assignee: Apache Spark (was: Andrew Or) > Python SparkSession API >

[jira] [Assigned] (SPARK-14945) Python SparkSession API

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14945: Assignee: Andrew Or (was: Apache Spark) > Python SparkSession API >

[jira] [Commented] (SPARK-14014) Replace existing analysis.Catalog with SessionCatalog

2016-04-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261177#comment-15261177 ] Andrew Or commented on SPARK-14014: --- Pretty sure this was fixed. :) > Replace existing

[jira] [Resolved] (SPARK-14014) Replace existing analysis.Catalog with SessionCatalog

2016-04-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14014. --- Resolution: Fixed Fix Version/s: 2.0.0 > Replace existing analysis.Catalog with

[jira] [Resolved] (SPARK-14671) Pipeline.setStages needs to handle Array non-covariance

2016-04-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14671. --- Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue

[jira] [Commented] (SPARK-14785) Support correlated scalar subquery

2016-04-27 Thread Frederick Reiss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261157#comment-15261157 ] Frederick Reiss commented on SPARK-14785: - Note that the rewritten query in the example above

[jira] [Commented] (SPARK-13323) Type cast support in type inference during merging types.

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261154#comment-15261154 ] Davies Liu commented on SPARK-13323: This API is not designed to use in this way, I'd like to not do

[jira] [Closed] (SPARK-13323) Type cast support in type inference during merging types.

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-13323. -- Resolution: Not A Problem > Type cast support in type inference during merging types. >

[jira] [Commented] (SPARK-7898) pyspark merges stderr into stdout

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261148#comment-15261148 ] Davies Liu commented on SPARK-7898: --- [~sds] So this is not a problem for PySpark, right? > pyspark

[jira] [Commented] (SPARK-14683) Configure external links in ScalaDoc

2016-04-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261128#comment-15261128 ] Josh Rosen commented on SPARK-14683: Re-opened because I reverted this patch. > Configure external

[jira] [Reopened] (SPARK-14683) Configure external links in ScalaDoc

2016-04-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reopened SPARK-14683: > Configure external links in ScalaDoc > > > Key:

[jira] [Updated] (SPARK-14683) Configure external links in ScalaDoc

2016-04-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14683: --- Fix Version/s: (was: 2.0.0) > Configure external links in ScalaDoc >

[jira] [Commented] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

2016-04-27 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261126#comment-15261126 ] Bo Meng commented on SPARK-14959: - I have tried on master branch, it works fine with the latest code. >

[jira] [Resolved] (SPARK-13436) Add parameter drop to subsetting operator [

2016-04-27 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-13436. --- Resolution: Fixed Assignee: Oscar D. Lara Yejas Fix

[jira] [Resolved] (SPARK-11757) Incorrect join output for joining two dataframes loaded from Parquet format

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11757. Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 2.0.0 > Incorrect join

[jira] [Commented] (SPARK-14965) StructType throws exception for missing field

2016-04-27 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261096#comment-15261096 ] Bo Meng commented on SPARK-14965: - I believe returning null does not make sense here, so exception is

[jira] [Commented] (SPARK-10001) Allow Ctrl-C in spark-shell to kill running job

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261093#comment-15261093 ] Apache Spark commented on SPARK-10001: -- User 'jodersky' has created a pull request for this issue:

[jira] [Closed] (SPARK-13837) SQL Context function to_date() returns wrong date

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-13837. -- Resolution: Cannot Reproduce Assignee: Davies Liu Fix Version/s: 2.0.0 > SQL Context

[jira] [Commented] (SPARK-13837) SQL Context function to_date() returns wrong date

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261084#comment-15261084 ] Davies Liu commented on SPARK-13837: @Amaud Caruso I'm in the same time zone as you , but can't

[jira] [Closed] (SPARK-9807) pyspark.sql.createDataFrame does not infer data type of parsed TSV

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-9807. - Resolution: Not A Problem Assignee: Davies Liu > pyspark.sql.createDataFrame does not infer data

[jira] [Commented] (SPARK-9807) pyspark.sql.createDataFrame does not infer data type of parsed TSV

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261062#comment-15261062 ] Davies Liu commented on SPARK-9807: --- The type inferring here is talking about get the type from Python

[jira] [Resolved] (SPARK-11368) Spark shouldn't scan all partitions when using Python UDF and filter over partitioned column is given

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11368. Resolution: Fixed Assignee: Davies Liu Fix Version/s: 2.0.0 This was fixed by

[jira] [Assigned] (SPARK-14967) EXCEPT does not follow SQL compliance

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14967: Assignee: (was: Apache Spark) > EXCEPT does not follow SQL compliance >

[jira] [Commented] (SPARK-14967) EXCEPT does not follow SQL compliance

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261035#comment-15261035 ] Apache Spark commented on SPARK-14967: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14967) EXCEPT does not follow SQL compliance

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14967: Assignee: Apache Spark > EXCEPT does not follow SQL compliance >

[jira] [Updated] (SPARK-14967) EXCEPT does not follow SQL compliance

2016-04-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-14967: Description: {noformat} test("except") { val df_left = Seq(1, 2, 2, 3, 3, 4).toDF("id") val

[jira] [Created] (SPARK-14967) EXCEPT does not follow SQL compliance

2016-04-27 Thread Xiao Li (JIRA)
Xiao Li created SPARK-14967: --- Summary: EXCEPT does not follow SQL compliance Key: SPARK-14967 URL: https://issues.apache.org/jira/browse/SPARK-14967 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-14781) Support subquery in nested predicates

2016-04-27 Thread Frederick Reiss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261032#comment-15261032 ] Frederick Reiss commented on SPARK-14781: - [~davies] where is the definition of the SemiPlus

[jira] [Commented] (SPARK-14781) Support subquery in nested predicates

2016-04-27 Thread Frederick Reiss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261028#comment-15261028 ] Frederick Reiss commented on SPARK-14781: - I'm not so sure about Q45. Here's the template for

[jira] [Assigned] (SPARK-14935) DistributedSuite "local-cluster format" shouldn't actually launch clusters

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14935: Assignee: (was: Apache Spark) > DistributedSuite "local-cluster format" shouldn't

[jira] [Assigned] (SPARK-14935) DistributedSuite "local-cluster format" shouldn't actually launch clusters

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14935: Assignee: Apache Spark > DistributedSuite "local-cluster format" shouldn't actually

[jira] [Commented] (SPARK-14935) DistributedSuite "local-cluster format" shouldn't actually launch clusters

2016-04-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261012#comment-15261012 ] Apache Spark commented on SPARK-14935: -- User 'keypointt' has created a pull request for this issue:

[jira] [Commented] (SPARK-10069) Python's ReduceByKeyAndWindow DStream Keeps Growing

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261011#comment-15261011 ] Davies Liu commented on SPARK-10069: cc [~zsxwing] > Python's ReduceByKeyAndWindow DStream Keeps

[jira] [Resolved] (SPARK-7891) Python class in __main__ may trigger AssertionError

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-7891. --- Resolution: Duplicate Assignee: Shixiong Zhu Fix Version/s: 2.0.0 > Python class in

[jira] [Commented] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-27 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261005#comment-15261005 ] Shivaram Venkataraman commented on SPARK-14831: --- +1 > Make ML APIs in SparkR consistent >

[jira] [Closed] (SPARK-12683) SQL timestamp is wrong when accessed as Python datetime

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-12683. -- Resolution: Won't Fix Assignee: Davies Liu > SQL timestamp is wrong when accessed as Python

[jira] [Commented] (SPARK-12683) SQL timestamp is wrong when accessed as Python datetime

2016-04-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261004#comment-15261004 ] Davies Liu commented on SPARK-12683: Done some debugging on this, it seems that the Java library

[jira] [Commented] (SPARK-14831) Make ML APIs in SparkR consistent

2016-04-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260981#comment-15260981 ] Xiangrui Meng commented on SPARK-14831: --- +1 on `read.ml` and `write.ml`, which are consistent with

[jira] [Resolved] (SPARK-14940) Move ExternalCatalog to own file

2016-04-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14940. - Resolution: Fixed Fix Version/s: 2.0.0 > Move ExternalCatalog to own file >

[jira] [Updated] (SPARK-14315) GLMs model persistence in SparkR

2016-04-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14315: -- Assignee: Gayathri Murali > GLMs model persistence in SparkR >

[jira] [Updated] (SPARK-14314) K-means model persistence in SparkR

2016-04-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14314: -- Shepherd: Yanbo Liang Assignee: Gayathri Murali Target Version/s: 2.0.0

[jira] [Updated] (SPARK-14315) GLMs model persistence in SparkR

2016-04-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14315: -- Target Version/s: 2.0.0 > GLMs model persistence in SparkR >

[jira] [Updated] (SPARK-14315) GLMs model persistence in SparkR

2016-04-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14315: -- Shepherd: Yanbo Liang > GLMs model persistence in SparkR > >

[jira] [Resolved] (SPARK-14899) Remove spark.ml HashingTF hashingAlg option

2016-04-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14899. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12702

  1   2   3   >