[jira] [Resolved] (SPARK-17761) Simplify InternalRow hierarchy

2016-10-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17761. --- Resolution: Fixed Fix Version/s: 2.1.0 > Simplify InternalRow hierarchy >

[jira] [Updated] (SPARK-17832) TableIdentifier.quotedString creates un-parseable names when name contains a backtick

2016-10-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17832: -- Summary: TableIdentifier.quotedString creates un-parseable names when name contains a

[jira] [Commented] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556248#comment-15556248 ] Apache Spark commented on SPARK-17738: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17738: Assignee: Davies Liu (was: Apache Spark) > Flaky test:

[jira] [Assigned] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17738: Assignee: Apache Spark (was: Davies Liu) > Flaky test:

[jira] [Commented] (SPARK-15611) Got the same sequence random number in every forked worker.

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556249#comment-15556249 ] holdenk commented on SPARK-15611: - So this is marked as resolved but there is an open PR associated with

[jira] [Commented] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-07 Thread Ioana Delaney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556305#comment-15556305 ] Ioana Delaney commented on SPARK-17626: --- [~rxin] Hi Reynold, Thank you for your reply. My argument

[jira] [Created] (SPARK-17833) 'monotonicallyIncreasingId()' should probably be deterministic

2016-10-07 Thread Kevin Ushey (JIRA)
Kevin Ushey created SPARK-17833: --- Summary: 'monotonicallyIncreasingId()' should probably be deterministic Key: SPARK-17833 URL: https://issues.apache.org/jira/browse/SPARK-17833 Project: Spark

[jira] [Updated] (SPARK-17831) registerTempTable is ignoring database clarifications

2016-10-07 Thread Saif Addin Ellafi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saif Addin Ellafi updated SPARK-17831: -- Description: So, I am writing a general spark api to use a Hive Metastore. In case I

[jira] [Updated] (SPARK-17806) Incorrect result when work with data from parquet

2016-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17806: --- Component/s: SQL > Incorrect result when work with data from parquet >

[jira] [Resolved] (SPARK-7182) [SQL] Can't remove columns from DataFrame or save DataFrame from a join due to duplicate columns

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7182. Resolution: Fixed > [SQL] Can't remove columns from DataFrame or save DataFrame from a join due > to

[jira] [Commented] (SPARK-17828) Changes file missing in Spark 2

2016-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1092#comment-1092 ] Sean Owen commented on SPARK-17828: --- I think I'd suggest just removing it. The release notes also link

[jira] [Reopened] (SPARK-17344) Kafka 0.8 support for Structured Streaming

2016-10-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reopened SPARK-17344: -- I'm going to reopen for now since there is interests and you can't vote on closed issues.

[jira] [Assigned] (SPARK-17749) Unresolved columns when nesting SQL join clauses

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17749: Assignee: (was: Apache Spark) > Unresolved columns when nesting SQL join clauses >

[jira] [Assigned] (SPARK-17749) Unresolved columns when nesting SQL join clauses

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17749: Assignee: Apache Spark > Unresolved columns when nesting SQL join clauses >

[jira] [Resolved] (SPARK-15621) BatchEvalPythonExec fails with OOM

2016-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15621. - Resolution: Fixed Fix Version/s: 2.1.0 > BatchEvalPythonExec fails with OOM >

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556324#comment-15556324 ] holdenk commented on SPARK-650: --- I think this is a duplicate of SPARK-636 yes? > Add a "setup hook" API for

[jira] [Resolved] (SPARK-17806) Incorrect result when work with data from parquet

2016-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17806. Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 Issue resolved by pull

[jira] [Commented] (SPARK-7365) Unable to custom bind the column in cached query result (temp table)

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556393#comment-15556393 ] Xiao Li commented on SPARK-7365: Could you retry it using the latest master branch with Cassandra

[jira] [Resolved] (SPARK-7480) Get exception when DataFrame saveAsTable and run sql on the same table at the same time

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7480. Resolution: Fixed > Get exception when DataFrame saveAsTable and run sql on the same table at the > same

[jira] [Commented] (SPARK-7480) Get exception when DataFrame saveAsTable and run sql on the same table at the same time

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556401#comment-15556401 ] Xiao Li commented on SPARK-7480: This has been resolved in the latest branch. Thanks! If you still hit the

[jira] [Commented] (SPARK-7875) Exception when using CLUSTER BY or ORDER BY

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556407#comment-15556407 ] Xiao Li commented on SPARK-7875: This should have been resolved in the latest branch. Close it now and

[jira] [Commented] (SPARK-14926) OneVsRest labelMetadata uses incorrect name

2016-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556180#comment-15556180 ] Joseph K. Bradley commented on SPARK-14926: --- Sorry, just catching up on this. I commented on

[jira] [Created] (SPARK-17832) TableIdentifier.quotedString creates un-parseable names when name contains a backpack

2016-10-07 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-17832: - Summary: TableIdentifier.quotedString creates un-parseable names when name contains a backpack Key: SPARK-17832 URL: https://issues.apache.org/jira/browse/SPARK-17832

[jira] [Commented] (SPARK-17824) QR solver for WeightedLeastSquares

2016-10-07 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556237#comment-15556237 ] Seth Hendrickson commented on SPARK-17824: -- Thank you for clarifying > QR solver for

[jira] [Resolved] (SPARK-7034) Support escaped double quotes on data source options

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7034. Resolution: Fixed > Support escaped double quotes on data source options >

[jira] [Commented] (SPARK-7182) [SQL] Can't remove columns from DataFrame or save DataFrame from a join due to duplicate columns

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556388#comment-15556388 ] Xiao Li commented on SPARK-7182: Please try it in the master branch. If it still does not work, please

[jira] [Resolved] (SPARK-7365) Unable to custom bind the column in cached query result (temp table)

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7365. Resolution: Fixed > Unable to custom bind the column in cached query result (temp table) >

[jira] [Commented] (SPARK-15899) file scheme should be used correctly

2016-10-07 Thread Daniel Barclay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556174#comment-15556174 ] Daniel Barclay commented on SPARK-15899: Regarding the reference to [RFC

[jira] [Commented] (SPARK-636) Add mechanism to run system management/configuration tasks on all workers

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556322#comment-15556322 ] holdenk commented on SPARK-636: --- Does broadcasting get us close enough to handling this or is this something

[jira] [Commented] (SPARK-7034) Support escaped double quotes on data source options

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556367#comment-15556367 ] Xiao Li commented on SPARK-7034: I tried it in the latest master. It should work at least after we have

[jira] [Resolved] (SPARK-6959) Support for datetime comparisions in filter for dataframes in pyspark

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-6959. Resolution: Duplicate Let us know if you still hit this issue. > Support for datetime comparisions in

[jira] [Resolved] (SPARK-7875) Exception when using CLUSTER BY or ORDER BY

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7875. Resolution: Fixed > Exception when using CLUSTER BY or ORDER BY >

[jira] [Commented] (SPARK-7611) Support HashJoin if the join condition uses eqNullSafe/<=>

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556417#comment-15556417 ] Xiao Li commented on SPARK-7611: Now, we can support EqualNullSafe in ExtractEquiJoinKeys. Thanks! >

[jira] [Resolved] (SPARK-7611) Support HashJoin if the join condition uses eqNullSafe/<=>

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7611. Resolution: Fixed > Support HashJoin if the join condition uses eqNullSafe/<=> >

[jira] [Commented] (SPARK-7730) Complex Teradata queries throwing Analysis Exception when running on spark

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556421#comment-15556421 ] Xiao Li commented on SPARK-7730: It should have been fixed. Please reopen it, if you still hit it. Thanks!

[jira] [Resolved] (SPARK-7730) Complex Teradata queries throwing Analysis Exception when running on spark

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7730. Resolution: Fixed > Complex Teradata queries throwing Analysis Exception when running on spark >

[jira] [Commented] (SPARK-7728) SQL: cannot add timestamp partition as string

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556423#comment-15556423 ] Xiao Li commented on SPARK-7728: It should have been fixed in the native DDL support. Please check it. If

[jira] [Resolved] (SPARK-7728) SQL: cannot add timestamp partition as string

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7728. Resolution: Fixed > SQL: cannot add timestamp partition as string >

[jira] [Commented] (SPARK-7646) Create table support to JDBC Datasource

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556426#comment-15556426 ] Xiao Li commented on SPARK-7646: Now, our save API can support JDBC data sources in the latest branch.

[jira] [Commented] (SPARK-14804) Graph vertexRDD/EdgeRDD checkpoint results ClassCastException:

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556429#comment-15556429 ] Apache Spark commented on SPARK-14804: -- User 'tdas' has created a pull request for this issue:

[jira] [Resolved] (SPARK-7646) Create table support to JDBC Datasource

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7646. Resolution: Fixed > Create table support to JDBC Datasource > --- > >

[jira] [Created] (SPARK-17829) Stable format for offset log

2016-10-07 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-17829: Summary: Stable format for offset log Key: SPARK-17829 URL: https://issues.apache.org/jira/browse/SPARK-17829 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-17830) Annotate Spark SQL public APIs with InterfaceStability

2016-10-07 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-17830: --- Summary: Annotate Spark SQL public APIs with InterfaceStability Key: SPARK-17830 URL: https://issues.apache.org/jira/browse/SPARK-17830 Project: Spark Issue

[jira] [Resolved] (SPARK-17707) Web UI prevents spark-submit application to be finished

2016-10-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17707. -- Resolution: Fixed Assignee: Sean Owen Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

2016-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1759#comment-1759 ] Davies Liu commented on SPARK-17738: I will looking into that. > Flaky test:

[jira] [Assigned] (SPARK-17830) Annotate Spark SQL public APIs with InterfaceStability

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17830: Assignee: Reynold Xin (was: Apache Spark) > Annotate Spark SQL public APIs with

[jira] [Assigned] (SPARK-17830) Annotate Spark SQL public APIs with InterfaceStability

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17830: Assignee: Apache Spark (was: Reynold Xin) > Annotate Spark SQL public APIs with

[jira] [Commented] (SPARK-17830) Annotate Spark SQL public APIs with InterfaceStability

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1757#comment-1757 ] Apache Spark commented on SPARK-17830: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-17806) Incorrect result when work with data from parquet

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1697#comment-1697 ] Apache Spark commented on SPARK-17806: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17806) Incorrect result when work with data from parquet

2016-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-17806: -- Assignee: Davies Liu > Incorrect result when work with data from parquet >

[jira] [Reopened] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

2016-10-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reopened SPARK-17738: -- It's still flaky. See:

[jira] [Closed] (SPARK-17375) Star Join Optimization

2016-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-17375. --- Resolution: Duplicate Marking this as duplicate of SPARK-17626 > Star Join Optimization >

[jira] [Commented] (SPARK-10643) Support HDFS application download in client mode spark submit

2016-10-07 Thread Laurent Hoss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1581#comment-1581 ] Laurent Hoss commented on SPARK-10643: -- +1 would be very useful when using Zeppelin (running in

[jira] [Resolved] (SPARK-17800) Introduce InterfaceStability annotation definition

2016-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17800. - Resolution: Fixed Fix Version/s: 2.1.0 > Introduce InterfaceStability annotation

[jira] [Updated] (SPARK-17806) Incorrect result when work with data from parquet

2016-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-17806: --- Priority: Blocker (was: Critical) > Incorrect result when work with data from parquet >

[jira] [Assigned] (SPARK-17806) Incorrect result when work with data from parquet

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17806: Assignee: Apache Spark (was: Davies Liu) > Incorrect result when work with data from

[jira] [Assigned] (SPARK-17806) Incorrect result when work with data from parquet

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17806: Assignee: Davies Liu (was: Apache Spark) > Incorrect result when work with data from

[jira] [Updated] (SPARK-17417) Fix sorting of part files while reconstructing RDD/partition from checkpointed files.

2016-10-07 Thread Dhruve Ashar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruve Ashar updated SPARK-17417: - Summary: Fix sorting of part files while reconstructing RDD/partition from checkpointed files.

[jira] [Commented] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-07 Thread Harrison Kiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1467#comment-1467 ] Harrison Kiang commented on SPARK-17794: [~rxin] [~srowen] 2.0.1 is in maven.

[jira] [Commented] (SPARK-17824) QR solver for WeightedLeastSquares

2016-10-07 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1448#comment-1448 ] Yanbo Liang commented on SPARK-17824: - [~sethah] I saw your proposal at SPARK-17748: {code} class

[jira] [Commented] (SPARK-14503) spark.ml API for FPGrowth

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1202#comment-1202 ] holdenk commented on SPARK-14503: - [~jeffzhang] & [~yuhaoyan] are you still working on this? > spark.ml

[jira] [Commented] (SPARK-14503) spark.ml API for FPGrowth

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1198#comment-1198 ] holdenk commented on SPARK-14503: - +1 for porting the current functionality then updating from there. >

[jira] [Commented] (SPARK-17419) Mesos virtual network support

2016-10-07 Thread John Omernik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1258#comment-1258 ] John Omernik commented on SPARK-17419: -- In spending some time with Calico Networks Mesos

[jira] [Commented] (SPARK-17824) QR solver for WeightedLeastSquares

2016-10-07 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1385#comment-1385 ] Seth Hendrickson commented on SPARK-17824: -- [~yanboliang] Can you please post your design plans?

[jira] [Commented] (SPARK-17824) QR solver for WeightedLeastSquares

2016-10-07 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1460#comment-1460 ] Yanbo Liang commented on SPARK-17824: - [~sethah] That's cool. Let's work together and I will wait

[jira] [Comment Edited] (SPARK-17824) QR solver for WeightedLeastSquares

2016-10-07 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1385#comment-1385 ] Seth Hendrickson edited comment on SPARK-17824 at 10/7/16 3:42 PM: ---

[jira] [Comment Edited] (SPARK-17824) QR solver for WeightedLeastSquares

2016-10-07 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1448#comment-1448 ] Yanbo Liang edited comment on SPARK-17824 at 10/7/16 3:53 PM: -- [~sethah] I

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

2016-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1883#comment-1883 ] Joseph K. Bradley commented on SPARK-17219: --- I agree with [~thunterdb]. The 2 ways of handling

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

2016-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1925#comment-1925 ] Sean Owen commented on SPARK-17219: --- OK. Let me just work up a patch to fix forward rather than bother

[jira] [Created] (SPARK-17831) registerTempTable is ignoring database clarifications

2016-10-07 Thread Saif Addin Ellafi (JIRA)
Saif Addin Ellafi created SPARK-17831: - Summary: registerTempTable is ignoring database clarifications Key: SPARK-17831 URL: https://issues.apache.org/jira/browse/SPARK-17831 Project: Spark

[jira] [Updated] (SPARK-17346) Kafka 0.10 support in Structured Streaming

2016-10-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17346: - Fix Version/s: 2.0.2 > Kafka 0.10 support in Structured Streaming >

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

2016-10-07 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1968#comment-1968 ] Barry Becker commented on SPARK-17219: -- I'll make another attempt to clarify my use case. Nulls are

[jira] [Assigned] (SPARK-17346) Kafka 0.10 support in Structured Streaming

2016-10-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-17346: Assignee: Shixiong Zhu > Kafka 0.10 support in Structured Streaming >

[jira] [Commented] (SPARK-17811) SparkR cannot parallelize data.frame with NA or NULL in Date columns

2016-10-07 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1996#comment-1996 ] Miao Wang commented on SPARK-17811: --- > df <- data.frame(Date = as.POSIXlt(as.Date(c(rep("2016-01-10",

[jira] [Comment Edited] (SPARK-17811) SparkR cannot parallelize data.frame with NA or NULL in Date columns

2016-10-07 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1996#comment-1996 ] Miao Wang edited comment on SPARK-17811 at 10/7/16 7:10 PM: > df <-

[jira] [Commented] (SPARK-6115) Description for SparkSQL Jobs doesn't show up correctly until after the job finishes

2016-10-07 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1962#comment-1962 ] Kay Ousterhout commented on SPARK-6115: --- I'm going to close this due to inactivity, and it's so old

[jira] [Closed] (SPARK-6115) Description for SparkSQL Jobs doesn't show up correctly until after the job finishes

2016-10-07 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout closed SPARK-6115. - Resolution: Won't Fix > Description for SparkSQL Jobs doesn't show up correctly until after the

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

2016-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556130#comment-15556130 ] Joseph K. Bradley commented on SPARK-17219: --- That does make sense, and I agree we should

[jira] [Commented] (SPARK-17344) Kafka 0.8 support for Structured Streaming

2016-10-07 Thread Jeremy Smith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556129#comment-15556129 ] Jeremy Smith commented on SPARK-17344: -- +1 We're on CDH, and it will probably be a while before

[jira] [Resolved] (SPARK-17665) SparkR does not support options in other types consistently other APIs

2016-10-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-17665. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.1.0

[jira] [Resolved] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17794. --- Resolution: Fixed Fix Version/s: 2.0.1 I can confirm the artifacts were deployed as of about

[jira] [Commented] (SPARK-5881) RDD remains cached after the table gets overridden by "CACHE TABLE"

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556469#comment-15556469 ] Xiao Li commented on SPARK-5881: I think this has been resolved. Let me close it. Thanks! > RDD remains

[jira] [Resolved] (SPARK-7936) Add configuration for initial size and limit of hash for aggregation

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-7936. Resolution: Fixed > Add configuration for initial size and limit of hash for aggregation >

[jira] [Commented] (SPARK-8081) Problems with Optimized Logical Plan Generation

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556519#comment-15556519 ] Xiao Li commented on SPARK-8081: Could you retry it in the latest Spark version? Thanks! > Problems with

[jira] [Resolved] (SPARK-8527) StructType's Factory method does not work in java code

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-8527. Resolution: Fixed > StructType's Factory method does not work in java code >

[jira] [Commented] (SPARK-8527) StructType's Factory method does not work in java code

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556567#comment-15556567 ] Xiao Li commented on SPARK-8527: This should have been resolved. Could you retry it in the master branch.

[jira] [Commented] (SPARK-2868) Support named accumulators in Python

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556582#comment-15556582 ] holdenk commented on SPARK-2868: Is this something we are still interested in pursuing (cc [~rxin] who did

[jira] [Assigned] (SPARK-17834) Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17834: Assignee: Apache Spark (was: Shixiong Zhu) > Fetch the earliest offsets manually in

[jira] [Closed] (SPARK-5160) Python module in jars

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk closed SPARK-5160. -- Resolution: Fixed This is now supported. > Python module in jars > - > >

[jira] [Commented] (SPARK-7638) Python API for pmml.export

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556732#comment-15556732 ] holdenk commented on SPARK-7638: Do we still want to do this or focus on adding PMML export on ML given

[jira] [Commented] (SPARK-7177) Create standard way to wrap Spark CLI scripts for external projects

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556780#comment-15556780 ] holdenk commented on SPARK-7177: I've run into similar challenges when working on Sparkling Pandas. >

[jira] [Commented] (SPARK-8605) Exclude files in StreamingContext. textFileStream(directory)

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556785#comment-15556785 ] holdenk commented on SPARK-8605: This is semi-documented (namely only atomic moves are supported), but

[jira] [Updated] (SPARK-8605) Exclude files in StreamingContext. textFileStream(directory)

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-8605: --- Component/s: (was: PySpark) Streaming > Exclude files in StreamingContext.

[jira] [Closed] (SPARK-8719) Adding Python support for 1-sample, 2-sided Kolmogorov Smirnov Test

2016-10-07 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk closed SPARK-8719. -- Resolution: Duplicate > Adding Python support for 1-sample, 2-sided Kolmogorov Smirnov Test >

[jira] [Commented] (SPARK-11272) Support importing and exporting event logs from HistoryServer web portal

2016-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556866#comment-15556866 ] Apache Spark commented on SPARK-11272: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Updated] (SPARK-7603) Crash of thrift server when doing SQL without "limit"

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-7603: --- Component/s: (was: Web UI) SQL > Crash of thrift server when doing SQL without "limit" >

[jira] [Commented] (SPARK-7149) Defalt system alias problem

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556451#comment-15556451 ] Xiao Li commented on SPARK-7149: Since we changed the way to generate the alias name. c0 is not the alias

[jira] [Commented] (SPARK-17819) Specified database in JDBC URL is ignored when connecting to thriftserver

2016-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556450#comment-15556450 ] Dongjoon Hyun commented on SPARK-17819: --- Hi, [~tnemet]. I'm working on this and make a PR for this

[jira] [Commented] (SPARK-5781) Add metadata files for JSON datasets

2016-10-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556480#comment-15556480 ] Xiao Li commented on SPARK-5781: This sounds already resolved. Let me close it. Thanks! > Add metadata

  1   2   3   >