[jira] [Commented] (SPARK-15101) Audit: ml.clustering and ml.recommendation

2016-05-20 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294737#comment-15294737 ] yuhao yang commented on SPARK-15101: Resolve the issue here as all sub tasks are finished. cc

[jira] [Assigned] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15461: Assignee: (was: Apache Spark) > modify python test script using default version 2.7 >

[jira] [Assigned] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15461: Assignee: Apache Spark > modify python test script using default version 2.7 >

[jira] [Commented] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294732#comment-15294732 ] Apache Spark commented on SPARK-15461: -- User 'WeichenXu123' has created a pull request for this

[jira] [Updated] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15461: --- Component/s: Tests PySpark > modify python test script using default version 2.7 >

[jira] [Created] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Xiao Li (JIRA)
Xiao Li created SPARK-15460: --- Summary: Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir Key: SPARK-15460 URL: https://issues.apache.org/jira/browse/SPARK-15460

[jira] [Commented] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294730#comment-15294730 ] Apache Spark commented on SPARK-15460: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Created] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-15461: -- Summary: modify python test script using default version 2.7 Key: SPARK-15461 URL: https://issues.apache.org/jira/browse/SPARK-15461 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15460: Assignee: (was: Apache Spark) > Issue Exceptions from Thrift Server and Spark-SQL Cli

[jira] [Updated] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15460: Issue Type: Improvement (was: Bug) > Issue Exceptions from Thrift Server and Spark-SQL Cli when Users

[jira] [Assigned] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15460: Assignee: Apache Spark > Issue Exceptions from Thrift Server and Spark-SQL Cli when Users

[jira] [Closed] (SPARK-15320) Spark-SQL Cli Ignores Parameter hive.metastore.warehouse.dir

2016-05-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-15320. --- Resolution: Not A Problem > Spark-SQL Cli Ignores Parameter hive.metastore.warehouse.dir >

[jira] [Commented] (SPARK-15098) Audit: ml.classification

2016-05-20 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294719#comment-15294719 ] yuhao yang commented on SPARK-15098: I've made a pass and found no notable changes required for user

[jira] [Resolved] (SPARK-15437) Failed to create HiveContext in SparkR

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15437. - Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.0.0 > Failed to create

[jira] [Resolved] (SPARK-15424) Revert SPARK-14807 Create a hivecontext-compatibility module

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15424. - Resolution: Fixed > Revert SPARK-14807 Create a hivecontext-compatibility module >

[jira] [Commented] (SPARK-15459) Make Range logical and physical explain consistent

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294686#comment-15294686 ] Apache Spark commented on SPARK-15459: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15459) Make Range logical and physical explain consistent

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15459: Assignee: Reynold Xin (was: Apache Spark) > Make Range logical and physical explain

[jira] [Assigned] (SPARK-15459) Make Range logical and physical explain consistent

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15459: Assignee: Apache Spark (was: Reynold Xin) > Make Range logical and physical explain

[jira] [Created] (SPARK-15459) Make Range logical and physical explain consistent

2016-05-20 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15459: --- Summary: Make Range logical and physical explain consistent Key: SPARK-15459 URL: https://issues.apache.org/jira/browse/SPARK-15459 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-15429) When `spark.streaming.concurrentJobs > 1`, PIDRateEstimator cannot estimate the receiving rate accurately.

2016-05-20 Thread Albert Cheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292908#comment-15292908 ] Albert Cheng edited comment on SPARK-15429 at 5/21/16 2:58 AM: --- I have a

[jira] [Commented] (SPARK-15329) When start spark with yarn: spark.SparkContext: Error initializing SparkContext.

2016-05-20 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294653#comment-15294653 ] Saisai Shao commented on SPARK-15329: - {code} 2016-05-15 00:06:08,368 WARN

[jira] [Updated] (SPARK-15423) why it is very slow to clean resources in Spark-2.0.0-preview

2016-05-20 Thread zszhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zszhong updated SPARK-15423: Summary: why it is very slow to clean resources in Spark-2.0.0-preview (was: why it is very slow to clean

[jira] [Commented] (SPARK-15423) why it is very slow to clean resources in Spark

2016-05-20 Thread zszhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294637#comment-15294637 ] zszhong commented on SPARK-15423: - I've also downloaded spark-1.6.1 to run the same code and application.

[jira] [Commented] (SPARK-15453) Sort Merge Join to use bucketing metadata to optimize query plan

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294627#comment-15294627 ] Tejas Patil commented on SPARK-15453: - [~rxin] Yes. I updated the jira title. If we avoid the

[jira] [Comment Edited] (SPARK-15453) Sort Merge Join to use bucketing metadata to optimize query plan

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294627#comment-15294627 ] Tejas Patil edited comment on SPARK-15453 at 5/21/16 1:42 AM: -- [~rxin] Yes.

[jira] [Assigned] (SPARK-15458) Disable schema inference for streaming datasets on file streams

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15458: Assignee: Apache Spark (was: Tathagata Das) > Disable schema inference for streaming

[jira] [Commented] (SPARK-15458) Disable schema inference for streaming datasets on file streams

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294623#comment-15294623 ] Apache Spark commented on SPARK-15458: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15458) Disable schema inference for streaming datasets on file streams

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15458: Assignee: Tathagata Das (was: Apache Spark) > Disable schema inference for streaming

[jira] [Created] (SPARK-15458) Disable schema inference for streaming datasets on file streams

2016-05-20 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-15458: - Summary: Disable schema inference for streaming datasets on file streams Key: SPARK-15458 URL: https://issues.apache.org/jira/browse/SPARK-15458 Project: Spark

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294591#comment-15294591 ] Joseph K. Bradley commented on SPARK-15457: --- By the way, I plan to deprecate spark.mllib

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294590#comment-15294590 ] Joseph K. Bradley commented on SPARK-15457: --- OK I have a WIP one for the SGD issues. Shall we

[jira] [Commented] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-05-20 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294588#comment-15294588 ] DB Tsai commented on SPARK-7159: Hello [~sethah], I think we will make it as separate SoftmaxRegression

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-05-20 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294580#comment-15294580 ] Liwei Lin commented on SPARK-15406: --- Hi [~c...@koeninger.org], any plan on this? Thanks! > Structured

[jira] [Resolved] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15456. --- Resolution: Fixed Assignee: Bryan Cutler Fix Version/s: 2.0.0 Target

[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR

2016-05-20 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294522#comment-15294522 ] Miao Wang commented on SPARK-15439: --- Reproduced. Now I am analyzing the reason. > Failed to run unit

[jira] [Updated] (SPARK-15273) YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect OnOutOfMemoryError parameter given by user

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15273: -- Assignee: Ted Yu > YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect >

[jira] [Resolved] (SPARK-15273) YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect OnOutOfMemoryError parameter given by user

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15273. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13057

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294495#comment-15294495 ] Sean Owen commented on SPARK-15457: --- Yeah I have a PR brewing that will fix some of them, like the ones

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294494#comment-15294494 ] Kazuaki Ishizaki commented on SPARK-15285: -- I can take it today if they are busy. > Generated

[jira] [Assigned] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15456: Assignee: Apache Spark > PySpark Shell fails to create SparkContext if HiveConf not found

[jira] [Assigned] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15456: Assignee: (was: Apache Spark) > PySpark Shell fails to create SparkContext if

[jira] [Commented] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294462#comment-15294462 ] Apache Spark commented on SPARK-15456: -- User 'BryanCutler' has created a pull request for this

[jira] [Updated] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15457: -- Description: Several classes and methods have been deprecated and are creating lots of

[jira] [Commented] (SPARK-15455) For IsolatedClientLoader, we need to provide a conf to disable sharing Hadoop classes

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294454#comment-15294454 ] Apache Spark commented on SPARK-15455: -- User 'yhuai' has created a pull request for this issue:

[jira] [Created] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-15457: - Summary: Eliminate MLlib 2.0 build warnings from deprecations Key: SPARK-15457 URL: https://issues.apache.org/jira/browse/SPARK-15457 Project: Spark

[jira] [Created] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-15456: Summary: PySpark Shell fails to create SparkContext if HiveConf not found Key: SPARK-15456 URL: https://issues.apache.org/jira/browse/SPARK-15456 Project: Spark

[jira] [Commented] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294446#comment-15294446 ] Bryan Cutler commented on SPARK-15456: -- I can submit a fix for this > PySpark Shell fails to create

[jira] [Commented] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294437#comment-15294437 ] Apache Spark commented on SPARK-15327: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15327: Assignee: Apache Spark (was: Davies Liu) > Catalyst code generation fails with complex

[jira] [Assigned] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15327: Assignee: Davies Liu (was: Apache Spark) > Catalyst code generation fails with complex

[jira] [Commented] (SPARK-15451) Spark PR builder should fail if code doesn't compile against JDK 7

2016-05-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294426#comment-15294426 ] Marcelo Vanzin commented on SPARK-15451: I think there's still some value even if 2.1.0 switches

[jira] [Updated] (SPARK-15449) Wrong Data Format - Documentation Issue

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15449: -- Target Version/s: (was: 1.6.1) Fix Version/s: (was: 1.6.1) [~wangmiao1981] the problem is

[jira] [Resolved] (SPARK-15078) Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15078. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13188

[jira] [Comment Edited] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-05-20 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294372#comment-15294372 ] Seth Hendrickson edited comment on SPARK-7159 at 5/20/16 10:17 PM: ---

[jira] [Resolved] (SPARK-15446) catalyst using BigInteger.longValueExact that not supporting java 7 and compile error

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15446. --- Resolution: Duplicate > catalyst using BigInteger.longValueExact that not supporting java 7 and >

[jira] [Reopened] (SPARK-15446) catalyst using BigInteger.longValueExact that not supporting java 7 and compile error

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15446: --- [~WeichenXu123] yes please search JIRA first, and "Fixed" is not the correct resolution > catalyst

[jira] [Commented] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-05-20 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294372#comment-15294372 ] Seth Hendrickson commented on SPARK-7159: - [~dbtsai][~josephkb] I'd like to take this one if it's

[jira] [Commented] (SPARK-15451) Spark PR builder should fail if code doesn't compile against JDK 7

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294370#comment-15294370 ] Sean Owen commented on SPARK-15451: --- That's fine with me, but so is just going ahead and requiring Java

[jira] [Resolved] (SPARK-15454) HadoopFsRelation should filter out files starting with _

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15454. - Resolution: Fixed Fix Version/s: 2.0.0 > HadoopFsRelation should filter out files

[jira] [Assigned] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15327: -- Assignee: Davies Liu > Catalyst code generation fails with complex data structure >

[jira] [Created] (SPARK-15455) For IsolatedClientLoader, we need to provide a conf to disable sharing Hadoop classes

2016-05-20 Thread Yin Huai (JIRA)
Yin Huai created SPARK-15455: Summary: For IsolatedClientLoader, we need to provide a conf to disable sharing Hadoop classes Key: SPARK-15455 URL: https://issues.apache.org/jira/browse/SPARK-15455

[jira] [Commented] (SPARK-8426) Add blacklist mechanism for YARN container allocation

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294242#comment-15294242 ] Apache Spark commented on SPARK-8426: - User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-15449) Wrong Data Format - Documentation Issue

2016-05-20 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294237#comment-15294237 ] Miao Wang commented on SPARK-15449: --- This example doesn't require libsvm format as it has its own data

[jira] [Commented] (SPARK-11827) Support java.math.BigInteger in Type-Inference utilities for POJOs

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294219#comment-15294219 ] Apache Spark commented on SPARK-11827: -- User 'ted-yu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15405) YARN uploading the same __spark_conf__.zip twice

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15405: Assignee: Apache Spark > YARN uploading the same __spark_conf__.zip twice >

[jira] [Assigned] (SPARK-15405) YARN uploading the same __spark_conf__.zip twice

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15405: Assignee: (was: Apache Spark) > YARN uploading the same __spark_conf__.zip twice >

[jira] [Commented] (SPARK-15405) YARN uploading the same __spark_conf__.zip twice

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294212#comment-15294212 ] Apache Spark commented on SPARK-15405: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-4563) Allow spark driver to bind to different ip then advertise ip

2016-05-20 Thread Miles Crawford (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294181#comment-15294181 ] Miles Crawford commented on SPARK-4563: --- Any chance we could boost the priority of this? I think

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294178#comment-15294178 ] Davies Liu commented on SPARK-15285: cc [~cloud_fan] > Generated SpecificSafeProjection.apply method

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15285: --- Assignee: Wenchen Fan > Generated SpecificSafeProjection.apply method grows beyond 64 KB >

[jira] [Updated] (SPARK-15453) Sort Merge Join to use bucketing metadata to optimize query plan

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15453: Summary: Sort Merge Join to use bucketing metadata to optimize query plan (was: Improve join

[jira] [Commented] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294168#comment-15294168 ] Davies Liu commented on SPARK-14331: Could you post the full stacktrace? This exception should be

[jira] [Commented] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294165#comment-15294165 ] Apache Spark commented on SPARK-15453: -- User 'tejasapatil' has created a pull request for this

[jira] [Assigned] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15453: Assignee: Apache Spark > Improve join planning for bucketed / sorted tables >

[jira] [Assigned] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15453: Assignee: (was: Apache Spark) > Improve join planning for bucketed / sorted tables >

[jira] [Commented] (SPARK-15165) Codegen can break because toCommentSafeString is not actually safe

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294144#comment-15294144 ] Apache Spark commented on SPARK-15165: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-15205) Codegen can compile the same source code more than twice

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294145#comment-15294145 ] Apache Spark commented on SPARK-15205: -- User 'sarutak' has created a pull request for this issue:

[jira] [Closed] (SPARK-15448) Flaky test:pyspark.ml.tests.DefaultValuesTests.test_java_params

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-15448. -- Resolution: Duplicate Fix Version/s: 2.0.0 > Flaky

[jira] [Assigned] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14031: Assignee: Apache Spark (was: Davies Liu) > Dataframe to csv IO, system performance

[jira] [Assigned] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14031: Assignee: Davies Liu (was: Apache Spark) > Dataframe to csv IO, system performance

[jira] [Commented] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294127#comment-15294127 ] Apache Spark commented on SPARK-14031: -- User 'davies' has created a pull request for this issue:

[jira] [Resolved] (SPARK-15438) Improve the explain of whole-stage codegen

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15438. - Resolution: Fixed Fix Version/s: 2.0.0 > Improve the explain of whole-stage codegen >

[jira] [Commented] (SPARK-15447) Performance test for ALS in Spark 2.0

2016-05-20 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294116#comment-15294116 ] Nick Pentreath commented on SPARK-15447: [~mengxr] yes will aim to run some tests during early

[jira] [Commented] (SPARK-15294) Add pivot functionality to SparkR

2016-05-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294091#comment-15294091 ] Felix Cheung commented on SPARK-15294: -- Feel free to ping me if you need any help! On Thu, May

[jira] [Assigned] (SPARK-15442) PySpark QuantileDiscretizer missing "relativeError" param

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15442: Assignee: Apache Spark (was: Nick Pentreath) > PySpark QuantileDiscretizer missing

[jira] [Assigned] (SPARK-15442) PySpark QuantileDiscretizer missing "relativeError" param

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15442: Assignee: Nick Pentreath (was: Apache Spark) > PySpark QuantileDiscretizer missing

[jira] [Assigned] (SPARK-15442) PySpark QuantileDiscretizer missing "relativeError" param

2016-05-20 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath reassigned SPARK-15442: -- Assignee: Nick Pentreath > PySpark QuantileDiscretizer missing "relativeError" param

[jira] [Assigned] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-14031: -- Assignee: Davies Liu > Dataframe to csv IO, system performance enters high CPU state and

[jira] [Commented] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294053#comment-15294053 ] Reynold Xin commented on SPARK-15453: - [~tejasp] there are multiple issues here right? The ticket is

[jira] [Updated] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15453: Summary: Improve join planning for bucketed / sorted tables (was: Support for SMB Join) >

[jira] [Assigned] (SPARK-15454) HadoopFsRelation should filter out files starting with _

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15454: Assignee: Reynold Xin (was: Apache Spark) > HadoopFsRelation should filter out files

[jira] [Commented] (SPARK-15454) HadoopFsRelation should filter out files starting with _

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294047#comment-15294047 ] Apache Spark commented on SPARK-15454: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15454) HadoopFsRelation should filter out files starting with _

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15454: Assignee: Apache Spark (was: Reynold Xin) > HadoopFsRelation should filter out files

[jira] [Comment Edited] (SPARK-15451) Spark PR builder should fail if code doesn't compile against JDK 7

2016-05-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293958#comment-15293958 ] Marcelo Vanzin edited comment on SPARK-15451 at 5/20/16 7:38 PM: - I'm not

[jira] [Resolved] (SPARK-15190) Support using SQLUserDefinedType for case classes

2016-05-20 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-15190. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12965

[jira] [Created] (SPARK-15454) HadoopFsRelation should filter out files starting with _

2016-05-20 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15454: --- Summary: HadoopFsRelation should filter out files starting with _ Key: SPARK-15454 URL: https://issues.apache.org/jira/browse/SPARK-15454 Project: Spark Issue

[jira] [Updated] (SPARK-15453) Support for SMB Join

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15453: Description: Datasource allows creation of bucketed and sorted tables but performing joins on

[jira] [Updated] (SPARK-15453) Support for SMB Join

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15453: Description: Datasource allows creation of bucketed and sorted tables but performing joins on

[jira] [Updated] (SPARK-15453) Support for SMB Join

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15453: Description: Datasource allows creation of bucketed and sorted tables but performing joins on

[jira] [Created] (SPARK-15453) Support for SMB Join

2016-05-20 Thread Tejas Patil (JIRA)
Tejas Patil created SPARK-15453: --- Summary: Support for SMB Join Key: SPARK-15453 URL: https://issues.apache.org/jira/browse/SPARK-15453 Project: Spark Issue Type: New Feature

  1   2   3   >