[jira] [Commented] (SPARK-17020) Materialization of RDD via DataFrame.rdd forces a poor re-distribution of data

[ https://issues.apache.org/jira/browse/SPARK-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417204#comment-15417204 ] Roi Reshef commented on SPARK-17020: [~srowen] I have 2 DataFrames that are generated from spark-csv

[jira] [Commented] (SPARK-17020) Materialization of RDD via DataFrame.rdd forces a poor re-distribution of data

[ https://issues.apache.org/jira/browse/SPARK-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417194#comment-15417194 ] Sean Owen commented on SPARK-17020: --- Can you provide more detail on how you created the DataFrame, the

[jira] [Commented] (SPARK-16993) model.transform without label column in random forest regression

[ https://issues.apache.org/jira/browse/SPARK-16993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417192#comment-15417192 ] Sean Owen commented on SPARK-16993: --- Yes, that's clear. You haven't said what the error is, and I

[jira] [Commented] (SPARK-16975) Spark-2.0.0 unable to infer schema for parquet data written by Spark-1.6.2

[ https://issues.apache.org/jira/browse/SPARK-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417186#comment-15417186 ] immerrr again commented on SPARK-16975: --- The figures were: 1.6.2: ~6s 2.0.0: ~12s Interestingly

[jira] [Updated] (SPARK-17020) Materialization of RDD via DataFrame.rdd forces a poor re-distribution of data

[ https://issues.apache.org/jira/browse/SPARK-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roi Reshef updated SPARK-17020: --- Affects Version/s: 2.0.0 > Materialization of RDD via DataFrame.rdd forces a poor re-distribution of

[jira] [Commented] (SPARK-16975) Spark-2.0.0 unable to infer schema for parquet data written by Spark-1.6.2

[ https://issues.apache.org/jira/browse/SPARK-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417172#comment-15417172 ] immerrr again commented on SPARK-16975: --- But it works. I have suppressed WARN logs and

[jira] [Updated] (SPARK-17020) Materialization of RDD via DataFrame.rdd forces a poor re-distribution of data

[ https://issues.apache.org/jira/browse/SPARK-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roi Reshef updated SPARK-17020: --- Attachment: rdd_cache.PNG dataframe_cache.PNG > Materialization of RDD via

[jira] [Created] (SPARK-17020) Materialization of RDD via DataFrame.rdd forces a poor re-distribution of data

Roi Reshef created SPARK-17020: -- Summary: Materialization of RDD via DataFrame.rdd forces a poor re-distribution of data Key: SPARK-17020 URL: https://issues.apache.org/jira/browse/SPARK-17020 Project:

[jira] [Commented] (SPARK-16993) model.transform without label column in random forest regression

[ https://issues.apache.org/jira/browse/SPARK-16993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417144#comment-15417144 ] Dulaj Rajitha commented on SPARK-16993: --- Here is the scenario. My train data set has : features,and

[jira] [Commented] (SPARK-16975) Spark-2.0.0 unable to infer schema for parquet data written by Spark-1.6.2

[ https://issues.apache.org/jira/browse/SPARK-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417113#comment-15417113 ] immerrr again commented on SPARK-16975: --- I have built the code from the PR and it indeed succeeds

[jira] [Updated] (SPARK-15899) file scheme should be used correctly

[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15899: -- Fix Version/s: 2.0.1 > file scheme should be used correctly > > >

[jira] [Commented] (SPARK-16966) App Name is a randomUUID even when "spark.app.name" exists

[ https://issues.apache.org/jira/browse/SPARK-16966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417007#comment-15417007 ] Apache Spark commented on SPARK-16966: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13979: Assignee: Apache Spark > Killed executor is respawned without AWS keys in standalone

[jira] [Commented] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417005#comment-15417005 ] Apache Spark commented on SPARK-13979: -- User 'agsachin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13979: Assignee: (was: Apache Spark) > Killed executor is respawned without AWS keys in

[jira] [Updated] (SPARK-16952) [MESOS] MesosCoarseGrainedSchedulerBackend requires spark.mesos.executor.home even if spark.executor.uri is set

[ https://issues.apache.org/jira/browse/SPARK-16952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16952: -- Assignee: Michael Gummelt > [MESOS] MesosCoarseGrainedSchedulerBackend requires

[jira] [Resolved] (SPARK-16952) [MESOS] MesosCoarseGrainedSchedulerBackend requires spark.mesos.executor.home even if spark.executor.uri is set

[ https://issues.apache.org/jira/browse/SPARK-16952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16952. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14552

[jira] [Updated] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16886: -- External issue URL: (was: https://github.com/apache/spark/pull/13816/files) Labels:

[jira] [Updated] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16886: -- Assignee: Hyukjin Kwon > StructuredNetworkWordCount code comment incorrectly refers to DataFrame >

[jira] [Resolved] (SPARK-16886) StructuredNetworkWordCount code comment incorrectly refers to DataFrame instead of Dataset

[ https://issues.apache.org/jira/browse/SPARK-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16886. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14564

[jira] [Updated] (SPARK-16941) SparkSQLOperationManager should use synchronized Map to store SessionHandle

[ https://issues.apache.org/jira/browse/SPARK-16941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16941: -- Assignee: carlmartin > SparkSQLOperationManager should use synchronized Map to store SessionHandle >

[jira] [Resolved] (SPARK-16941) SparkSQLOperationManager should use synchronized Map to store SessionHandle

[ https://issues.apache.org/jira/browse/SPARK-16941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16941. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14534

[jira] [Commented] (SPARK-15899) file scheme should be used correctly

[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416977#comment-15416977 ] Apache Spark commented on SPARK-15899: -- User 'avulanov' has created a pull request for this issue:

[jira] [Updated] (SPARK-17001) Enable standardScaler to standardize sparse vectors when withMean=True

[ https://issues.apache.org/jira/browse/SPARK-17001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17001: -- Shepherd: (was: Tobi Bosede) Flags: (was: Important) Affects

[jira] [Commented] (SPARK-5160) Python module in jars

[ https://issues.apache.org/jira/browse/SPARK-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416907#comment-15416907 ] Semet commented on SPARK-5160: -- zip files are already supported, just add zip to --pyfiles and they get added

[jira] [Reopened] (SPARK-17004) Fix warning "method declarations in class TypeApi is deprecated"

[ https://issues.apache.org/jira/browse/SPARK-17004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-17004: --- > Fix warning "method declarations in class TypeApi is deprecated" >

[jira] [Resolved] (SPARK-17004) Fix warning "method declarations in class TypeApi is deprecated"

[ https://issues.apache.org/jira/browse/SPARK-17004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17004. --- Resolution: Not A Problem > Fix warning "method declarations in class TypeApi is deprecated" >

[jira] [Commented] (SPARK-17016) group-by/order-by ordinal should throw AnalysisException instead of UnresolvedException

[ https://issues.apache.org/jira/browse/SPARK-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416898#comment-15416898 ] Reynold Xin commented on SPARK-17016: - This was subsumed by

[jira] [Updated] (SPARK-17010) [MINOR]Wrong description in memory management document

[ https://issues.apache.org/jira/browse/SPARK-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17010: -- Priority: Trivial (was: Minor) > [MINOR]Wrong description in memory management document >

[jira] [Resolved] (SPARK-17016) group-by/order-by ordinal should throw AnalysisException instead of UnresolvedException

[ https://issues.apache.org/jira/browse/SPARK-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17016. - Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.1.0 > group-by/order-by

[jira] [Updated] (SPARK-17011) Support testing exceptions in queries

[ https://issues.apache.org/jira/browse/SPARK-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17011: Fix Version/s: 2.0.1 > Support testing exceptions in queries >

[jira] [Updated] (SPARK-17015) group-by-ordinal and order-by-ordinal test cases

[ https://issues.apache.org/jira/browse/SPARK-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17015: Summary: group-by-ordinal and order-by-ordinal test cases (was: group-by-ordinal and

[jira] [Updated] (SPARK-17007) Move test data files into a test-data folder

[ https://issues.apache.org/jira/browse/SPARK-17007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17007: Fix Version/s: 2.0.1 > Move test data files into a test-data folder >

[jira] [Commented] (SPARK-17014) arithmetic.sql

[ https://issues.apache.org/jira/browse/SPARK-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416896#comment-15416896 ] Sean Owen commented on SPARK-17014: --- This seems to be missing a description [~petermaxlee] -- was it an

[jira] [Updated] (SPARK-17006) WithColumn Performance Degrades with Number of Invocations

[ https://issues.apache.org/jira/browse/SPARK-17006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17006: -- Issue Type: Improvement (was: Bug) > WithColumn Performance Degrades with Number of Invocations >

[jira] [Updated] (SPARK-17008) Normalize query results using sorting

[ https://issues.apache.org/jira/browse/SPARK-17008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17008: Fix Version/s: 2.0.1 > Normalize query results using sorting >

[jira] [Updated] (SPARK-17009) Use a new SparkSession for each test case

[ https://issues.apache.org/jira/browse/SPARK-17009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17009: Fix Version/s: 2.0.1 > Use a new SparkSession for each test case >

[jira] [Created] (SPARK-17019) Expose off-heap memory usage in various places

Saisai Shao created SPARK-17019: --- Summary: Expose off-heap memory usage in various places Key: SPARK-17019 URL: https://issues.apache.org/jira/browse/SPARK-17019 Project: Spark Issue Type:

[jira] [Updated] (SPARK-16866) Basic infrastructure for file-based SQL end-to-end tests

[ https://issues.apache.org/jira/browse/SPARK-16866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16866: Fix Version/s: 2.0.1 > Basic infrastructure for file-based SQL end-to-end tests >

[jira] [Resolved] (SPARK-17015) group-by-ordinal and order-by-ordinal

[ https://issues.apache.org/jira/browse/SPARK-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17015. - Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.1.0 > group-by-ordinal

[jira] [Assigned] (SPARK-17013) negative numeric literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17013: Assignee: Apache Spark > negative numeric literal parsing >

[jira] [Commented] (SPARK-17013) negative numeric literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416884#comment-15416884 ] Apache Spark commented on SPARK-17013: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17013) negative numeric literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17013: Assignee: (was: Apache Spark) > negative numeric literal parsing >

[jira] [Commented] (SPARK-17013) negative numeric literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416871#comment-15416871 ] Peter Lee commented on SPARK-17013: --- I have a fix for this, but it would be easier to review after

[jira] [Commented] (SPARK-17018) literals.sql for testing literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416720#comment-15416720 ] Apache Spark commented on SPARK-17018: -- User 'petermaxlee' has created a pull request for this

[jira] [Assigned] (SPARK-17018) literals.sql for testing literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17018: Assignee: (was: Apache Spark) > literals.sql for testing literal parsing >

[jira] [Updated] (SPARK-17017) Add a chiSquare Selector based on False Positive Rate (FPR) test

[ https://issues.apache.org/jira/browse/SPARK-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peng Meng updated SPARK-17017: -- Target Version/s: (was: 2.1.0) > Add a chiSquare Selector based on False Positive Rate (FPR) test >

[jira] [Assigned] (SPARK-17018) literals.sql for testing literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17018: Assignee: Apache Spark > literals.sql for testing literal parsing >

[jira] [Updated] (SPARK-17017) Add a chiSquare Selector based on False Positive Rate (FPR) test

[ https://issues.apache.org/jira/browse/SPARK-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peng Meng updated SPARK-17017: -- Affects Version/s: (was: 2.0.0) > Add a chiSquare Selector based on False Positive Rate (FPR) test

[jira] [Created] (SPARK-17018) literals.sql for testing literal parsing

Peter Lee created SPARK-17018: - Summary: literals.sql for testing literal parsing Key: SPARK-17018 URL: https://issues.apache.org/jira/browse/SPARK-17018 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-17017) Add a chiSquare Selector based on False Positive Rate (FPR) test

[ https://issues.apache.org/jira/browse/SPARK-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17017: Assignee: (was: Apache Spark) > Add a chiSquare Selector based on False Positive Rate

[jira] [Commented] (SPARK-17017) Add a chiSquare Selector based on False Positive Rate (FPR) test

[ https://issues.apache.org/jira/browse/SPARK-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416695#comment-15416695 ] Apache Spark commented on SPARK-17017: -- User 'mpjlu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17017) Add a chiSquare Selector based on False Positive Rate (FPR) test

[ https://issues.apache.org/jira/browse/SPARK-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17017: Assignee: Apache Spark > Add a chiSquare Selector based on False Positive Rate (FPR) test

[jira] [Created] (SPARK-17017) Add a chiSquare Selector based on False Positive Rate (FPR) test

Peng Meng created SPARK-17017: - Summary: Add a chiSquare Selector based on False Positive Rate (FPR) test Key: SPARK-17017 URL: https://issues.apache.org/jira/browse/SPARK-17017 Project: Spark

[jira] [Commented] (SPARK-14887) Generated SpecificUnsafeProjection Exceeds JVM Code Size Limits

[ https://issues.apache.org/jira/browse/SPARK-14887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416680#comment-15416680 ] Hayri Volkan Agun commented on SPARK-14887: --- Hi, It is OneVsRest classifier on large number of

[jira] [Assigned] (SPARK-12370) Documentation should link to examples from its own release version

[ https://issues.apache.org/jira/browse/SPARK-12370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12370: Assignee: (was: Apache Spark) > Documentation should link to examples from its own

[jira] [Commented] (SPARK-12370) Documentation should link to examples from its own release version

[ https://issues.apache.org/jira/browse/SPARK-12370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416654#comment-15416654 ] Apache Spark commented on SPARK-12370: -- User 'jagadeesanas2' has created a pull request for this

[jira] [Assigned] (SPARK-12370) Documentation should link to examples from its own release version

[ https://issues.apache.org/jira/browse/SPARK-12370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12370: Assignee: Apache Spark > Documentation should link to examples from its own release

[jira] [Assigned] (SPARK-17015) group-by-ordinal and order-by-ordinal

[ https://issues.apache.org/jira/browse/SPARK-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17015: Assignee: (was: Apache Spark) > group-by-ordinal and order-by-ordinal >

[jira] [Assigned] (SPARK-17015) group-by-ordinal and order-by-ordinal

[ https://issues.apache.org/jira/browse/SPARK-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17015: Assignee: Apache Spark > group-by-ordinal and order-by-ordinal >

[jira] [Commented] (SPARK-17015) group-by-ordinal and order-by-ordinal

[ https://issues.apache.org/jira/browse/SPARK-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416637#comment-15416637 ] Apache Spark commented on SPARK-17015: -- User 'petermaxlee' has created a pull request for this

[jira] [Resolved] (SPARK-17011) Support testing exceptions in queries

[ https://issues.apache.org/jira/browse/SPARK-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17011. - Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.1.0 > Support testing

[jira] [Assigned] (SPARK-17016) group-by/order-by ordinal should throw AnalysisException instead of UnresolvedException

[ https://issues.apache.org/jira/browse/SPARK-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17016: Assignee: Apache Spark > group-by/order-by ordinal should throw AnalysisException instead

[jira] [Assigned] (SPARK-17016) group-by/order-by ordinal should throw AnalysisException instead of UnresolvedException

[ https://issues.apache.org/jira/browse/SPARK-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17016: Assignee: (was: Apache Spark) > group-by/order-by ordinal should throw

[jira] [Commented] (SPARK-17016) group-by/order-by ordinal should throw AnalysisException instead of UnresolvedException

[ https://issues.apache.org/jira/browse/SPARK-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416628#comment-15416628 ] Apache Spark commented on SPARK-17016: -- User 'petermaxlee' has created a pull request for this

[jira] [Created] (SPARK-17016) group-by/order-by ordinal should throw AnalysisException instead of UnresolvedException

Peter Lee created SPARK-17016: - Summary: group-by/order-by ordinal should throw AnalysisException instead of UnresolvedException Key: SPARK-17016 URL: https://issues.apache.org/jira/browse/SPARK-17016

<    1   2