[jira] [Updated] (SPARK-19994) Wrong outputOrdering for right/full outer smj

2017-03-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19994: Labels: correctness (was: ) > Wrong outputOrdering for right/full outer smj >

[jira] [Updated] (SPARK-19994) Wrong outputOrdering for right/full outer smj

2017-03-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19994: Target Version/s: 2.1.1, 2.2.0 > Wrong outputOrdering for right/full outer smj >

[jira] [Updated] (SPARK-19994) Wrong outputOrdering for right/full outer smj

2017-03-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19994: Affects Version/s: 2.0.2 2.1.0 > Wrong outputOrdering for right/full outer smj >

[jira] [Commented] (SPARK-15790) Audit @Since annotations in ML

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931580#comment-15931580 ] Apache Spark commented on SPARK-15790: -- User 'ehsun7b' has created a pull request for this issue:

[jira] [Commented] (SPARK-18165) Kinesis support in Structured Streaming

2017-03-18 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931574#comment-15931574 ] Gaurav Shah commented on SPARK-18165: - anything that I can do to help for this feature ? > Kinesis

[jira] [Commented] (SPARK-19990) Flaky test: org.apache.spark.sql.hive.execution.HiveCatalogedDDLSuite: create temporary view using

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931571#comment-15931571 ] Apache Spark commented on SPARK-19990: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18579) spark-csv strips whitespace (pyspark)

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18579: Assignee: Apache Spark > spark-csv strips whitespace (pyspark) >

[jira] [Assigned] (SPARK-18579) spark-csv strips whitespace (pyspark)

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18579: Assignee: (was: Apache Spark) > spark-csv strips whitespace (pyspark) >

[jira] [Commented] (SPARK-18579) spark-csv strips whitespace (pyspark)

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931567#comment-15931567 ] Apache Spark commented on SPARK-18579: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-19019) PySpark does not work with Python 3.6.0

2017-03-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931566#comment-15931566 ] Hyukjin Kwon commented on SPARK-19019: -- Let me try to make a PR to backport this if this is

[jira] [Assigned] (SPARK-19994) Wrong outputOrdering for right/full outer smj

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19994: Assignee: Apache Spark > Wrong outputOrdering for right/full outer smj >

[jira] [Commented] (SPARK-19994) Wrong outputOrdering for right/full outer smj

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931546#comment-15931546 ] Apache Spark commented on SPARK-19994: -- User 'wzhfy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19994) Wrong outputOrdering for right/full outer smj

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19994: Assignee: (was: Apache Spark) > Wrong outputOrdering for right/full outer smj >

[jira] [Comment Edited] (SPARK-19019) PySpark does not work with Python 3.6.0

2017-03-18 Thread Henry Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931527#comment-15931527 ] Henry Zhang edited comment on SPARK-19019 at 3/19/17 2:00 AM: -- Would also be

[jira] [Commented] (SPARK-19019) PySpark does not work with Python 3.6.0

2017-03-18 Thread Henry Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931527#comment-15931527 ] Henry Zhang commented on SPARK-19019: - Would also be interested in the answer to Maciej's question

[jira] [Updated] (SPARK-20014) Optimize mergeSpillsWithFileStream method

2017-03-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-20014: Description: When the individual partition size in a spill is small, mergeSpillsWithTransferTo

[jira] [Assigned] (SPARK-20014) Optimize mergeSpillsWithFileStream method

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20014: Assignee: Apache Spark > Optimize mergeSpillsWithFileStream method >

[jira] [Commented] (SPARK-20014) Optimize mergeSpillsWithFileStream method

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931499#comment-15931499 ] Apache Spark commented on SPARK-20014: -- User 'sitalkedia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20014) Optimize mergeSpillsWithFileStream method

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20014: Assignee: (was: Apache Spark) > Optimize mergeSpillsWithFileStream method >

[jira] [Updated] (SPARK-19237) SparkR package on Windows waiting for a long time when no java is found launching spark-submit

2017-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19237: - Description: When installing SparkR as a R package (install.packages) on Windows, it will check

[jira] [Updated] (SPARK-19237) SparkR package on Windows waiting for a long time when no java is found launching spark-submit

2017-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19237: - Summary: SparkR package on Windows waiting for a long time when no java is found launching

[jira] [Commented] (SPARK-19993) Caching logical plans containing subquery expressions does not work.

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931497#comment-15931497 ] Apache Spark commented on SPARK-19993: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-19993) Caching logical plans containing subquery expressions does not work.

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19993: Assignee: (was: Apache Spark) > Caching logical plans containing subquery expressions

[jira] [Assigned] (SPARK-19993) Caching logical plans containing subquery expressions does not work.

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19993: Assignee: Apache Spark > Caching logical plans containing subquery expressions does not

[jira] [Assigned] (SPARK-19990) Flaky test: org.apache.spark.sql.hive.execution.HiveCatalogedDDLSuite: create temporary view using

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19990: Assignee: Apache Spark > Flaky test:

[jira] [Commented] (SPARK-19990) Flaky test: org.apache.spark.sql.hive.execution.HiveCatalogedDDLSuite: create temporary view using

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931496#comment-15931496 ] Apache Spark commented on SPARK-19990: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19990) Flaky test: org.apache.spark.sql.hive.execution.HiveCatalogedDDLSuite: create temporary view using

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19990: Assignee: (was: Apache Spark) > Flaky test:

[jira] [Commented] (SPARK-7146) Should ML sharedParams be a public API?

2017-03-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931480#comment-15931480 ] François Garillot commented on SPARK-7146: -- +1 to the Java interface & Scala traits in the

[jira] [Updated] (SPARK-19573) Make NaN/null handling consistent in approxQuantile

2017-03-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19573: Target Version/s: 2.2.0 > Make NaN/null handling consistent in approxQuantile >

[jira] [Created] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide

2017-03-18 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20015: Summary: Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide Key: SPARK-20015 URL: https://issues.apache.org/jira/browse/SPARK-20015

[jira] [Resolved] (SPARK-19654) Structured Streaming API for R

2017-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-19654. -- Resolution: Fixed Fix Version/s: 2.2.0 Target Version/s: 2.2.0 > Structured

[jira] [Created] (SPARK-20014) Optimize mergeSpillsWithFileStream method

2017-03-18 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-20014: --- Summary: Optimize mergeSpillsWithFileStream method Key: SPARK-20014 URL: https://issues.apache.org/jira/browse/SPARK-20014 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19528) external shuffle service would close while still have request from executor when dynamic allocation is enabled

2017-03-18 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-19528: --- Description: when dynamic allocation is enabled, the external shuffle service is used for maintain

[jira] [Assigned] (SPARK-19970) Table owner should be USER instead of PRINCIPAL in kerberized clusters

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19970: Assignee: Apache Spark > Table owner should be USER instead of PRINCIPAL in kerberized

[jira] [Assigned] (SPARK-19970) Table owner should be USER instead of PRINCIPAL in kerberized clusters

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19970: Assignee: (was: Apache Spark) > Table owner should be USER instead of PRINCIPAL in

[jira] [Commented] (SPARK-18910) Can't use UDF that jar file in hdfs

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931350#comment-15931350 ] Apache Spark commented on SPARK-18910: -- User 'weiqingy' has created a pull request for this issue:

[jira] [Commented] (SPARK-12868) ADD JAR via sparkSQL JDBC will fail when using a HDFS URL

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931351#comment-15931351 ] Apache Spark commented on SPARK-12868: -- User 'weiqingy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19998) BlockRDD block not found Exception add RDD id info

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19998: Assignee: (was: Apache Spark) > BlockRDD block not found Exception add RDD id info >

[jira] [Commented] (SPARK-19998) BlockRDD block not found Exception add RDD id info

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931325#comment-15931325 ] Apache Spark commented on SPARK-19998: -- User 'jianran' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19998) BlockRDD block not found Exception add RDD id info

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19998: Assignee: Apache Spark > BlockRDD block not found Exception add RDD id info >

[jira] [Updated] (SPARK-19237) SparkR package waiting for a long time when no java is found launching spark-submit

2017-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19237: - Summary: SparkR package waiting for a long time when no java is found launching spark-submit

[jira] [Updated] (SPARK-19237) SparkR package waiting for a long time when no java is found launching spark-submit

2017-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19237: - Component/s: Spark Core > SparkR package waiting for a long time when no java is found launching

[jira] [Commented] (SPARK-20007) Make SparkR apply() functions robust to workers that return empty data.frame

2017-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931310#comment-15931310 ] Felix Cheung commented on SPARK-20007: -- +1 - also I've been meaning to add checks for data type

[jira] [Commented] (SPARK-19019) PySpark does not work with Python 3.6.0

2017-03-18 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931288#comment-15931288 ] Maciej Szymkiewicz commented on SPARK-19019: [~davies] Could it be backported to 1.6 and 2.0?

[jira] [Assigned] (SPARK-20013) merge renameTable to alterTable in ExternalCatalog

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20013: Assignee: Apache Spark > merge renameTable to alterTable in ExternalCatalog >

[jira] [Commented] (SPARK-20013) merge renameTable to alterTable in ExternalCatalog

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931282#comment-15931282 ] Apache Spark commented on SPARK-20013: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20013) merge renameTable to alterTable in ExternalCatalog

2017-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20013: Assignee: (was: Apache Spark) > merge renameTable to alterTable in ExternalCatalog >

[jira] [Commented] (SPARK-19977) Scheduler Delay (in UI Advanced Metrics) for a task gradually increases from 5 ms to 30 seconds in Spark Streaming application

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931280#comment-15931280 ] Sean Owen commented on SPARK-19977: --- I don't think this enough information to act on this. If you can

[jira] [Updated] (SPARK-19519) Groupby for multiple columns not working

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19519: -- Priority: Major (was: Blocker) Yes, don't set Blocker. I don't think this is clear. If you can

[jira] [Commented] (SPARK-19934) code comments are not very clearly in BlackListTracker.scala

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931278#comment-15931278 ] Sean Owen commented on SPARK-19934: --- I am not sure this improves the comment. Yes, enough executors

[jira] [Resolved] (SPARK-19930) table operator as an alternative to saveAsTable

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19930. --- Resolution: Not A Problem > table operator as an alternative to saveAsTable >

[jira] [Commented] (SPARK-20012) spark.read.csv schemas effectively ignore headers

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931277#comment-15931277 ] Sean Owen commented on SPARK-20012: --- I don't understand the problem from this description. The headers

[jira] [Updated] (SPARK-20013) merge renameTable to alterTable in ExternalCatalog

2017-03-18 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-20013: - Description: Currently when we create / rename a managed table, we should get the defaultTablePath for

[jira] [Updated] (SPARK-20008) hiveContext.emptyDataFrame.except(hiveContext.emptyDataFrame).count() returns 1

2017-03-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-20008: - Affects Version/s: 2.2.0 > hiveContext.emptyDataFrame.except(hiveContext.emptyDataFrame).count()

[jira] [Commented] (SPARK-20008) hiveContext.emptyDataFrame.except(hiveContext.emptyDataFrame).count() returns 1

2017-03-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931225#comment-15931225 ] Hyukjin Kwon commented on SPARK-20008: -- I could reproduce this in the current master with {code}

[jira] [Updated] (SPARK-20008) hiveContext.emptyDataFrame.except(hiveContext.emptyDataFrame).count() returns 1

2017-03-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-20008: - Component/s: (was: Spark Core) SQL >

[jira] [Commented] (SPARK-20008) hiveContext.emptyDataFrame.except(hiveContext.emptyDataFrame).count() returns 1

2017-03-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931224#comment-15931224 ] Hyukjin Kwon commented on SPARK-20008: -- This was fine in 1.6.3 with {{ExceptExec}} too but this

[jira] [Commented] (SPARK-20008) hiveContext.emptyDataFrame.except(hiveContext.emptyDataFrame).count() returns 1

2017-03-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931221#comment-15931221 ] Hyukjin Kwon commented on SPARK-20008: -- I just took a quick look. {{BroadcastNestedLoopJoin}} looks

[jira] [Comment Edited] (SPARK-20001) Support PythonRunner executing inside a Conda env

2017-03-18 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930922#comment-15930922 ] Jeff Zhang edited comment on SPARK-20001 at 3/18/17 1:08 PM: - Thanks

[jira] [Created] (SPARK-20013) merge renameTable to alterTable in ExternalCatalog

2017-03-18 Thread Song Jun (JIRA)
Song Jun created SPARK-20013: Summary: merge renameTable to alterTable in ExternalCatalog Key: SPARK-20013 URL: https://issues.apache.org/jira/browse/SPARK-20013 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-20012) spark.read.csv schemas effectively ignore headers

2017-03-18 Thread david cottrell (JIRA)
david cottrell created SPARK-20012: -- Summary: spark.read.csv schemas effectively ignore headers Key: SPARK-20012 URL: https://issues.apache.org/jira/browse/SPARK-20012 Project: Spark Issue

[jira] [Commented] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931165#comment-15931165 ] Sean Owen commented on SPARK-19941: --- I'm not sure I agree with that. If the app wants N executors, as

[jira] [Commented] (SPARK-20004) Spark thrift server ovewrites spark.app.name

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931164#comment-15931164 ] Sean Owen commented on SPARK-20004: --- I don't think the user is supposed to be able to change this app

[jira] [Updated] (SPARK-20005) There is no "Newline" in UI in describtion

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20005: -- Priority: Trivial (was: Minor) Yeah, either that cell is set to 'nowrap' or something similar. Either

[jira] [Commented] (SPARK-19999) Test failures in Spark Core due to java.nio.Bits.unaligned()

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931153#comment-15931153 ] Sean Owen commented on SPARK-1: --- Isn't this a JDK bug then? you show it's already fixed in a recent

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931150#comment-15931150 ] Sean Owen commented on SPARK-19644: --- I don't think there is evidence of a memory leak here. It's not

[jira] [Commented] (SPARK-20011) inconsistent terminology in als api docs and tutorial

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931136#comment-15931136 ] Sean Owen commented on SPARK-20011: --- You don't need it assigned, just go ahead > inconsistent

[jira] [Commented] (SPARK-20011) inconsistent terminology in als api docs and tutorial

2017-03-18 Thread chris snow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931132#comment-15931132 ] chris snow commented on SPARK-20011: Ok, thanks. Can you please assign to me? > inconsistent

[jira] [Updated] (SPARK-20011) inconsistent terminology in als api docs and tutorial

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20011: -- Priority: Trivial (was: Minor) Yes, these are the same thing. go ahead. > inconsistent terminology

[jira] [Created] (SPARK-20011) inconsistent terminology in als api docs and tutorial

2017-03-18 Thread chris snow (JIRA)
chris snow created SPARK-20011: -- Summary: inconsistent terminology in als api docs and tutorial Key: SPARK-20011 URL: https://issues.apache.org/jira/browse/SPARK-20011 Project: Spark Issue

[jira] [Commented] (SPARK-18890) Do all task serialization in CoarseGrainedExecutorBackend thread (rather than TaskSchedulerImpl)

2017-03-18 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931115#comment-15931115 ] Guoqiang Li commented on SPARK-18890: - [~kayousterhout] done. > Do all task serialization in

[jira] [Resolved] (SPARK-19840) Disallow creating permanent functions with invalid class names

2017-03-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-19840. --- Resolution: Won't Fix Later, this will be handled better by fixing permanent function

[jira] [Updated] (SPARK-19915) Improve join reorder: Exclude cartesian product candidates to reduce the search space

2017-03-18 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-19915: - Description: We only consider consecutive inner joinable items, thus excluding cartesian

[jira] [Updated] (SPARK-19915) Improve join reorder: Exclude cartesian product candidates to reduce the search space

2017-03-18 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-19915: - Summary: Improve join reorder: Exclude cartesian product candidates to reduce the search space

[jira] [Assigned] (SPARK-19896) toDS throws StackOverflowError if case classes have circular references

2017-03-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19896: --- Assignee: Takeshi Yamamuro > toDS throws StackOverflowError if case classes have circular

[jira] [Resolved] (SPARK-19896) toDS throws StackOverflowError if case classes have circular references

2017-03-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19896. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17318

[jira] [Updated] (SPARK-20010) Sort information is lost after sort merge join

2017-03-18 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20010: - Description: After sort merge join for inner join, now we only keep left key ordering. However,

[jira] [Updated] (SPARK-20010) Sort information is lost after sort merge join

2017-03-18 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20010: - Description: After sort merge join for inner join, now we only keep left key ordering. However,

[jira] [Updated] (SPARK-20010) Sort information is lost after sort merge join

2017-03-18 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20010: - Description: After sort merge join for inner join, now we only keep left key ordering. However,

[jira] [Created] (SPARK-20010) Sort information is lost after sort merge join

2017-03-18 Thread Zhenhua Wang (JIRA)
Zhenhua Wang created SPARK-20010: Summary: Sort information is lost after sort merge join Key: SPARK-20010 URL: https://issues.apache.org/jira/browse/SPARK-20010 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-19915) Improve join reorder: simplify cost evaluation, postpone column pruning, exclude cartesian product

2017-03-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19915: --- Assignee: Zhenhua Wang > Improve join reorder: simplify cost evaluation, postpone column

[jira] [Resolved] (SPARK-19915) Improve join reorder: simplify cost evaluation, postpone column pruning, exclude cartesian product

2017-03-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19915. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17286