[jira] [Assigned] (SPARK-20373) Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20373: Assignee: Apache Spark > Batch queries with 'Dataset/DataFrame.withWatermark()` does not

[jira] [Assigned] (SPARK-20373) Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20373: Assignee: (was: Apache Spark) > Batch queries with

[jira] [Commented] (SPARK-20373) Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000294#comment-16000294 ] Apache Spark commented on SPARK-20373: -- User 'uncleGen' has created a pull request for this issue:

[jira] [Commented] (SPARK-20588) from_utc_timestamp causes bottleneck

2017-05-07 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000290#comment-16000290 ] Takuya Ueshin commented on SPARK-20588: --- I agree with caching per thread for now. >

[jira] [Commented] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-07 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000213#comment-16000213 ] Takuya Ueshin commented on SPARK-12297: --- Issue resolved by pull request 16781

[jira] [Resolved] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-07 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-12297. --- Resolution: Fixed Fix Version/s: 2.3.0 > Add work-around for Parquet/Hive int96

[jira] [Created] (SPARK-20634) result of MLlib KMeans cluster is not stabilize

2017-05-07 Thread Simon.J (JIRA)
Simon.J created SPARK-20634: --- Summary: result of MLlib KMeans cluster is not stabilize Key: SPARK-20634 URL: https://issues.apache.org/jira/browse/SPARK-20634 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-16931) PySpark access to data-frame bucketing api

2017-05-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16931. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 17077

[jira] [Assigned] (SPARK-16931) PySpark access to data-frame bucketing api

2017-05-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-16931: --- Assignee: Maciej Szymkiewicz > PySpark access to data-frame bucketing api >

[jira] [Assigned] (SPARK-17134) Use level 2 BLAS operations in LogisticAggregator

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17134: Assignee: Seth Hendrickson (was: Apache Spark) > Use level 2 BLAS operations in

[jira] [Assigned] (SPARK-17134) Use level 2 BLAS operations in LogisticAggregator

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17134: Assignee: Apache Spark (was: Seth Hendrickson) > Use level 2 BLAS operations in

[jira] [Commented] (SPARK-17134) Use level 2 BLAS operations in LogisticAggregator

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000188#comment-16000188 ] Apache Spark commented on SPARK-17134: -- User 'VinceShieh' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20633) FileFormatWriter wrap the FetchFailedException which breaks job's failover

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20633: Assignee: (was: Apache Spark) > FileFormatWriter wrap the FetchFailedException which

[jira] [Assigned] (SPARK-20633) FileFormatWriter wrap the FetchFailedException which breaks job's failover

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20633: Assignee: Apache Spark > FileFormatWriter wrap the FetchFailedException which breaks

[jira] [Commented] (SPARK-20633) FileFormatWriter wrap the FetchFailedException which breaks job's failover

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000184#comment-16000184 ] Apache Spark commented on SPARK-20633: -- User 'lshmouse' has created a pull request for this issue:

[jira] [Created] (SPARK-20633) FileFormatWriter wrap the FetchFailedException which breaks job's failover

2017-05-07 Thread Liu Shaohui (JIRA)
Liu Shaohui created SPARK-20633: --- Summary: FileFormatWriter wrap the FetchFailedException which breaks job's failover Key: SPARK-20633 URL: https://issues.apache.org/jira/browse/SPARK-20633 Project:

[jira] [Commented] (SPARK-17134) Use level 2 BLAS operations in LogisticAggregator

2017-05-07 Thread Vincent (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000176#comment-16000176 ] Vincent commented on SPARK-17134: - I will submit a PR for this issue soon. > Use level 2 BLAS operations

[jira] [Commented] (SPARK-18350) Support session local timezone

2017-05-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000173#comment-16000173 ] Reynold Xin commented on SPARK-18350: - [~srowen] why was this reopened? > Support session local

[jira] [Commented] (SPARK-19112) add codec for ZStandard

2017-05-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000148#comment-16000148 ] Takeshi Yamamuro commented on SPARK-19112: -- I also put the result here: {code} scaleFactor: 4

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

2017-05-07 Thread Helena Edelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000130#comment-16000130 ] Helena Edelson commented on SPARK-18057: Did that a while ago, my only point is not modifying

[jira] [Assigned] (SPARK-20626) Fix SparkR test warning on Windows with timestamp time zone

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20626: Assignee: Apache Spark > Fix SparkR test warning on Windows with timestamp time zone >

[jira] [Assigned] (SPARK-20626) Fix SparkR test warning on Windows with timestamp time zone

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20626: Assignee: (was: Apache Spark) > Fix SparkR test warning on Windows with timestamp

[jira] [Commented] (SPARK-20626) Fix SparkR test warning on Windows with timestamp time zone

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000110#comment-16000110 ] Apache Spark commented on SPARK-20626: -- User 'felixcheung' has created a pull request for this

[jira] [Updated] (SPARK-18350) Support session local timezone

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18350: -- Fix Version/s: (was: 2.2.0) > Support session local timezone > -- > >

[jira] [Updated] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20372: -- Fix Version/s: (was: 2.2.0) > Word2Vec Continuous Bag Of Words model >

[jira] [Updated] (SPARK-20548) Flaky Test: ReplSuite.newProductSeqEncoder with REPL defined class

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20548: -- Fix Version/s: (was: 2.2.0) > Flaky Test: ReplSuite.newProductSeqEncoder with REPL defined class

[jira] [Updated] (SPARK-18891) Support for specific collection types

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18891: -- Fix Version/s: (was: 2.2.0) > Support for specific collection types >

[jira] [Updated] (SPARK-20617) pyspark.sql filtering fails when using ~isin when there are nulls in column

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20617: -- Fix Version/s: (was: 2.2.0) > pyspark.sql filtering fails when using ~isin when there are nulls in

[jira] [Updated] (SPARK-20626) Fix SparkR test warning on Windows with timestamp time zone

2017-05-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20626: - Summary: Fix SparkR test warning on Windows with timestamp time zone (was: Fix SparkR test on

[jira] [Commented] (SPARK-11834) Ignore thresholds in LogisticRegression and update documentation

2017-05-07 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1625#comment-1625 ] Maciej Szymkiewicz commented on SPARK-11834: Sorry, for that. Wrong ticket in PR. > Ignore

[jira] [Assigned] (SPARK-20631) LogisticRegression._checkThresholdConsistency should use values not Params

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20631: Assignee: Apache Spark > LogisticRegression._checkThresholdConsistency should use values

[jira] [Assigned] (SPARK-20631) LogisticRegression._checkThresholdConsistency should use values not Params

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20631: Assignee: (was: Apache Spark) > LogisticRegression._checkThresholdConsistency should

[jira] [Commented] (SPARK-20631) LogisticRegression._checkThresholdConsistency should use values not Params

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1623#comment-1623 ] Apache Spark commented on SPARK-20631: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-20631) LogisticRegression._checkThresholdConsistency should use values not Params

2017-05-07 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-20631: --- Description: {{_checkThresholdConsistency}} incorrectly uses {{getParam}} in attempt

[jira] [Commented] (SPARK-11834) Ignore thresholds in LogisticRegression and update documentation

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1612#comment-1612 ] Apache Spark commented on SPARK-11834: -- User 'zero323' has created a pull request for this issue:

[jira] [Created] (SPARK-20632) Allow 'Column.getItem()' API to accept Vector columns

2017-05-07 Thread Kevin Ushey (JIRA)
Kevin Ushey created SPARK-20632: --- Summary: Allow 'Column.getItem()' API to accept Vector columns Key: SPARK-20632 URL: https://issues.apache.org/jira/browse/SPARK-20632 Project: Spark Issue

[jira] [Created] (SPARK-20631) LogisticRegression._checkThresholdConsistency should use values not Params

2017-05-07 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20631: -- Summary: LogisticRegression._checkThresholdConsistency should use values not Params Key: SPARK-20631 URL: https://issues.apache.org/jira/browse/SPARK-20631

[jira] [Updated] (SPARK-20630) Thread Dump link available in Executors tab irrespective of spark.ui.threadDumpsEnabled

2017-05-07 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-20630: Description: Irrespective of {{spark.ui.threadDumpsEnabled}} property web UI's Executors

[jira] [Updated] (SPARK-20630) Thread Dump link available in Executors tab irrespective of spark.ui.threadDumpsEnabled

2017-05-07 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-20630: Attachment: spark-webui-executors-threadDump.png > Thread Dump link available in Executors

[jira] [Created] (SPARK-20630) Thread Dump link available in Executors tab irrespective of spark.ui.threadDumpsEnabled

2017-05-07 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-20630: --- Summary: Thread Dump link available in Executors tab irrespective of spark.ui.threadDumpsEnabled Key: SPARK-20630 URL: https://issues.apache.org/jira/browse/SPARK-20630

[jira] [Updated] (SPARK-20617) pyspark.sql filtering fails when using ~isin when there are nulls in column

2017-05-07 Thread Ed Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Lee updated SPARK-20617: --- Description: Hello encountered a filtering bug using 'isin' in pyspark sql on version 2.2.0, Ubuntu 16.04.

[jira] [Updated] (SPARK-20617) pyspark.sql filtering fails when using ~isin when there are nulls in column

2017-05-07 Thread Ed Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Lee updated SPARK-20617: --- Summary: pyspark.sql filtering fails when using ~isin when there are nulls in column (was: pyspark.sql,

[jira] [Updated] (SPARK-20617) pyspark.sql, filtering with ~isin missing rows

2017-05-07 Thread Ed Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Lee updated SPARK-20617: --- Environment: Ubuntu Xenial 16.04, Python 3.5 (was: Ubuntu Xenial 16.04) > pyspark.sql, filtering with

[jira] [Updated] (SPARK-20617) pyspark.sql, filtering with ~isin missing rows

2017-05-07 Thread Ed Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Lee updated SPARK-20617: --- Fix Version/s: 2.2.0 Description: Hello encountered a filtering bug using 'isin' in pyspark sql on

[jira] [Commented] (SPARK-19900) [Standalone] Master registers application again when driver relaunched

2017-05-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999751#comment-15999751 ] Apache Spark commented on SPARK-19900: -- User 'liyichao' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-19900) [Standalone] Master registers application again when driver relaunched

2017-05-07 Thread lyc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999741#comment-15999741 ] lyc edited comment on SPARK-19900 at 5/7/17 9:19 AM: - I can successfully reproduce in

[jira] [Comment Edited] (SPARK-19900) [Standalone] Master registers application again when driver relaunched

2017-05-07 Thread lyc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999741#comment-15999741 ] lyc edited comment on SPARK-19900 at 5/7/17 9:18 AM: - I can successfully reproduce in

[jira] [Assigned] (SPARK-7481) Add spark-hadoop-cloud module to pull in object store support

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-7481: Assignee: Steve Loughran > Add spark-hadoop-cloud module to pull in object store support >

[jira] [Resolved] (SPARK-7481) Add spark-hadoop-cloud module to pull in object store support

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7481. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 17834

[jira] [Assigned] (SPARK-20484) Add documentation to ALS code

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20484: - Assignee: Daniel Li Priority: Minor (was: Trivial) > Add documentation to ALS code >

[jira] [Resolved] (SPARK-20484) Add documentation to ALS code

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20484. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 17793

[jira] [Assigned] (SPARK-20518) Supplement the new blockidsuite unit tests

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20518: - Assignee: caoxuewen Priority: Minor (was: Major) > Supplement the new blockidsuite unit

[jira] [Resolved] (SPARK-20518) Supplement the new blockidsuite unit tests

2017-05-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20518. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 17794

[jira] [Commented] (SPARK-19900) [Standalone] Master registers application again when driver relaunched

2017-05-07 Thread lyc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999741#comment-15999741 ] lyc commented on SPARK-19900: - I can successfully reproduce in spark master ( commit 63d90e7d, and spark