[jira] [Commented] (SPARK-28556) Error should also be sent to QueryExecutionListener.onFailure

2020-01-28 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025655#comment-17025655 ] Shixiong Zhu commented on SPARK-28556: -- This is not deprecating the API. We just fixed an issue in

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

2020-01-28 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025651#comment-17025651 ] Maxim Gekk commented on SPARK-30668: > This is not mentioned in the migration guide. It is

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

2020-01-28 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025644#comment-17025644 ] Xiao Li commented on SPARK-30668: - Can we let users choose different parsing mechanisms between

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

2020-01-28 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025643#comment-17025643 ] Xiao Li commented on SPARK-30668: - This will make the migration very painful. This is not mentioned in

[jira] [Comment Edited] (SPARK-21823) ALTER TABLE table statements such as RENAME and CHANGE columns should raise error if there are any dependent constraints.

2020-01-28 Thread sakshi chourasia (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025611#comment-17025611 ] sakshi chourasia edited comment on SPARK-21823 at 1/29/20 6:52 AM: --- Hi

[jira] [Commented] (SPARK-28556) Error should also be sent to QueryExecutionListener.onFailure

2020-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025639#comment-17025639 ] Dongjoon Hyun commented on SPARK-28556: --- [~smilegator]. How do we need to handle this? Do you want

[jira] [Commented] (SPARK-3847) Enum.hashCode is only consistent within the same JVM

2020-01-28 Thread Kaspar Fischer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025631#comment-17025631 ] Kaspar Fischer commented on SPARK-3847: --- This issue is still present in Spark 2.4.0. The PR

[jira] [Commented] (SPARK-21823) ALTER TABLE table statements such as RENAME and CHANGE columns should raise error if there are any dependent constraints.

2020-01-28 Thread sakshi chourasia (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025611#comment-17025611 ] sakshi chourasia commented on SPARK-21823: -- I work with spark sql. Need this command to alter

[jira] [Updated] (SPARK-30619) org.slf4j.Logger and org.apache.commons.collections classes not built as part of hadoop-provided profile

2020-01-28 Thread Abhishek Rao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Rao updated SPARK-30619: - Issue Type: Bug (was: Question) > org.slf4j.Logger and org.apache.commons.collections classes

[jira] [Commented] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025574#comment-17025574 ] Dongjoon Hyun commented on SPARK-30658: --- Hi, [~tdas]. Can we have `2.4.5` at `Target Version`,

[jira] [Commented] (SPARK-30657) Streaming limit after streaming dropDuplicates can throw error

2020-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025575#comment-17025575 ] Dongjoon Hyun commented on SPARK-30657: --- Hi, [~tdas]. Can we have `2.4.5` at `Target Version`,

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

2020-01-28 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025549#comment-17025549 ] Maxim Gekk commented on SPARK-30668: Date/timestamp parsing is based on Java 8 DateTimeFormat in

[jira] [Updated] (SPARK-30637) upgrade testthat on jenkins workers to 2.0.0

2020-01-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30637: - Parent: (was: SPARK-23435) Issue Type: Test (was: Sub-task) > upgrade testthat on

[jira] [Updated] (SPARK-23435) R tests should support latest testthat

2020-01-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23435: - Fix Version/s: 2.4.5 > R tests should support latest testthat >

[jira] [Created] (SPARK-30669) Introduce AdmissionControl API to Structured Streaming

2020-01-28 Thread Burak Yavuz (Jira)
Burak Yavuz created SPARK-30669: --- Summary: Introduce AdmissionControl API to Structured Streaming Key: SPARK-30669 URL: https://issues.apache.org/jira/browse/SPARK-30669 Project: Spark Issue

[jira] [Resolved] (SPARK-23435) R tests should support latest testthat

2020-01-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23435. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27359

[jira] [Commented] (SPARK-27298) Dataset except operation gives different results(dataset count) on Spark 2.3.0 Windows and Spark 2.3.0 Linux environment

2020-01-28 Thread Sunitha Kambhampati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025522#comment-17025522 ] Sunitha Kambhampati commented on SPARK-27298: - 1) Would it be possible to run your program

[jira] [Resolved] (SPARK-30481) Integrate event log compactor into Spark History Server

2020-01-28 Thread Marcelo Masiero Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30481. Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution:

[jira] [Updated] (SPARK-30663) Remove 1.x testthat switch once Jenkins version is updated to 2.x

2020-01-28 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30663: --- Issue Type: Planned Work (was: Bug) > Remove 1.x testthat switch once Jenkins

[jira] [Assigned] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

2020-01-28 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-30668: --- Assignee: (was: Xiao Li) > to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using

[jira] [Assigned] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

2020-01-28 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-30668: --- Assignee: Xiao Li > to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern >

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

2020-01-28 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025467#comment-17025467 ] Xiao Li commented on SPARK-30668: - cc [~maxgekk] > to_timestamp failed to parse

[jira] [Updated] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern

2020-01-28 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30668: Summary: to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern (was: to_timestamp

[jira] [Updated] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

2020-01-28 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30668: Summary: to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern

[jira] [Updated] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800

2020-01-28 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30668: Description: {code:java} SELECT to_timestamp("2020-01-27T20:06:11.847-0800",

[jira] [Created] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800

2020-01-28 Thread Xiao Li (Jira)
Xiao Li created SPARK-30668: --- Summary: to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 Key: SPARK-30668 URL: https://issues.apache.org/jira/browse/SPARK-30668 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30656) Support the "minPartitions" option in Kafka batch source and streaming source v1

2020-01-28 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-30656: - Issue Type: Improvement (was: Bug) > Support the "minPartitions" option in Kafka batch source

[jira] [Commented] (SPARK-30667) Support simple all gather in barrier task context

2020-01-28 Thread Sarth Frey (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025390#comment-17025390 ] Sarth Frey commented on SPARK-30667: I will work on this > Support simple all gather in barrier

[jira] [Updated] (SPARK-30667) Support simple all gather in barrier task context

2020-01-28 Thread Xiangrui Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-30667: -- Description: Currently we offer task.barrier() to coordinate tasks in barrier mode. Tasks

[jira] [Created] (SPARK-30667) Support simple all gather in barrier task context

2020-01-28 Thread Xiangrui Meng (Jira)
Xiangrui Meng created SPARK-30667: - Summary: Support simple all gather in barrier task context Key: SPARK-30667 URL: https://issues.apache.org/jira/browse/SPARK-30667 Project: Spark Issue

[jira] [Updated] (SPARK-30666) Reliable single-stage accumulators

2020-01-28 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30666: -- Component/s: (was: SQL) Spark Core > Reliable single-stage accumulators

[jira] [Updated] (SPARK-30666) Reliable single-stage accumulators

2020-01-28 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30666: -- Description: This proposes a pragmatic improvement to allow for reliable single-stage

[jira] [Created] (SPARK-30666) Reliable single-stage accumulators

2020-01-28 Thread Enrico Minack (Jira)
Enrico Minack created SPARK-30666: - Summary: Reliable single-stage accumulators Key: SPARK-30666 URL: https://issues.apache.org/jira/browse/SPARK-30666 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-12378) CREATE EXTERNAL TABLE AS SELECT EXPORT AWS S3 ERROR

2020-01-28 Thread Ori Popowski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025383#comment-17025383 ] Ori Popowski commented on SPARK-12378: -- [~arun6445] there's a workaround here:

[jira] [Created] (SPARK-30665) Remove Pandoc dependency in PySpark setup.py

2020-01-28 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-30665: Summary: Remove Pandoc dependency in PySpark setup.py Key: SPARK-30665 URL: https://issues.apache.org/jira/browse/SPARK-30665 Project: Spark Issue

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-01-28 Thread nabacg (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025254#comment-17025254 ] nabacg commented on SPARK-12312: My suggestion was for people who need a working solution now and can't

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-01-28 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025241#comment-17025241 ] Gabor Somogyi commented on SPARK-12312: --- [~nabacg] thanks, the approach is more or less clear but

[jira] [Updated] (SPARK-30664) Add more metrics to the all stages page

2020-01-28 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30664: -- Attachment: (was: Show Additional Metrics.png) > Add more metrics to the all stages page

[jira] [Updated] (SPARK-30664) Add more metrics to the all stages page

2020-01-28 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30664: -- Description: The web UI page for individual stages has many useful metrics to diagnose

[jira] [Updated] (SPARK-30664) Add more metrics to the all stages page

2020-01-28 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30664: -- Attachment: image-2020-01-28-16-15-20-258.png > Add more metrics to the all stages page >

[jira] [Updated] (SPARK-30664) Add more metrics to the all stages page

2020-01-28 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30664: -- Attachment: image-2020-01-28-16-13-36-174.png > Add more metrics to the all stages page >

[jira] [Updated] (SPARK-30664) Add more metrics to the all stages page

2020-01-28 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30664: -- Attachment: image-2020-01-28-16-12-49-807.png > Add more metrics to the all stages page >

[jira] [Updated] (SPARK-30664) Add more metrics to the all stages page

2020-01-28 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30664: -- Attachment: Show Additional Metrics.png > Add more metrics to the all stages page >

[jira] [Created] (SPARK-30664) Add more metrics to the all stages page

2020-01-28 Thread Enrico Minack (Jira)
Enrico Minack created SPARK-30664: - Summary: Add more metrics to the all stages page Key: SPARK-30664 URL: https://issues.apache.org/jira/browse/SPARK-30664 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-01-28 Thread nabacg (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025176#comment-17025176 ] nabacg commented on SPARK-12312: I originally got around this problem using the approach sketched in

[jira] [Updated] (SPARK-30663) Remove 1.x testthat switch once Jenkins version is updated to 2.x

2020-01-28 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30663: --- Description: As part of SPARK-23435 proposal we include {{testthat}} 1.x

[jira] [Created] (SPARK-30663) Remove 1.x testthat switch once Jenkins version is updated to 2.x

2020-01-28 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30663: -- Summary: Remove 1.x testthat switch once Jenkins version is updated to 2.x Key: SPARK-30663 URL: https://issues.apache.org/jira/browse/SPARK-30663

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-01-28 Thread John Lonergan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025145#comment-17025145 ] John Lonergan commented on SPARK-12312: --- Re documentation - it really would be helpful and avoid

[jira] [Created] (SPARK-30662) ALS/MLP extend HasBlockSize

2020-01-28 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30662: Summary: ALS/MLP extend HasBlockSize Key: SPARK-30662 URL: https://issues.apache.org/jira/browse/SPARK-30662 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-30661) KMeans blockify input vectors

2020-01-28 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-30661: Assignee: zhengruifeng > KMeans blockify input vectors > - >

[jira] [Created] (SPARK-30661) KMeans blockify input vectors

2020-01-28 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30661: Summary: KMeans blockify input vectors Key: SPARK-30661 URL: https://issues.apache.org/jira/browse/SPARK-30661 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-30660) LinearRegression blockify input vectors

2020-01-28 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30660: Summary: LinearRegression blockify input vectors Key: SPARK-30660 URL: https://issues.apache.org/jira/browse/SPARK-30660 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-28067) Incorrect results in decimal aggregation with whole-stage code gen enabled

2020-01-28 Thread Javier Fuentes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025117#comment-17025117 ] Javier Fuentes commented on SPARK-28067: The expected result here should be an 

[jira] [Created] (SPARK-30659) LogisticRegression blockify input vectors

2020-01-28 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30659: Summary: LogisticRegression blockify input vectors Key: SPARK-30659 URL: https://issues.apache.org/jira/browse/SPARK-30659 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13

2020-01-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025099#comment-17025099 ] Sean R. Owen commented on SPARK-29292: -- I do, I have a large (190 files) PR that fixes all of these

[jira] [Resolved] (SPARK-30642) LinearSVC blockify input vectors

2020-01-28 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-30642. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27360

[jira] [Assigned] (SPARK-30642) LinearSVC blockify input vectors

2020-01-28 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-30642: Assignee: zhengruifeng > LinearSVC blockify input vectors >

[jira] [Commented] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

2020-01-28 Thread Sivakumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024981#comment-17024981 ] Sivakumar commented on SPARK-30542: --- Sure, Thanks Hyukjin > Two Spark structured streaming jobs

[jira] [Created] (SPARK-30658) Limit after on streaming dataframe before streaming agg returns wrong results

2020-01-28 Thread Tathagata Das (Jira)
Tathagata Das created SPARK-30658: - Summary: Limit after on streaming dataframe before streaming agg returns wrong results Key: SPARK-30658 URL: https://issues.apache.org/jira/browse/SPARK-30658

[jira] [Created] (SPARK-30657) Streaming limit after streaming dropDuplicates can throw error

2020-01-28 Thread Tathagata Das (Jira)
Tathagata Das created SPARK-30657: - Summary: Streaming limit after streaming dropDuplicates can throw error Key: SPARK-30657 URL: https://issues.apache.org/jira/browse/SPARK-30657 Project: Spark

[jira] [Commented] (SPARK-23829) spark-sql-kafka source in spark 2.3 causes reading stream failure frequently

2020-01-28 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024973#comment-17024973 ] Gabor Somogyi commented on SPARK-23829: --- Cannot fetch record means Spark initiated the fetch but