[
https://issues.apache.org/jira/browse/BEAM-11023?focusedWorklogId=568572&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-568572
]
ASF GitHub Bot logged work on BEAM-11023:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Mar/21 18:19
Start Date: 18/Mar/21 18:19
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #14275:
URL: https://github.com/apache/beam/pull/14275#issuecomment-802180894
@benWize You probably noticed that the Spark Structured Streaming runner
does not support yet Spark 3 in the code base (that's why it is in a different
path than the common one). What is missing is to migrate a pair of classes to
use the new Spark 3 DataSource V2 APIs. I have not had the time to dig deeper
but in case you feel motivated/interested to tackle this don't hesitate to take
the issue [BEAM-8827](https://issues.apache.org/jira/browse/BEAM-8827) and if
you have more questions don't hesitate to ping me in TheASF slack.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 568572)
Time Spent: 40m (was: 0.5h)
> GroupByKeyTest testLargeKeys100MB and testGroupByKeyWithBadEqualsHashCode are
> failing on Spark Structured Streaming runner
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-11023
> URL: https://issues.apache.org/jira/browse/BEAM-11023
> Project: Beam
> Issue Type: Test
> Components: runner-spark
> Reporter: Ismaël Mejía
> Assignee: Benjamin Gonzalez
> Priority: P3
> Labels: structured-streaming
> Time Spent: 40m
> Remaining Estimate: 0h
>
> While working on BEAM-10969 I noticed that some VR tests of the Structured
> Streaming runner were failing. After further look it seems they have been
> broken for almost two weeks
> [https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/2098/]
> Probably worth to check this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
