[
https://issues.apache.org/jira/browse/SPARK-34427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284496#comment-17284496
]
Jungtaek Lim commented on SPARK-34427:
--------------------------------------
This assignee case is quite different from what I've seen committers have been
doing, because these issues are not "new" (there has been existing efforts just
not on right time) and the idea is quite well known so many of contributors can
simply plan in parallel. e.g. In SPARK-34198 you'd realize one contributor in
FB is also working on the solution in parallel.
I don't think we are happy with someone occupies the major feature without even
providing design doc or so. No one knows about the plan - no one knows whether
the effort is started or even in backlog actually. In parallel, someone may
have more progress. Stepping on others toes has been normal in Spark community
and setting assignee never avoids it properly. It just makes an unfair
competition between contributor and committer.
If you want to make clear on the ownership for the major feature, then please
prepare SPIP and raise it on dev@ mailing list. That ensures recognition that
you're making meaningful progress already, and others could help on reviewing.
(Even in that case someone argue with another SPIP, then either collaboration
or competition should happen. I don't think committer can simply preempt.)
Also, I think we should try to find the JIRA issue which did the same or
similar, and leverage the one. There're lots of information and history of
efforts which we can leverage "even" we take the different PR. Once you're
filing a new JIRA issue and let the old one be ignored then the efforts were
lost.
I don't think you could simply raise a PR for SPARK-34427 and ask for review,
as from SPARK-10816 we found there're various ways to implement it, which
requires design doc to make sure the implementation considers these designs as
well and picks up the best one. The implementation should also run the
performance test and ensure it's superior or at least on par. That establishes
the "minimum bar" on the efforts. Before achieving that, consider my voice as
-1 on the proposal. To make the comparison easier I think you should really
continue your work in SPARK-10816, not here.
I'm happy to see some other committer finally found the necessity of the
feature, but also unhappy that resurrection of the existing effort is not
considered "at first" which would save a bunch of time among us. The existing
effort wasn't discarded because of technical issue, that said, the design and
implementation are still valid. That wasn't just put right on time.
> Session window support in SS
> ----------------------------
>
> Key: SPARK-34427
> URL: https://issues.apache.org/jira/browse/SPARK-34427
> Project: Spark
> Issue Type: New Feature
> Components: Structured Streaming
> Affects Versions: 3.2.0
> Reporter: L. C. Hsieh
> Priority: Major
>
> Currently structured streaming supports two kinds of windows: tumbling window
> and sliding window. Another useful window function is session window. Which
> is not supported by SS. We have user requirement to use session window. We'd
> like to have this support in the upstream.
> About session window, there is some info:
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#session-windows.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
