[
https://issues.apache.org/jira/browse/APEXCORE-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Weise resolved APEXCORE-608.
---
Resolution: Fixed
Fix Version/s: 3.6.0
> Streaming Containers use stale RPC proxy
[
https://issues.apache.org/jira/browse/APEXCORE-608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873382#comment-15873382
]
ASF GitHub Bot commented on APEXCORE-608:
-
Github user asfgit closed the pull request at:
Github user asfgit closed the pull request at:
https://github.com/apache/apex-core/pull/448
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user sanjaypujare opened a pull request:
https://github.com/apache/apex-core/pull/474
APEXCORE-624 decrement unallocated containers and released containers so
exit condition for shutdown check is satisfied.
@vrozov pls review and merge as appropriate. I have also removed
[
https://issues.apache.org/jira/browse/APEXCORE-624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873330#comment-15873330
]
ASF GitHub Bot commented on APEXCORE-624:
-
GitHub user sanjaypujare opened a pull request:
Bhupesh,
That is true, but in reality watermarks do not solve a design problem in
the DAG where data is getting mixed up. All the watermarks do is to convey
"start" and "end" within the stream. The start and end control tuples
should have the physical operator id, + a monotonically increasing
Thomas,
I believe Bhupesh's proposal is to have a monotonically increasing
watermark and filename as extra information. The usage of "file start" may
have caused confusion. I agree, we do not need explicit "file start"
watermark. I am at loss of words, maybe "start "->"end
"; and then a "final-all
Hi Bhupesh,
I think this needs a generic watermark concept that is independent of
source and destination and can be understood by intermediate
transformations. File names don't meet this criteria.
One possible approach is to have a monotonic increasing file sequence
(instead of time, if it is
Amol, agreed. We can address event time based watermarks once file batch is
done.
Regarding, file batch support: by allowing to partition an input (file)
operator, we are implicitly mixing multiple batches. Even if the user does
not do any transformations, we should be able to write the correct
Hi Thomas,
For an input operator which is supposed to generate watermarks for
downstream operators, I can think about the following watermarks that the
operator can emit:
1. Time based watermarks (the high watermark / low watermark)
2. Number of tuple based watermarks (Every n tuples)
3. File
10 matches
Mail list logo
