[
https://issues.apache.org/jira/browse/FLINK-6085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945991#comment-15945991
]
Chen Qin commented on FLINK-6085:
---------------------------------
To unlock scenarios like blocking rpc call or async callback, I am currently
thinking of way to get rid of using distributed queue by connecting web front
directly to pipeline.
I put a bit more thoughts on this topic, exact once seems really hard to
achieve through rpc source. Same issue as using web front ingestion to
distributed queue at a matter of fact. Clients can do arbitrary retry within
long time span.
I think what we can do at this point is to assume client will do retry after
connection failure and flink as a micro service maintain at least once
semantics. So the problem simplified to implement a web front source and
feedback loop from sink to source & locate pending connection to response.
What do you think [~till.rohrmann] [~tudandan]
> flink as micro service
> ----------------------
>
> Key: FLINK-6085
> URL: https://issues.apache.org/jira/browse/FLINK-6085
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API, JobManager
> Reporter: Chen Qin
> Priority: Minor
> Attachments: Untitled document.jpg
>
>
> Track discussion around run flink as a micro service, includes but not
> limited to
> - RPC (web service endpoint) source
> as web service endpoint accept RPC call, ingest to the streaming job(only
> one)
> - callback mechanism
> - task assignment should honor deployment group (web tier hosts should be
> isolated from rest of task assignment)
> https://docs.google.com/document/d/1MSsTOz7xUu50dAf_8v3gsQFfJFFy9LKnULdIl26yj0o/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
