[ 
https://issues.apache.org/jira/browse/FLINK-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511359#comment-16511359
 ] 

ASF GitHub Bot commented on FLINK-9560:
---------------------------------------

Github user pnowojski commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6149#discussion_r195138248
  
    --- Diff: 
flink-core/src/main/java/org/apache/flink/core/fs/LimitedConnectionsFileSystem.java
 ---
    @@ -741,6 +804,10 @@ public void write(int b) throws IOException {
                public void write(byte[] b, int off, int len) throws 
IOException {
                        try {
                                originalStream.write(b, off, len);
    +                           if (fs.outputRateLimiter != null){
    --- End diff --
    
    With optional: `rate.ifPresent(limiter -> limiter.acquire(len));`


> RateLimiting for FileSystem
> ---------------------------
>
>                 Key: FLINK-9560
>                 URL: https://issues.apache.org/jira/browse/FLINK-9560
>             Project: Flink
>          Issue Type: Improvement
>          Components: FileSystem
>    Affects Versions: 1.5.0
>            Reporter: Etienne CARRIERE
>            Priority: Major
>
> *Pain*: On our system, we see that during checkpoint , all the bandwidth is 
> take to send the checkpoint to object storage (s3 in our case)
> *Proposal* : After the creation of some limitation on Filesystem (mostly 
> number of connections with the  tickets FLINK-8125/FLINK-8198/FLINK-9468), I 
> propose to add ratelimiting "per Filesystem" .
> *Proposal of implementation* : Modify LimitedConnectionsFileSystem to add a 
> ratelimiter on both Input and OutputStream.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to