Hello Richard, I'll summarize my answers from the other thread.
Unfortunately, due to other arrangements, we won't be able to attend the hackathon this weekend. However, we are very excited about the proposed objectives. We already established a way for the FTS Optimizer zero-order algorithm to make it's way into production. Once the feature is ready, with the help of Mario, we can organize a large-scale test and compare the results we get in production with the simulated results. On further ALTO and FTS integration, one aspect I see particularly promising is: 1. Use ALTO information to assess the network capability of a given RSE 2. Set a maximum limit for how much network percentage FTS can occupy. In a feedback loop via ALTO, FTS would find whether it may increase or should decrease traffic involving that RSE This maps directly to what we see in production: FTS can hit some sites too hard. Any extra knowledge FTS gets on this aspect would allow it to improve. Best Regards, Mihai for the FTS team > On 20/07/2022 03:58 Y. Richard Yang <[email protected]> wrote: > > > > > > Hi all, > > > > During the weekly ALTO meeting today, we discussed the coming hackathon and > Qin suggested that we send an update to the mailing list on the FTS+ALTO > project, and here is the update: > > > - In short, if you want to work on ALTO integration with production, open > projects, this can be a wonderful project to work on: it may help both ALTO > (in terms of its crucial deployment mandate) and the largest data-intensive > science projects at CERN. > > > Specifically: > - For those who have not tracked the hackathon, as Jordi said, it is a > continuation of the 113 Hackathon. In particular, the 113 Hackathon focused > on the integration with Rucio, which is a wonderfully designed tool used > widely in CERN and some other projects for data movement of a large amount of > data (PB -> EB). In 113, the hackathon modified the manual workflow, which is > the workflow to select the source to download a dataset when a client issues > a download command. The other workflow of Rucio is the automatic workflow, > which selects the sources and destinations to realize user-specified > replication rules. In our original plan, 114 would focus on the automatic > workflow. However, the team realized that Rucio is built on top of FTS, which > is the main tool used at CERN to schedule which transfer is scheduled at what > time, where Rucio is at a higher layer providing the transfers for FTS to > schedule. > > > - The 114 hackathon includes 2 objectives: (1) introducing resource control > to FTS, and (2) evaluating an alternative design of FTS core: the FTS > Optimizer. Hence, the work includes 3 components, integrating the FTS > production code and framework: > (1/basic) allow FTS to specify resource control goal and use ALTO to map FTS > control state (called links in FTS, where a link is a pair consisting of a > source node to a destination node, where a node is called RSE) to network > state; > (2/basic) implement a full zero-order algorithm, to achieve fully efficient, > zero-order gradient control as FTS Optimizer; > (3/stretch) realize a composition framework to compose end-to-end resource > performance function, including both zero-order and first-order gradient, > covering the bottleneck structure. > > > The project is exciting both in terms of its technical content and also in > terms of its potential impacts. The hackathon project will work under the > guidance of the Rucio project lead (Dr. Mario Lassnig) and the FTS project > lead (Mihai Patrascoiu and Steven Murray); all cc'd in this email. This can > be a wonderful opportunity for IETF, the networked systems community and the > data-intensive sciences communities to work together. > > > Please feel free to reach out to us (Jordi, Mahdi and me) if you want to join > the hackathon or later. > > > Thanks, > Richard on behalf of the IETF 114 ALTO+FTS Hackathon Team > > > On Tue, Jul 12, 2022 at 1:14 AM Jordi Ros Giralt <[email protected] > <mailto:[email protected]>> wrote: > > > > Hi all, > > > > > > I have uploaded to the IETF Hackathon Wiki the project description for > > ALTO. The proposed project is a natural evolution from the work done during > > the 113 Hackathon. I want to also thank and credit Jensen for providing the > > initial plan for the Hackathon. > > > > > > https://trac.ietf.org/trac/ietf/meeting/wiki/114hackathon > > > > > > > > > > > > You will also see that in addition to the ALTO hackathon, Ziyang will be > > leading a second project under the name "The SDN-based MPTCP-aware and > > MPQUIC-aware Transmission Control Model using ALTO". Many thanks Ziyang for > > leading this very interesting project too. > > > > > > Jordi > > On behalf of ALTO Hackathon group > > _______________________________________________ > > alto mailing list > > [email protected] <mailto:[email protected]> > > https://www.ietf.org/mailman/listinfo/alto >
_______________________________________________ alto mailing list [email protected] https://www.ietf.org/mailman/listinfo/alto
