Tomaso, Awesome! Thanks for the links. I will be happy to join, (But I wont be able to contribute to the actual paper untill Oct 24).
I suggest we should consider popular NMT toolkits for the survey regardless of their compatibility with AL-2. We should see all the tricks and features, and know if we are missing out on any useful features after enforcing the AL-2 filter (and create issues for adding those features). here are some more NMT toolkits to be included in the survey. - Fairseq https://github.com/pytorch/fairseq - Tensor2tensor https://github.com/tensorflow/tensor2tensor/ - Nematus https://github.com/EdinburghNLP/nematus - xNMT https://github.com/neulab/xnmt - XLM https://github.com/facebookresearch/XLM/ |-> MASS https://github.com/microsoft/MASS/ --> https://github.com/thammegowda/unmass (took that and made it easier to install and use) Some old stuff which we are defnitely not going to use but worth mentioning in the survey (for the sake of completion) - https://github.com/google/seq2seq - https://github.com/tensorflow/nmt - https://github.com/isi-nlp/Zoph_RNN Cheers, TG ಭಾನು, ಅಕ್ಟೋ 18, 2020 ರಂದು 11:17 ಅಪರಾಹ್ನ ಸಮಯಕ್ಕೆ ರಂದು Tommaso Teofili < tommaso.teof...@gmail.com> ಅವರು ಬರೆದಿದ್ದಾರೆ: > Following up on the report topic, I've created an overleaf doc for everyone > who's interested in working on this [1]. > > First set of (AL-2 compatible) NMT toolkits I've found: > - Joey NMT [2] > - OpenNMT [3] > - MarianNMT [4] > - Sockeye [5] > - and of course RTG already shared by Thamme [6] > > Regards, > Tommaso > > [1] : https://www.overleaf.com/8617554857qkvtqtpcxxmw > [2] : https://github.com/joeynmt/joeynmt > [3] : https://github.com/OpenNMT > [4] : https://github.com/marian-nmt/marian > [5] : https://github.com/awslabs/sockeye > [6] : https://github.com/isi-nlp/rtg-xt > > On Wed, 14 Oct 2020 at 11:06, Tommaso Teofili <tommaso.teof...@gmail.com> > wrote: > > > very good idea Thamme! > > I'd be up for writing such a short survey paper as a result of our > > analysis. > > > > Regards, > > Tommaso > > > > > > On Wed, 14 Oct 2020 at 05:23, Thamme Gowda <tgow...@gmail.com> wrote: > > > >> Tomasso and others, > >> > >> > I think we may now go into a research phase to understand what > existing > >> toolkit we can more easily integrate with. > >> Agreed. > >> if we can write a (short) report that compares various NMT toolkits of > >> 2020, it would be useful for us to make this decision as well as to the > >> NMT > >> community. > >> Something like a survey paper on NMT research but focus on toolkits and > >> software engineering part. > >> > >> > >> > >> ಶುಕ್ರ, ಅಕ್ಟೋ 9, 2020 ರಂದು 11:39 ಅಪರಾಹ್ನ ಸಮಯಕ್ಕೆ ರಂದು Tommaso Teofili < > >> tommaso.teof...@gmail.com> ಅವರು ಬರೆದಿದ್ದಾರೆ: > >> > >> > Thamme, Jeff, > >> > > >> > your contributions will be very important for the project and the > >> > community, especially given your NLP background, thanks for your > >> support! > >> > > >> > I agree moving towards NMT is the best thing to do at this point for > >> > Joshua. > >> > > >> > Thamme, thanks for your suggestions! > >> > I think we may now go into a research phase to understand what > existing > >> > toolkit we can more easily integrate with. > >> > Of course if you like to integrate your own toolkit then that'd be > even > >> > more interesting to see how it compares to others. > >> > > >> > If that means moving to Python I think it's not a problem; we can > still > >> > work on Java bindings to ship a new Joshua Decoder implementation. > >> > > >> > The pretrained models topic is imho something we will have to embrace > at > >> > some point, so that others can: > >> > a) just download new LPs > >> > b) eventually fine tune with their own data > >> > > >> > I am looking forward to start this new phase of research on Joshua. > >> > > >> > Regards, > >> > Tommaso > >> > > >> > On Tue, 6 Oct 2020 at 18:30, Jeff Zemerick <jzemer...@apache.org> > >> wrote: > >> > > >> > > I haven't contributed to this point but I would like to see Apache > >> Joshua > >> > > remain an active project so I am volunteering to help. I may not be > a > >> lot > >> > > of help with code for a bit but I will help out with documentation, > >> > > releases, etc. > >> > > > >> > > I do agree that NMT is the best path forward but I will leave the > >> choice > >> > of > >> > > integrating an existing library into Joshua versus a new NMT > >> > implementation > >> > > in Joshua to those more familiar with the code and what they think > is > >> > best > >> > > for the project. > >> > > > >> > > Jeff > >> > > > >> > > > >> > > On Tue, Oct 6, 2020 at 2:28 AM Thamme Gowda <tgow...@gmail.com> > >> wrote: > >> > > > >> > > > Hi Tomasso, and others > >> > > > > >> > > > *1. I support the addition of neural MT decoder. * > >> > > > The world has moved on, and NMT is clearly the way to go forward. > >> > > > If you dont believe my words, read what Matt Post himself said [1] > >> > three > >> > > > years ago! > >> > > > > >> > > > I have spent the past three years focusing on NMT as part of my > job > >> > and > >> > > > Ph.D -- I'd be glad to contribute in that direction. > >> > > > There are many NMT toolkits out there today. (Fairseq, sockeye, > >> > > > tensor2tensor, ....) > >> > > > > >> > > > The right thing to do, IMHO, is simply merge one of the NMT > toolkits > >> > with > >> > > > Joshua project. We can do that as long as it's Apache License > >> right? > >> > > > We will now have to move towards python land as most toolkits are > in > >> > > > python. On the positive side, we will be losing the ancient perl > >> > scripts > >> > > > that many are not fan of. > >> > > > > >> > > > I have been working on my own NMT toolkit for my work and research > >> -- > >> > > RTG > >> > > > https://isi-nlp.github.io/rtg/#conf > >> > > > I had worked on Joshua in the past, mainly, I improved the code > >> quality > >> > > > [2]. So you can tell my new code'd be upto Apache's standards ;) > >> > > > > >> > > > *2. Pretrained MT models for lots of languages* > >> > > > I have been working on a lib to retrieve parallel data from many > >> > sources > >> > > -- > >> > > > MTData [3] > >> > > > There is so much parallel data out their for hundreds of > languages. > >> > > > My recent estimate is over a billion lines of parallel sentences > for > >> > over > >> > > > 500 languages is freely and publicly available for download using > >> > MTData > >> > > > tool. > >> > > > If we find some sponsors to lend us some resources, we could train > >> > better > >> > > > MT models and update the Language Packs section [4]. > >> > > > Perhaps, one massively multilingual NMT model that supports many > >> > > > translation directions (I know its possible with NMT; I tested it > >> > > recently > >> > > > with RTG) > >> > > > > >> > > > I am interested in hearing what others are thinking. > >> > > > > >> > > > [1] > >> > > > > >> > > > > >> > > > >> > > >> > https://mail-archives.apache.org/mod_mbox/joshua-dev/201709.mbox/%3CA481E867-A845-4BC0-B5AF-5CEAAB3D0B7D%40cs.jhu.edu%3E > >> > > > [2] - > >> https://github.com/apache/joshua/pulls?q=author%3Athammegowda+ > >> > > > [3] - https://github.com/thammegowda/mtdata > >> > > > [4] - > >> > https://cwiki.apache.org/confluence/display/JOSHUA/Language+Packs > >> > > > > >> > > > > >> > > > Cheers, > >> > > > TG > >> > > > > >> > > > -- > >> > > > *Thamme Gowda * > >> > > > @thammegowda <https://twitter.com/thammegowda> | > >> https://isi.edu/~tg > >> > > > ~Sent via somebody's Webmail server > >> > > > > >> > > > > >> > > > ಸೋಮ, ಅಕ್ಟೋ 5, 2020 ರಂದು 12:16 ಪೂರ್ವಾಹ್ನ ಸಮಯಕ್ಕೆ ರಂದು Tommaso > >> Teofili < > >> > > > tommaso.teof...@gmail.com> ಅವರು ಬರೆದಿದ್ದಾರೆ: > >> > > > > >> > > > > Hi all, > >> > > > > > >> > > > > This is a roll call for people interested in contributing to > >> Apache > >> > > > Joshua > >> > > > > going forward. > >> > > > > Contributing could be not just code, but anything that may help > >> the > >> > > > project > >> > > > > or serve the community. > >> > > > > > >> > > > > In case you're interested in helping out please speak up :-) > >> > > > > > >> > > > > Code-wise Joshua has not evolved much in the latest months, > >> there's > >> > > room > >> > > > > for both improvements to the current code (make a new minor > >> release) > >> > > and > >> > > > > new ideas / code branches (e.g. neural MT based Joshua Decoder). > >> > > > > > >> > > > > Regards, > >> > > > > Tommaso > >> > > > > > >> > > > > >> > > > >> > > >> > > >
