Hi Taka, I can't see the image! Maybe the Apache email list don't support attachments.
Best! Rafael J. R. Novello Skype: rafael.novello Blog: http://rafanovello.blogspot.com.br/ Em qua, 24 de out de 2018 às 21:14, Daniel Takabayashi < daniel.takabaya...@gmail.com> escreveu: > Hi all, > > To try to give a little more details about this "new concept" that we want > to bring to the marvin toolbox, I did this simple architecture draw. > > [image: marvin-architecture-views (5).png] > > The general idea here is try to transform the toolbox something > disconnected with the "language", something more agnostic. Also in this new > architecture we could use remote resource to process engines and make easy > the support for new languages. > > This "new toolbox" will be the only thing that a Marvin user must to > install and also we could start to support multiples O.S (once the REPL is > a dummy application that only interprets and by pass commands). > > Regards, > Taka > > Em qua, 24 de out de 2018 às 09:52, Rafael Novello < > rafa.reis.nove...@gmail.com> escreveu: > >> Alan, >> >> Yes! We are using the Docker SDK and it's possible to use the API to >> automate, but it's a little bit harder than automate CLI calls. >> >> Atenciosamente, >> Rafael J. R. Novello >> >> Skype: rafael.novello >> Blog: http://rafanovello.blogspot.com.br/ >> >> >> Em qua, 24 de out de 2018 às 12:02, Alan Silva <ju...@apache.org> >> escreveu: >> >> > Hi, >> > >> > One question here, I understand that we start to use with this PoC the >> > Docker SDK API, right? >> > >> > Why not use the API to expose some endpoints to permit this kind of >> > automation by devops? >> > >> > I think it is possible and it solves the CLI problem, right? >> > >> > >> > On Tue, Oct 23, 2018 at 1:05 PM Rafael Novello < >> > rafa.reis.nove...@gmail.com> >> > wrote: >> > >> > > Lucas, >> > > >> > > Sorry, I didn't understood your question bellow. >> > > >> > > "Would it make sense to use the same solution that we will use for >> > having a >> > > single-language REPL to have a single-language CLI?" >> > > >> > > For DevOps purposes, maybe this new toolbox concept is not ideal. I >> think >> > > we can keep the CLI inside the docker container but it will not easy >> to >> > > automate >> > > jobs by this way. >> > > >> > > How to deal with this issue? Voting? >> > > Atenciosamente, >> > > Rafael J. R. Novello >> > > >> > > Skype: rafael.novello >> > > Blog: http://rafanovello.blogspot.com.br/ >> > > >> > > >> > > Em sex, 19 de out de 2018 às 19:00, Lucas Bonatto Miguel < >> > > lucasb...@apache.org> escreveu: >> > > >> > > > Got it! Thanks for clarifying. >> > > > >> > > > Would it make sense to use the same solution that we will use for >> > having >> > > a >> > > > single-language REPL to have a single-language CLI? My only concern >> > with >> > > > killing the CLI is that you remove an essential feature for DevOps. >> > > > >> > > > - Lucas >> > > > >> > > > On Fri, Oct 19, 2018 at 1:52 PM Rafael Novello < >> > > > rafa.reis.nove...@gmail.com> >> > > > wrote: >> > > > >> > > > > Lucas, >> > > > > >> > > > > The idea is that REPL will substitute the actual CLI. It's because >> > with >> > > > the >> > > > > actual concept (using language specific CLI) we will need to >> > > re-implement >> > > > > the same features many times and probably each CLI will have a >> > > different >> > > > > behavior because some language specific restrictions and/or >> > > limitations. >> > > > > >> > > > > With this new concept, all users will have the same experience >> > > > interacting >> > > > > with Marvin REPL and they will use the bests tools to develop >> their >> > > > engines >> > > > > (Jupyter Notebook and/or Lab with different languages support or >> even >> > > > > Apache Zeppelin). All the interact will occur through REPL and >> docker >> > > > > protocol. >> > > > > >> > > > > Alan, Yes! As Lucas said, the concept is the same but we can use >> > docker >> > > > to >> > > > > do the same job. >> > > > > >> > > > > Best! >> > > > > >> > > > > Em sex, 19 de out de 2018 às 00:39, Lucas Bonatto Miguel < >> > > > > lucasb...@apache.org> escreveu: >> > > > > >> > > > > > Thanks for the clarifications Rafael, so from what I understood, >> > > > Marvin's >> > > > > > developers would not use the REPL to explore data and test >> models, >> > > i.e. >> > > > > > develop the engine. Is the idea to build something more like an >> > > > > interactive >> > > > > > CLI? I think an interactive CLI would be useful for the >> developer >> > > > > > experience, however, it's important to keep the unattended >> > (current) >> > > > CLI >> > > > > in >> > > > > > place for scenarios where users want to automate Marvin tasks. >> > > > > > >> > > > > > Alan, I believe the idea is still the same as you started, but >> > using >> > > > > > docker SDK now. >> > > > > > >> > > > > > - Lucas >> > > > > > >> > > > > > On Thu, Oct 18, 2018 at 1:29 PM Rafael Novello < >> > > > > > rafa.reis.nove...@gmail.com> >> > > > > > wrote: >> > > > > > >> > > > > > > Hi Lucas! >> > > > > > > >> > > > > > > First of all +1 for REPL POCs to Apache! >> > > > > > > >> > > > > > > Let me help with some comments: >> > > > > > > >> > > > > > > 1 - We have tested NodeJS, Scala and Python and the easiest >> one >> > was >> > > > > > Python. >> > > > > > > We have found a small project [1] that have all features we >> > desired >> > > > for >> > > > > > > REPL: >> > > > > > > - Autocomplete >> > > > > > > - Python commands disabled (the user have only the commands we >> > > > > provide). >> > > > > > > The other languages REPL options don't have this feature. >> > > > > > > - Easy to show output >> > > > > > > - Etc >> > > > > > > So, I think the language chosen will not be important here >> > because >> > > > the >> > > > > > user >> > > > > > > will only interact with the commands that we create. >> > > > > > > >> > > > > > > 2 - The "engine-generate" command will download a docker image >> > that >> > > > we >> > > > > > > create for that language and start a container with the basic >> > > project >> > > > > > > structure for that language. >> > > > > > > >> > > > > > > 3 - The REPL client will use the "docker protocol" to run >> > command, >> > > > > > > start/stop services and etc inside the container and it will >> > > receive >> > > > > the >> > > > > > > log stream to show. No, the REPL will no pass code snippets >> for >> > > > docker >> > > > > > > container (I think it will not be necessary) >> > > > > > > >> > > > > > > 4 - Yep! Like I said on the first item. >> > > > > > > >> > > > > > > [1] - https://github.com/italorossi/ishell >> > > > > > > >> > > > > > > Let me know if there is any other question! >> > > > > > > Best! >> > > > > > > Rafael J. R. Novello >> > > > > > > >> > > > > > > Skype: rafael.novello >> > > > > > > Blog: http://rafanovello.blogspot.com.br/ >> > > > > > > >> > > > > > > >> > > > > > > Em qui, 18 de out de 2018 às 17:00, Lucas Bonatto Miguel < >> > > > > > > lucasb...@apache.org> escreveu: >> > > > > > > >> > > > > > > > + 1 for migrating the REPL repo to Apache >> > > > > > > > >> > > > > > > > I have a few questions about the previous explanation: >> > > > > > > > 1) The REPL itself, it would be an application in which >> > > language? >> > > > > > > Remember >> > > > > > > > that the main idea is to allow the user to program on his >> > > preferred >> > > > > > > > language in the REPL. >> > > > > > > > 2) Should the engine-generate command also generate a >> docker >> > > image >> > > > > > with >> > > > > > > > the user's application? >> > > > > > > > 3) What type of communication would happen between the REPL >> > and >> > > > the >> > > > > > > engine >> > > > > > > > via Docker SDK? Would the REPL pass snippets of code to be >> > > executed >> > > > > by >> > > > > > > the >> > > > > > > > docker container? >> > > > > > > > 4) Have you considered code completion in the REPL? >> > > > > > > > >> > > > > > > > >> > > > > > > > On Thu, Oct 18, 2018 at 7:53 AM Zhang Yifei < >> > > > yifei.z.l...@gmail.com> >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > Ok guys, >> > > > > > > > > >> > > > > > > > > The basic idea is to provide only one Toolbox for multiple >> > > > > languages. >> > > > > > > > > We are looking for possibility to build a single Marvin >> Repl, >> > > > > > > > > instead of severals toolboxes with differentes interfaces >> or >> > > > > > commands. >> > > > > > > > > >> > > > > > > > > In this case, the engine-generate command will download >> and >> > > > start a >> > > > > > > > Docker >> > > > > > > > > container with basic engine structure corresponding the >> > choosed >> > > > > > > language. >> > > > > > > > > this means we don't need to build Toolboxes of differents >> > > > > languages, >> > > > > > we >> > > > > > > > > just >> > > > > > > > > build the engine template of all languages that we want to >> > > > support >> > > > > > and >> > > > > > > > > provide it >> > > > > > > > > as Docker containers >> > > > > > > > > >> > > > > > > > > We have started researches around our basic requirements >> > like: >> > > > > > > > > - Repl interface >> > > > > > > > > - System communication >> > > > > > > > > - Connection security >> > > > > > > > > - Tool popularity >> > > > > > > > > - Update complexity >> > > > > > > > > - Languages support >> > > > > > > > > - ...... >> > > > > > > > > >> > > > > > > > > And we did some POC with code here: >> > > > > > > > > https://github.com/marvin-ai/marvin-repl >> > > > > > > > > >> > > > > > > > > There is POC testing gRPC using Scala and Python, >> > > > > > > > > Repl inteface and Docker SDK with NodeJS, >> > > > > > > > > Repl interface and Docker SDK with Python. >> > > > > > > > > >> > > > > > > > > At this moment we prefer the Repl interface + Docker SDK >> way, >> > > > > because >> > > > > > > > good >> > > > > > > > > part of the requirements >> > > > > > > > > will be guaranteed by Docker. >> > > > > > > > > >> > > > > > > > > With this informations, what do you think? Should we >> submit >> > all >> > > > > this >> > > > > > > POCs >> > > > > > > > > to Apache Repo? >> > > > > > > > > Please feel free to opine. >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > Thats all, thanks!!! >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > Em ter, 16 de out de 2018 às 18:55, Daniel Takabayashi < >> > > > > > > > > daniel.takabaya...@gmail.com> escreveu: >> > > > > > > > > >> > > > > > > > > > Zhang, >> > > > > > > > > > >> > > > > > > > > > I think the best approach is give us a better >> explanation >> > > about >> > > > > > this >> > > > > > > > new >> > > > > > > > > > feature and how this can help us to archive the desired >> > > feature >> > > > > > > > (support >> > > > > > > > > > multiple languages). Than if the majority agree than I >> can >> > do >> > > > the >> > > > > > > > merge, >> > > > > > > > > > just like I did before, but in another branch. >> > > > > > > > > > >> > > > > > > > > > Let me know if makes sense, ok? >> > > > > > > > > > >> > > > > > > > > > Taka >> > > > > > > > > > >> > > > > > > > > > Em ter, 16 de out de 2018 às 14:00, Luciano Resende < >> > > > > > > > > luckbr1...@gmail.com> >> > > > > > > > > > escreveu: >> > > > > > > > > > >> > > > > > > > > > > So, what is the POC, is it a refactoring of the >> existing >> > > > repo? >> > > > > Or >> > > > > > > is >> > > > > > > > > > > it a new rewrite of the repo? >> > > > > > > > > > > >> > > > > > > > > > > Just asking as it might make sense to make it a branch >> > then >> > > > > > > actually >> > > > > > > > a >> > > > > > > > > > > thing in parallel, as this will have an effect for >> > > releases, >> > > > > etc. >> > > > > > > but >> > > > > > > > > > > you guys know more here than I do. >> > > > > > > > > > > >> > > > > > > > > > > Also, it's probably good to have a write up of the >> main >> > > > > direction >> > > > > > > of >> > > > > > > > > > > the design that can help people get familiar with the >> new >> > > > > > approach. >> > > > > > > > > > > On Tue, Oct 16, 2018 at 11:12 AM Zhang Yifei < >> > > > > > > yifei.z.l...@gmail.com >> > > > > > > > > >> > > > > > > > > > > wrote: >> > > > > > > > > > > > >> > > > > > > > > > > > Hey guys, >> > > > > > > > > > > > >> > > > > > > > > > > > We have reorganized the Poc repo, and want to merge >> it >> > to >> > > > our >> > > > > > > > Apache >> > > > > > > > > > > repo. >> > > > > > > > > > > > Just making sure here before we do the merge, >> because >> > im >> > > > not >> > > > > > sure >> > > > > > > > if >> > > > > > > > > > > doing >> > > > > > > > > > > > this will perserve the Git commit history. >> > > > > > > > > > > > What we are planning to do is: >> > > > > > > > > > > > >> > > > > > > > > > > > git filter-branch --subdirectory-filter <git >> > > > > origin_repository >> > > > > > > > > > directory> >> > > > > > > > > > > > -- --all >> > > > > > > > > > > > mkdir POCs/ >> > > > > > > > > > > > git mv * POCs/ >> > > > > > > > > > > > git commit -m "colleted the data to move" >> > > > > > > > > > > > >> > > > > > > > > > > > git clone <git destination_repository_url> cd <git >> > > > > > > > > > > > destination_repository_directory> git remote add >> > > > > > > > > > origin_repository_branch >> > > > > > > > > > > > <git origin_repository directory> git pull >> > > > > > > origin_repository_branch >> > > > > > > > > > > master >> > > > > > > > > > > > --allow-unrelated-histories git remote rm >> > > > > > > origin_repository_branch >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > Thanks in advance ! >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > -------------------------------------------------------------- >> > > > > > > > > > > > Zhang Yifei >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > -- >> > > > > > > > > > > Luciano Resende >> > > > > > > > > > > http://twitter.com/lresende1975 >> > > > > > > > > > > http://lresende.blogspot.com/ >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > -- >> > > > > > > > > >> > -------------------------------------------------------------- >> > > > > > > > > Zhang Yifei >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
