Hello Leonardo, Lots of good advice! You can find Marvin's issues here: issues.apache.org/jira
Best regards Em seg., 25 de mai. de 2020 às 23:10, Leonardo Tavares < leonardot1...@gmail.com> escreveu: > Hello everyone, first let me introduce myself: > My name is Leonardo Tavares and I'm an undergrad Computer Engineering > student at UFSCar doing my undergraduate dissertation supervised by > Professor Daniel Lucrédio. > > I'm using Marvin as a MLOps framework for implementing a proof of concept > architecture based on a project from work. It consists mainly on having > multiple models running at the same time and a "model selector" that parses > all requests and sends to the correct model. > > Along the development of the project I was writing down every issue I faced > and points where Marvin helped me compared to vanilla Python. I was going > to open some issues in the GitHub repository, but I couldn't find the tab, > so I'm going to drop all feedback here and if needed I can send each issue > in a separate email. > > Regarding specs, I was using a Windows 10 laptop but the Marvin > engines/notebooks were running in a Docker container using Python 3.7. To > reduce the message size, I'm going to focus on the issues I had during the > project development: > > - Probably my biggest issue, Marvin's documentation is scarce, meaning > that most of the problems I faced I had to open the source code and try > to > understand what was happening. > - The public engines available on GitHub have an older Marvin template, > resulting in broken dependencies. > - The usage of gRPC depends on compiling the "actions.proto" file from > source (or has some other easier way to use that is not documented). > - Reloading artifacts using gRPC does not work. Only works when using > rest. > - The artifact names are static, not allowing the creation of other > artifacts and demanding the user knowledge of each artifact for each > DASFE > action, despite having artifacts definition on "engine.metadata". > - If you select a blank cell and tag it as a DASFE action, it will cause > the engine to break (maybe could have a "pass" when the cell is blank?). > - "marvin test" command always fails, even with thedryrun and http/gRPC > server working. > - "marvin pkg-updatedeps" is broken, get_installed_distributions was > deprecated in newer versions of pip. > - "make docker-build" is broken, due to the deprecation of the > "ppa:webupd8team/java" (along with possibly some other issues). > > I'd also like to suggest on a point that isn't a issue but that I think > would be a great addition to the framework: > > - If I managed to correctly understand the artifact versioning, > it currently works by dumping each "marvin_" variable into a pickle and > loading it. Many problems can occur if the source code of the project > changes between artifact versions. > - A possible solution is to tag each artifact with a git commit, > saving the pickle of each artifact along with the commit that it was > generated. > - When reloading, the engine executor would not only load the pickle > of the artifact but also change the source code being executed to the > version tagged with the artifact. > > What do you think? > Overall, even with all the issues presented above, the framework have great > features that speedup the development and management of the engines, having > great potential to be used in several fields. > To me, the most glaring issue is the lack of detailed documentation and > tutorials, turning away possible adopters due to the steep learning curve. > To help with this issue I'd like to help on writing some tutorials on gRPC, > if possible :) > > Thanks! > > -- > Leonardo Tavares Oliveira > Graduando em Engenharia de Computação > Universidade Federal de São Carlos > > <http://www.instructables.com/member/leonardots/> > <https://github.com/leotavares> > <https://www.linkedin.com/in/leonardot1802/> > <http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K8064478U3> > -- -------------------------------------------------------------- Zhang Yifei
