On Fri, Apr 5, 2019 at 1:42 AM Jeff Forcier <j...@bitprophet.org> wrote:
> Hi Mich, > > Thanks a lot for your detailed feedback; I appreciate the difficulty > you're going through. The development status has been in limbo since my > employer was acquired late last year, so I haven't had time to polish all > these rough edges. This ought to change in the near future now that I've > found new employment. > > I'm in the middle of a cross-country move this week and next, so I can't > be as detailed as I'd like, but some quick points: > This is already good to know. I had the feeling that the project seemed kind of stagnant. But knowing that you are still motivated to continue is a helpful piece of information. > > [The upgrade page] was not really helpful and very confusing. I ended > up perusing the API docs to find what I needed, but this still involved > some guesswork. > > I've got my own ideas for how to improve that page's usability, but I'd > love to hear about what you were expecting to find / what sorts of info (or > style of organization) would have helped you here. > This is really hard to answer as it is pretty much different for everyone. So what might be good for me might be useless for someone else. From the top of my head I'm wondering if it were possible to add some examples? And maybe explicitly mention that you accept PRs for such "old vs new" examples. I'd certainly be willing to share at least the simple tasks I have lying around. This might even open up the possibility for some minor review by the maintainers giving tips on how to do things more effectively with fabric-2 > > I now believe that the "fab" executable is useless for us and that we > could do everything with "invoke" and just import fabric connections in our > "tasks.py" file. > > The `fab` shim needs more work to avoid a lot of the rough edges (to be > understated) you've run into, but... > > > it does not seem that this is the intended way to do it. > > ... this isn't accurate - using Invoke as the main interface/framework and > then importing Fabric into that code *is* definitely a supported/intended > use case. (A major goal here was to be less opinionated and more just > providing building block APIs that advanced users can use however they > need, without us having to anticipate every use case.) > > I _do_ want to get to where v2's `fab` is more readily usable for folks > coming from v1, as well, but I suspect for now an intermediate-to-advanced > team would find more utility with the other approach. > Fair enough. Since I wrote my last mail I've moved more into the direction of using "invoke" as main task-runner and use the fabric API internally once I need to run stuff remotely. > > Dropping the concept of "roles" surprised me. > > Roles have NOT been dropped purposefully, they just haven't been migrated > yet - getting the lower level APIs set up (again, in the spirit of "let > advanced users build what they need instead of making them wait for higher > level stuff to get finished") was done first. > > The upgrading page should have a link to the issue or issues around > building a "v2" style functionality that fills the roles use case better > (mostly by bridging configuration to the Group class). If you didn't run > across that line, it's likely part of the previous issue re: searchability > of that page. > It may indeed be true that I've not come across this yet. Our use-cases are all really simple and don't need much of configuration. So, while the configuration system looks very powerful, it seems unnecessarily complex for our work. Note that we also use ansible to manage our servers, and the new complexity in fabric blurs the lines between ansible and fabric/invoke. We see fabric/invoke as a tool to automate common development tasks (like syncing the dev DB from the prod DB or setting up a clean development environment), while ansible is dealing with everything system related like ensuring DBs and users exist, privileges and so on. We *could* do all that in fabric/invoke, but in the end, ansible is better suited for those tasks, and we also have the sys-admin and developer roles separated. Ansible is used by the sys-admins while invoke/fabric is used by the devs. To make a long story short, we don't really need complex configurations. So this was just barely skimmed when I was reading the docs. > > Running local tasks does not behave as expected due to the way the > console and environment variables are handled. > > This feature area ought to be an overall upgrade to how it worked in v1, > so I'd definitely appreciate details here (if you posted them in another > thread, just let me know, as noted I have not had time to pay close > attention). It's possible you're running into things related to the v2 APIs > having more fine-grained 'levers' for you to tweak, not sure. > The most prominent example is visible in one of our "default" tasks called "run" that each project is supposed to have. For example, when running an application using ANSI escape codes to colorise logs, those colors are no longer visible when running via fabric2. More importantly though, I have an application that runs a VueJS development server via "fab run". This get completely butchered by the new system. Another great example would be running a "docker push" (which exists in our "publish" tasks) via both fabric 1 and 2. In fabric1 it works just as if you were to run it manually on the console. In fabric2 the output is wonky. I have for now replaced "ctx.local" with "subprocess.call" which works just fine. But if I'm using invoke, I'd really love to *stay* in invoke and just use "ctx.run". But for now this causes too much problems with visual output. > > "put" no longer supports folders [...] > > I think this was noted on the upgrading page, but the recursive put() had > an absolutely dismal utility-to-bug-surface ratio (esp given it's just us > poorly reinventing functionality that native Unix tools have had for > decades). Is there a reason you're not able to use things like tar.gz? > We'll be able to work around that as you mention. It just puzzled me why this was dropped. It was very convenient. But you're right: using tools like zip/tar is proven to work and has some useful flags as well. My expectation is to make the 'ramp' to this easier (eg making it trivial > for users to determine what native tools they have available - tgz, rsync, > etc - or autoselection) without actually porting v1's bug-prone > implementation. > > > I am wary of the fact that I *need* this kind of scaffolding for > something that was much easier with fabric-1 > > As you can probably guess from my opening and the rest, this is (IMO) due > almost entirely to the fact that I wanted to get v2 out into the world > instead of waiting until it was "fully ready" (combined with unfortunate > timing in my professional and personal life) and does _not_ reflect the > intended final state of things. Building those kinds of scaffolding are > exactly where I intend the core dev to go next. > Fair enough. I'll try to translate mor of our tasks and will see how it goes :) In any case, thanks a lot for all the work you put in. With all said, fabric is still my favourite task runner :) Best, > Jeff > > On Thu, Apr 4, 2019 at 1:37 AM Michel Albert <mic...@albert.lu> wrote: > >> Hi, >> >> In our development team we have been using fabric-1 as task runner for >> many years and we are really happy with it. Initially we chose it for the >> following reasons: >> >> - Being Python, we can transfer our development logic to our >> task-runner as Python is our main development language >> - Defining tasks is really easy and flexible >> - It allowed us to create a library of shared tasks for the whole >> team making definition of new fabfiles a breeze for our projects >> - It "just worked" >> >> We are now planning to review and refresh our whole development process >> and environment. The task runner is part of this. >> As some preparation work and to gain experience I've tried to move some >> personal projects to fabric-2 and ran into several problems: >> >> - The first hurdle was the "migrating to fabric-2" document. It was >> not really helpful and very confusing. I ended up perusing the API docs to >> find what I needed, but this still involved some guesswork. >> - The split of "invoke" and "fabric" is confusing. Especially since >> the argument passed into the tasks depends on how they are called. If >> called without host information, the first argument will be an invoke >> context, if host information is available it becomes a fabric connection, >> and both are not API compatible. After several attempts, I now believe >> that >> the "fab" executable is useless for us and that we could do everything >> with >> "invoke" and just import fabric connections in our "tasks.py" file. This >> way we have consistency with the first argument, but it does not seem that >> this is the intended way to do it. >> - Dropping the concept of "roles" surprised me. It seems to me that >> this would be an integral part of a task-runner. This now forces me to >> implement "roles" logic in all of my task files. >> - Running local tasks does not behave as expected due to the way the >> console and environment variables are handled. Using "pty=True" does not >> help with everything and "replace_env=False" is also a bit cryptic. >> - "put" no longer supports folders and one is now reliant on >> "patchwork" to get the "rsync" task. I am worried that the docs state that >> patchwork is not a priority as we rely on the functionality to push >> folders >> onto remote machines. fabric-1 made this trivial. >> >> I *do *like the configuration system and how task-arguments are handled >> in the CLI tool. This is a big step-up from fabric-1 >> >> After discussing this with the team another member stated that he made >> the same effort on some of his personal projects and ran into the exact >> same issues and has given up on fabric-2 as it makes task-definitions too >> complicated compared to fabric-1. Although in defense of fabric, I wonder >> if this is not foremost an issue with the documentation. As I mentioned, I >> fled to the API docs after reading the "prose" docs as they really >> inundated me details I did not need to know and was unable to find what I >> was looking for. >> >> So in our team I am now faced with making a decision of picking a new >> task-runner. The options I see are: >> >> - *Switch to fabric-2* - I'm worried that the complexities will make >> it difficult for the less experienced developers in our team to define >> tasks. It also seems that fabric-2 is not really gaining traction which >> led >> me to the subject of this mail. >> - *Stick with fabric-1 and hope that fabric-2 is at some point on par >> with fabric-1 in terms of simplicity* - This does not seem like a >> good long-term plan and would force us to install Python2 on our machines >> solely for fabric. >> - *Use Makefiles instead* - This would solve most of our problems but >> makes it difficult to share common tasks and is definitely not as >> convenient as writing Python >> - *Find another task-runner* - This is unknown/unexplored territory >> >> All the points mentioned above are mainly inconveniences which can be >> mitigated by writing a bit of scaffolding around it. But I am wary of the >> fact that I *need* this kind of scaffolding for something that was much >> easier with fabric-1. The main goals for me is to have a task-runner where >> it is easy to define and execute tasks. Even for people that don't do >> Python development as their daily business. >> >> In order to make an informed decision about our future task-runner the >> question in the topic is very important to me: *What are the future >> plans for fabric-2?* >> >> >> >> BR >> Mich >> _______________________________________________ >> Fab-user mailing list >> Fab-user@nongnu.org >> https://lists.nongnu.org/mailman/listinfo/fab-user >> > > > -- > Jeff Forcier > Unix sysadmin; Python engineer > http://bitprophet.org > > On Thu, Apr 4, 2019 at 1:37 AM Michel Albert <mic...@albert.lu> wrote: > >> Hi, >> >> In our development team we have been using fabric-1 as task runner for >> many years and we are really happy with it. Initially we chose it for the >> following reasons: >> >> - Being Python, we can transfer our development logic to our >> task-runner as Python is our main development language >> - Defining tasks is really easy and flexible >> - It allowed us to create a library of shared tasks for the whole >> team making definition of new fabfiles a breeze for our projects >> - It "just worked" >> >> We are now planning to review and refresh our whole development process >> and environment. The task runner is part of this. >> As some preparation work and to gain experience I've tried to move some >> personal projects to fabric-2 and ran into several problems: >> >> - The first hurdle was the "migrating to fabric-2" document. It was >> not really helpful and very confusing. I ended up perusing the API docs to >> find what I needed, but this still involved some guesswork. >> - The split of "invoke" and "fabric" is confusing. Especially since >> the argument passed into the tasks depends on how they are called. If >> called without host information, the first argument will be an invoke >> context, if host information is available it becomes a fabric connection, >> and both are not API compatible. After several attempts, I now believe >> that >> the "fab" executable is useless for us and that we could do everything >> with >> "invoke" and just import fabric connections in our "tasks.py" file. This >> way we have consistency with the first argument, but it does not seem that >> this is the intended way to do it. >> - Dropping the concept of "roles" surprised me. It seems to me that >> this would be an integral part of a task-runner. This now forces me to >> implement "roles" logic in all of my task files. >> - Running local tasks does not behave as expected due to the way the >> console and environment variables are handled. Using "pty=True" does not >> help with everything and "replace_env=False" is also a bit cryptic. >> - "put" no longer supports folders and one is now reliant on >> "patchwork" to get the "rsync" task. I am worried that the docs state that >> patchwork is not a priority as we rely on the functionality to push >> folders >> onto remote machines. fabric-1 made this trivial. >> >> I *do *like the configuration system and how task-arguments are handled >> in the CLI tool. This is a big step-up from fabric-1 >> >> After discussing this with the team another member stated that he made >> the same effort on some of his personal projects and ran into the exact >> same issues and has given up on fabric-2 as it makes task-definitions too >> complicated compared to fabric-1. Although in defense of fabric, I wonder >> if this is not foremost an issue with the documentation. As I mentioned, I >> fled to the API docs after reading the "prose" docs as they really >> inundated me details I did not need to know and was unable to find what I >> was looking for. >> >> So in our team I am now faced with making a decision of picking a new >> task-runner. The options I see are: >> >> - *Switch to fabric-2* - I'm worried that the complexities will make >> it difficult for the less experienced developers in our team to define >> tasks. It also seems that fabric-2 is not really gaining traction which >> led >> me to the subject of this mail. >> - *Stick with fabric-1 and hope that fabric-2 is at some point on par >> with fabric-1 in terms of simplicity* - This does not seem like a >> good long-term plan and would force us to install Python2 on our machines >> solely for fabric. >> - *Use Makefiles instead* - This would solve most of our problems but >> makes it difficult to share common tasks and is definitely not as >> convenient as writing Python >> - *Find another task-runner* - This is unknown/unexplored territory >> >> All the points mentioned above are mainly inconveniences which can be >> mitigated by writing a bit of scaffolding around it. But I am wary of the >> fact that I *need* this kind of scaffolding for something that was much >> easier with fabric-1. The main goals for me is to have a task-runner where >> it is easy to define and execute tasks. Even for people that don't do >> Python development as their daily business. >> >> In order to make an informed decision about our future task-runner the >> question in the topic is very important to me: *What are the future >> plans for fabric-2?* >> >> >> >> BR >> Mich >> _______________________________________________ >> Fab-user mailing list >> Fab-user@nongnu.org >> https://lists.nongnu.org/mailman/listinfo/fab-user >> > > > -- > Jeff Forcier > Unix sysadmin; Python engineer > http://bitprophet.org >
_______________________________________________ Fab-user mailing list Fab-user@nongnu.org https://lists.nongnu.org/mailman/listinfo/fab-user
