This has been something that I've wanted to see for a while but as Fokko mentions, it can be hard for very large files that see lots of PRs. Probably easiest is to pull them apart one class at a time to minimize merge conflict risks.
Additionally, in some cases, there are interdependencies that may make it hard to pull them apart without creating circular dependencies, especially in the case of models.py Either way, I am hugely in favor of trying to pull these things apart. --George On Sun, Mar 11, 2018 at 1:44 PM Driesprong, Fokko <[email protected]> wrote: > Maybe wait until this one has been merged 🤪 > https://github.com/apache/incubator-airflow/pull/3116/files > > Cheers, Fokko > > Op zo 11 mrt. 2018 om 21:34 schreef Driesprong, Fokko <[email protected] > > > > > Hi Bruno, > > > > I agree that there are files that are too big as you mentioned. Sometimes > > structures are being refactored like I did with the sensors. > > > > The problem is that if you refactor one of these files, most PR requests > > will have merge conflicts. But this should not be seen as an impediment > to > > increase the quality of Airflow’s code. > > Especially the models.py should be fairly easy to refactor and split into > > different files. This file grew due historical reasons. If you feel like > > picking up this task, I encourage you to create a Jira for it and start > > cracking. > > > > Cheers, Fokko > > > > Op zo 11 mrt. 2018 om 20:11 schreef Bruno Bonagura <[email protected] > > > > > >> I mean 'big files into submodules'. > >> > >> On Sun, Mar 11, 2018 at 4:09 PM, Bruno Bonagura <[email protected]> > >> wrote: > >> > >> > Hello, > >> > > >> > I'm pretty newbie to the project, so I apologize in advance if I'm > being > >> > too silly. > >> > > >> > Is there any plan or goal to refactor big files into packages, like it > >> > happened to sensors.py? Has it been tried before with the files I list > >> > bellow and failed? I searched Jira for 'refactor' and didn't find > much. > >> > > >> > Airflow codebase has some giant ugly files, what makes it difficult to > >> > find things and organize. Also it makes difficult for new contributors > >> to > >> > understand the code and could even keep some from contributing, having > >> > impact in the community growth. > >> > > >> > Here are the bigger files: > >> > > >> > 5008 airflow/models.py > >> > 3121 tests/jobs.py > >> > 2844 airflow/www/views.py > >> > 2595 airflow/jobs.py > >> > 2494 tests/core.py > >> > 1847 tests/models.py > >> > 1674 airflow/bin/cli.py > >> > 1487 airflow/contrib/hooks/bigquery_hook.py > >> > 1045 airflow/contrib/operators/dataproc_operator.py > >> > > >> > I've been playing and experimenting with models.py a little. It's > >> > difficult, but it might be possible. > >> > > >> > Well, that's it. > >> > > >> > Best regards. > >> > Bruno > >> > > >> > > >
