Andres, Ok, I've got your opinion. Let's close this discussion.
17.02.2014 00:04, Andres Riancho пишет: > Taras, > > On Sun, Feb 16, 2014 at 4:28 PM, Taras <ox...@oxdef.info> wrote: >> Andres, >> >> I think it is my last attempt to change your opinion :) >> >> From the list of software you have provided I have found only flask, scrapy >> and tastypie in Ubuntu repo. Results of "apt-cache show" output are below >> inline. >> The problem is w3af built-in dependency checker duplicates >> OS (e.g. Debian/Ubuntu) packaging system. They can conflict in >> some cases. >> >> For example, I want to make package of w3af for Ubuntu 13.10. >> There is package python-xml version 3.2.0 in repository. At the same time >> w3af requires lxml version exactly 2.3.2. How can I make package of w3af? >> Should I add "sudo pip install" into preinstall script? > > Most likely not, that doesn't sound well. I don't know the right > answer because I'm not packaging expert. > > The package maintainer can always apply a patch on top of the original > software to remove the dependency check completely (I think Luciano > did something like this [0]) is he believes it is the best thing to > do. Then he's taking the responsibility of that change. My > responsibility is to tell you that with these specific package > versions it works; then people do whatever they want with it. > > [0] http://packages.ubuntu.com/precise/w3af-console - search for "diff" > >> Have you got any feedback from w3af package maintainers for Debian/Ubuntu >> and other distributions after you had add strict dependencies? > > There are no active package maintainers for w3af. They even don't > care, or don't want to maintain this software; so no, no package > maintainer told me anything about the "==". As I said above, they can > apply a diff to the software before packaging it, as done by Luciano a > while ago (not only for the dependency). > >> Is it >> important for you that w3af can be installed via simple command "apt-get >> install w3af" or through Ubuntu Software Center with single mouse click? > > Yes, and not. > > Some users would find it awesome to be able to install it from the > repo; but this has proven to be (at least for w3af) a failed path. I'm > not going to maintain a package for each distribution, because I don't > care enough as a user myself. > > Packagers who have come to the project have either failed to release > their initial package or released it and then moved their free time to > something else. In this process, they left very old versions of w3af > in the repositories of all linux distributions; which don't even make > sense for users. > > If users can install w3af with: > > git clone ... > cd w3af > ./w3af_console # Yields error with all dependencies to install > /tmp/install_w3af_dependencies.sh > > Then I'm happy. > >> If it is important for you then I recommend to add maintainers into this >> discussion and ask if it is easy for them to make package of w3af with such >> requirements. > > My opinion is that they don't care about the w3af package. > >> If it is not so important and "git clone + pip install" is preferable way of >> installation then thread can be closed. > > In the past I've thought that having w3af in the linux distribution > repos was THE BEST THING, now... not so much, because: > * Software packages are difficult to maintain > * Each time a new dependency is added the maintainer needs to create > a new package for that (python-foo) and then maintain that one also > * The whole process takes time, so from the minute I put something in > the repo to the time the new package is there it can be months; and > "hackers" love to use the latest and they will come to the repo > anyways > >> >> >>>>> Not 100% a workaround, this is also a best practice! >>>>> >>>>> https://devcenter.heroku.com/articles/python-pip#the-basics >>>> >>>> >>>> >>>> Could you please show at least one example of well-know software with >>>> such >>>> requirements? >>> >>> >>> I went through this list of the Top10 Python projects by github (not >>> sure how they choose that) and found many that either had no >>> dependencies or were not in a format in which we could compare them >>> with what we were talking about. Then found the following: >>> >>> * Strict dependencies used for this part of the project: >>> https://github.com/torchbox/wagtail/blob/master/requirements-dev.txt >>> * Gt used for the user installable part: >>> https://github.com/torchbox/wagtail/blob/master/setup.py >>> >>> * These guys install whatever is available on pypi: >>> https://github.com/jmcarp/robobrowser/blob/master/requirements.txt >>> >>> * Flask installs Gt: >>> https://github.com/mitsuhiko/flask/blob/master/setup.py >> >> Depends: python-itsdangerous, python (>= 2.7), python-jinja2 (>= 2.4), >> python (<< 2.8), python-werkzeug (>= 0.8) >> Recommends: python-pkg-resources, python-blinker >> >> >>> * A mix between Gt and "whatever" is used here: >>> https://github.com/Eugeny/ajenti/blob/dev/requirements.txt >>> >>> * Scrapy uses a mix of GT and "whatever": >>> https://github.com/scrapy/scrapy/blob/master/requirements.txt >> >> >> Depends: python2.7, python (>= 2.7.1-0ubuntu2), python (<< 2.8), >> python-twisted-core, python-twisted-web, python-twisted-conch, >> python-twisted-mail, python-libxml2, python-boto, python-w3lib >> Recommends: python-lxml, python-guppy, python-django, ipython, >> python-pygments, python-imaging, python-mysqldb >> >> >>> * Django-tastypie uses the most complex of them all, which is rather >>> interesting and makes me wonder why they didn't use "==" instead: >>> https://github.com/toastdriven/django-tastypie/blob/master/setup.py . >>> This is what I mean: 'dateutil(>=1.5, !=2.0)' >>> >> Replaces: python-django-tastypie (<= 0.9.9-2) >> Depends: python (>= 2.7.1-0ubuntu2), python (<< 2.8), python-mimeparse (>= >> 0.1.3), python-dateutil (>= 1.5), python-django (>= 1.2) >> Suggests: python-yaml, python-lxml >> > > I get your point, >= seems to be the preferred way of doing it in the > debian repos. If a packager wants, he can do that with w3af and apply > a patch to disable dependency check for w3af in the packaging process. > That way he's happy, we don't need to code anything and are also > happy. > >>> The first one is an example of "==", the rest were just to show that >>> now everyone agrees with me on what should be put on the >>> requirements.txt file (or the setup.py, which acts like the same many >>> times). >>> >>> Here are some other links where it says that "==" is a best practice: >>> * >>> https://lincolnloop.com/django-best-practices/deployment/bootstrap.html >>> (Ctrl+f "Pin your dependencies") >>> * >>> http://docs.dotcloud.com/tutorials/python/django/#specifying-requirements >>> (Ctrl+f "When you specify your requirements") >>> >>> And most importantly, the pip-installer user's guide: >>> * >>> http://www.pip-installer.org/en/latest/user_guide.html#ensuring-repeatability >>> >>> "The requirements file was generated by pip freeze or you're sure it >>> only contains requirements that specify a specific version." >>> >>> When we're talking about including a specific version in >>> requirements.txt file or not, we're talking about repeatability. I >>> want to be strict about repeatability, forcing all libraries to be >>> exactly the ones I know will work because I've tested them in the CI; >>> and your point is that it would be easier for users to install with >>> less strict version requirements (which could lead to issues in some >>> cases). >>> >>> Sadly, you believe in one thing and I can't seem to convince you of >>> the benefits of ==, and the same applies the other way (I can't be >>> convinced of the benefits of >=). Unless I hear a definitive reason on >>> why == is bad, I won't change it. >>> >>>> By the way in w3af dev list I see fresh discussion about >>>> similar problems in Mageia Linux distro >>>> http://sourceforge.net/mailarchive/message.php?msg_id=31315478 >>> >>> >>> I think that email thread was correctly answered? >>> >>>> >>>>>> 1. Bring back dependency check with >= condition >>>>> >>>>> >>>>> >>>>> Disagree with this, it will bring issues in the future, and it not a >>>>> best practice. >>>>> >>>>>> 2. We should separate core and plugins requirements >>>>>> 3. We should make possible to run w3af without installation of all >>>>>> plugins >>>>>> dependencies. It can be with special argument to w3af_console called >>>>>> "-l >>>>>> or >>>>>> --lazy". This parameter will force w3af not to check plugins >>>>>> dependencies >>>>>> (or even switch off dependency checker all!). >>>>> >>>>> >>>>> >>>>> You can disable checks for the dependencies which are used in plugins, >>>>> not for the ones in the core or stuff will break in the middle of the >>>>> scan. >>>> >>>> >>>> If default behavior will not be changed why you are still against >>>> disabling it at all by special parameter? This parameter will be used >>>> only >>>> by package maintainers who specifies these dependencies in the package >>>> and >>>> geeks who don't want to install stuff they don't really need. >>> >>> >>> Let me see if I understand, cause now I think I've read it >>> differently. Lets be specific so I don't imagine things: >>> * You will add a --lazy flag to w3af_console and w3af_gui >>> * You will pass the value of --lazy to the dependency_check [0] function >>> * If --lazy is False (the default) things will continue as they are now >>> * If --lazy is True (only if the user specifies that flag) then >>> instead of using strict version checking here [1] you will use ">=" >>> * If --lazy is True you'll user => here [2] >>> * You'll change the console and gtk-UI in such a way that when >>> enabling a plugin that requires a dependency that is not installed, it >>> will tell the user what it is required >>> * You'll make sure that it is possible to run w3af with different >>> versions of plugin dependencies >>> * You'll check that it is possible to run w3af even when some plugin >>> dependencies are not installed >>> * This has automated testing so that in the future I'm sure things >>> will continue to work as expected >>> >>> If that's it, I'm +1 on it! >>> >>> Sorry for not completely understanding your points in the previous email. >>> >>> PS: Still can't believe you'll work on this; I believe it is useless >>> for 95% of the user base. Of course, if you believe it will be useful >>> for you, and it is well coded / tested and doesn't disturb the >>> defaults, I'll merge! >>> >>> [0] >>> https://github.com/andresriancho/w3af/blob/feature/module/w3af/core/controllers/dependency_check/dependency_check.py >>> [1] >>> https://github.com/andresriancho/w3af/blob/feature/module/w3af/core/controllers/dependency_check/dependency_check.py#L68 >>> [2] >>> https://github.com/andresriancho/w3af/blob/feature/module/w3af/core/controllers/dependency_check/helper_script.py >>> >>>> -- >>>> Taras >>>> https://www.oxdef.info >>> >>> >>> >>> >> >> -- >> Taras >> https://www.oxdef.info > > > -- Taras https://www.oxdef.info ------------------------------------------------------------------------------ Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
