Taras, On Sun, Feb 16, 2014 at 4:28 PM, Taras <ox...@oxdef.info> wrote: > Andres, > > I think it is my last attempt to change your opinion :) > > From the list of software you have provided I have found only flask, scrapy > and tastypie in Ubuntu repo. Results of "apt-cache show" output are below > inline. > The problem is w3af built-in dependency checker duplicates > OS (e.g. Debian/Ubuntu) packaging system. They can conflict in > some cases. > > For example, I want to make package of w3af for Ubuntu 13.10. > There is package python-xml version 3.2.0 in repository. At the same time > w3af requires lxml version exactly 2.3.2. How can I make package of w3af? > Should I add "sudo pip install" into preinstall script?
Most likely not, that doesn't sound well. I don't know the right answer because I'm not packaging expert. The package maintainer can always apply a patch on top of the original software to remove the dependency check completely (I think Luciano did something like this [0]) is he believes it is the best thing to do. Then he's taking the responsibility of that change. My responsibility is to tell you that with these specific package versions it works; then people do whatever they want with it. [0] http://packages.ubuntu.com/precise/w3af-console - search for "diff" > Have you got any feedback from w3af package maintainers for Debian/Ubuntu > and other distributions after you had add strict dependencies? There are no active package maintainers for w3af. They even don't care, or don't want to maintain this software; so no, no package maintainer told me anything about the "==". As I said above, they can apply a diff to the software before packaging it, as done by Luciano a while ago (not only for the dependency). > Is it > important for you that w3af can be installed via simple command "apt-get > install w3af" or through Ubuntu Software Center with single mouse click? Yes, and not. Some users would find it awesome to be able to install it from the repo; but this has proven to be (at least for w3af) a failed path. I'm not going to maintain a package for each distribution, because I don't care enough as a user myself. Packagers who have come to the project have either failed to release their initial package or released it and then moved their free time to something else. In this process, they left very old versions of w3af in the repositories of all linux distributions; which don't even make sense for users. If users can install w3af with: git clone ... cd w3af ./w3af_console # Yields error with all dependencies to install /tmp/install_w3af_dependencies.sh Then I'm happy. > If it is important for you then I recommend to add maintainers into this > discussion and ask if it is easy for them to make package of w3af with such > requirements. My opinion is that they don't care about the w3af package. > If it is not so important and "git clone + pip install" is preferable way of > installation then thread can be closed. In the past I've thought that having w3af in the linux distribution repos was THE BEST THING, now... not so much, because: * Software packages are difficult to maintain * Each time a new dependency is added the maintainer needs to create a new package for that (python-foo) and then maintain that one also * The whole process takes time, so from the minute I put something in the repo to the time the new package is there it can be months; and "hackers" love to use the latest and they will come to the repo anyways > > >>>> Not 100% a workaround, this is also a best practice! >>>> >>>> https://devcenter.heroku.com/articles/python-pip#the-basics >>> >>> >>> >>> Could you please show at least one example of well-know software with >>> such >>> requirements? >> >> >> I went through this list of the Top10 Python projects by github (not >> sure how they choose that) and found many that either had no >> dependencies or were not in a format in which we could compare them >> with what we were talking about. Then found the following: >> >> * Strict dependencies used for this part of the project: >> https://github.com/torchbox/wagtail/blob/master/requirements-dev.txt >> * Gt used for the user installable part: >> https://github.com/torchbox/wagtail/blob/master/setup.py >> >> * These guys install whatever is available on pypi: >> https://github.com/jmcarp/robobrowser/blob/master/requirements.txt >> >> * Flask installs Gt: >> https://github.com/mitsuhiko/flask/blob/master/setup.py > > Depends: python-itsdangerous, python (>= 2.7), python-jinja2 (>= 2.4), > python (<< 2.8), python-werkzeug (>= 0.8) > Recommends: python-pkg-resources, python-blinker > > >> * A mix between Gt and "whatever" is used here: >> https://github.com/Eugeny/ajenti/blob/dev/requirements.txt >> >> * Scrapy uses a mix of GT and "whatever": >> https://github.com/scrapy/scrapy/blob/master/requirements.txt > > > Depends: python2.7, python (>= 2.7.1-0ubuntu2), python (<< 2.8), > python-twisted-core, python-twisted-web, python-twisted-conch, > python-twisted-mail, python-libxml2, python-boto, python-w3lib > Recommends: python-lxml, python-guppy, python-django, ipython, > python-pygments, python-imaging, python-mysqldb > > >> * Django-tastypie uses the most complex of them all, which is rather >> interesting and makes me wonder why they didn't use "==" instead: >> https://github.com/toastdriven/django-tastypie/blob/master/setup.py . >> This is what I mean: 'dateutil(>=1.5, !=2.0)' >> > Replaces: python-django-tastypie (<= 0.9.9-2) > Depends: python (>= 2.7.1-0ubuntu2), python (<< 2.8), python-mimeparse (>= > 0.1.3), python-dateutil (>= 1.5), python-django (>= 1.2) > Suggests: python-yaml, python-lxml > I get your point, >= seems to be the preferred way of doing it in the debian repos. If a packager wants, he can do that with w3af and apply a patch to disable dependency check for w3af in the packaging process. That way he's happy, we don't need to code anything and are also happy. >> The first one is an example of "==", the rest were just to show that >> now everyone agrees with me on what should be put on the >> requirements.txt file (or the setup.py, which acts like the same many >> times). >> >> Here are some other links where it says that "==" is a best practice: >> * >> https://lincolnloop.com/django-best-practices/deployment/bootstrap.html >> (Ctrl+f "Pin your dependencies") >> * >> http://docs.dotcloud.com/tutorials/python/django/#specifying-requirements >> (Ctrl+f "When you specify your requirements") >> >> And most importantly, the pip-installer user's guide: >> * >> http://www.pip-installer.org/en/latest/user_guide.html#ensuring-repeatability >> >> "The requirements file was generated by pip freeze or you're sure it >> only contains requirements that specify a specific version." >> >> When we're talking about including a specific version in >> requirements.txt file or not, we're talking about repeatability. I >> want to be strict about repeatability, forcing all libraries to be >> exactly the ones I know will work because I've tested them in the CI; >> and your point is that it would be easier for users to install with >> less strict version requirements (which could lead to issues in some >> cases). >> >> Sadly, you believe in one thing and I can't seem to convince you of >> the benefits of ==, and the same applies the other way (I can't be >> convinced of the benefits of >=). Unless I hear a definitive reason on >> why == is bad, I won't change it. >> >>> By the way in w3af dev list I see fresh discussion about >>> similar problems in Mageia Linux distro >>> http://sourceforge.net/mailarchive/message.php?msg_id=31315478 >> >> >> I think that email thread was correctly answered? >> >>> >>>>> 1. Bring back dependency check with >= condition >>>> >>>> >>>> >>>> Disagree with this, it will bring issues in the future, and it not a >>>> best practice. >>>> >>>>> 2. We should separate core and plugins requirements >>>>> 3. We should make possible to run w3af without installation of all >>>>> plugins >>>>> dependencies. It can be with special argument to w3af_console called >>>>> "-l >>>>> or >>>>> --lazy". This parameter will force w3af not to check plugins >>>>> dependencies >>>>> (or even switch off dependency checker all!). >>>> >>>> >>>> >>>> You can disable checks for the dependencies which are used in plugins, >>>> not for the ones in the core or stuff will break in the middle of the >>>> scan. >>> >>> >>> If default behavior will not be changed why you are still against >>> disabling it at all by special parameter? This parameter will be used >>> only >>> by package maintainers who specifies these dependencies in the package >>> and >>> geeks who don't want to install stuff they don't really need. >> >> >> Let me see if I understand, cause now I think I've read it >> differently. Lets be specific so I don't imagine things: >> * You will add a --lazy flag to w3af_console and w3af_gui >> * You will pass the value of --lazy to the dependency_check [0] function >> * If --lazy is False (the default) things will continue as they are now >> * If --lazy is True (only if the user specifies that flag) then >> instead of using strict version checking here [1] you will use ">=" >> * If --lazy is True you'll user => here [2] >> * You'll change the console and gtk-UI in such a way that when >> enabling a plugin that requires a dependency that is not installed, it >> will tell the user what it is required >> * You'll make sure that it is possible to run w3af with different >> versions of plugin dependencies >> * You'll check that it is possible to run w3af even when some plugin >> dependencies are not installed >> * This has automated testing so that in the future I'm sure things >> will continue to work as expected >> >> If that's it, I'm +1 on it! >> >> Sorry for not completely understanding your points in the previous email. >> >> PS: Still can't believe you'll work on this; I believe it is useless >> for 95% of the user base. Of course, if you believe it will be useful >> for you, and it is well coded / tested and doesn't disturb the >> defaults, I'll merge! >> >> [0] >> https://github.com/andresriancho/w3af/blob/feature/module/w3af/core/controllers/dependency_check/dependency_check.py >> [1] >> https://github.com/andresriancho/w3af/blob/feature/module/w3af/core/controllers/dependency_check/dependency_check.py#L68 >> [2] >> https://github.com/andresriancho/w3af/blob/feature/module/w3af/core/controllers/dependency_check/helper_script.py >> >>> -- >>> Taras >>> https://www.oxdef.info >> >> >> >> > > -- > Taras > https://www.oxdef.info -- Andrés Riancho Project Leader at w3af - http://w3af.org/ Web Application Attack and Audit Framework Twitter: @w3af GPG: 0x93C344F3 ------------------------------------------------------------------------------ Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
