Hi akuma-San,
On 2016年11月08日 12:58, fumihiko kakuma wrote: > Hi Iwase-san, > > On Tue, 8 Nov 2016 09:51:18 +0900 > Iwase Yusuke <[email protected]> wrote: > >> Hi Kakuma-San, >> >> Thank you for your reply! >> >> >> On 2016年11月07日 21:50, fumihiko kakuma wrote: >>> Hi Iwase-san, >>> >>> I am sorry for the delay in my reply. >>> >>> On Mon, 7 Nov 2016 16:08:09 +0900 >>> Iwase Yusuke <[email protected]> wrote: >>> >>>> Hi Kakuma-San, >>>> >>>> Could you check my answer as following? >>>> This patch will fix the Travis-CI build error of Ryu upstream. >>>> >>> >>> I also was checking it and I found that it is caused by the fact that >>> tinyrpc is not installed. >> >> Yes, the embedded tinyrpc has been deleted in the latest Ryu, >> but docker_base.py does not install it yet. >> >> >>> I answered in inline. >>> >>>> Thanks, >>>> Iwase >>>> >>>> >>>> On 2016年11月06日 14:28, Iwase Yusuke wrote: >>>>> Hi, Kakuma-San, >>>>> >>>>> >>>>> On 2016年11月04日 06:52, fumihiko kakuma wrote: >>>>>> Hi Iwase-san, >>>>>> >>>>>> On Wed, 2 Nov 2016 16:51:35 +0900 >>>>>> IWASE Yusuke <[email protected]> wrote: >>>>>> >>>>>>> Signed-off-by: IWASE Yusuke <[email protected]> >>>>>>> --- >>>>>>> ryu/tests/integrated/common/docker_base.py | 23 +++++++++++++---------- >>>>>>> 1 file changed, 13 insertions(+), 10 deletions(-) >>>>>>> >>>>>>> diff --git a/ryu/tests/integrated/common/docker_base.py >>>>>>> b/ryu/tests/integrated/common/docker_base.py >>>>>>> index 13fe223..fd3ceef 100644 >>>>>>> --- a/ryu/tests/integrated/common/docker_base.py >>>>>>> +++ b/ryu/tests/integrated/common/docker_base.py >>>>>>> @@ -196,25 +196,28 @@ class DockerImage(object): >>>>>>> def create_ryu(self, tagname='ryu', image=None, check_exist=False): >>>>>>> if check_exist and self.exist(tagname): >>>>>>> return tagname >>>>>>> - workdir = TEST_BASE_DIR + '/' + tagname >>>>>>> + workdir = '%s/%s' % (TEST_BASE_DIR, tagname) >>>>>>> workdir_ctn = '/root/osrg/ryu' >>>>>>> - pkges = 'telnet tcpdump iproute2 ' >>>>>>> - pkges += 'gcc python-pip python-dev libffi-dev libssl-dev' >>>>>>> + pkges = 'telnet tcpdump iproute2' >>>>>>> + pkges += ' gcc python-dev libffi-dev libssl-dev' >>>>>>> + pkges += ' libxml2-dev libxslt1-dev zlib1g-dev' >>>>>>> if image: >>>>>>> use_image = image >>>>>>> else: >>>>>>> use_image = self.baseimage >>>>>>> c = CmdBuffer() >>>>>>> - c << 'FROM ' + use_image >>>>>>> + c << 'FROM %s' % use_image >>>>>>> c << 'RUN apt-get update' >>>>>>> - c << 'RUN apt-get install -qy --no-install-recommends ' + pkges >>>>>>> - c << 'RUN pip install -U six paramiko msgpack-rpc-python' >>>>>>> - c << 'ADD ryu ' + workdir_ctn >>>>>>> - install = 'RUN cd %s && pip install -r tools/pip-requires ' % >>>>>>> workdir_ctn >>>>>>> - install += '&& python setup.py install' >>>>>>> + c << 'RUN apt-get install -qy --no-install-recommends %s' % >>>>>>> pkges >>>>>>> + c << 'ADD ryu %s' % workdir_ctn >>>>>>> + install = 'RUN cd %s' % workdir_ctn >>>>>>> + install += ' && pip install' >>>>>>> + install += ' -r tools/pip-requires' >>>>>>> + install += ' -r tools/test-requires' >>>>>> >>>>>> Why do we need the test-requires to run a ryu app on a container? >>>>>> I think that we may need a pip-requires which some packages are added or >>>>>> an another required file. >>>>> >>>>> test-requires also includes the optional requirements (e.g. paramiko, >>>>> tinyrpc) >>>>> to run BGPSpeaker and OF-Config, for example. >>>>> >>>>> http://ryu.readthedocs.io/en/latest/getting_started.html#optional-requirements >>>>> >>>>> So, when we want to test the other feature on osrg/ryu based Docker >>>>> container, >>>>> we must have installed them, I guess. >>> >>> Yes, but I didn't like to install test tool(mock, nose and etc). >> >> I agree with you. >> I don't prefer to installing unnecessary packages too. >> >> Well, I would like to separate test-requires into two files("test >> requirements" and "optional requirements"). >> > > Oh, it's greate! > >> # Removes "optional requirements" from test-requires. >> $ cat tools/test-requires >> coverage >> mock >> nose >> pep8 >> pylint >> formencode >> >> # Create a new file for "optional requirements". >> $ cat tools/optional-requires >> lxml; platform_python_implementation != 'PyPy' # OF-Config >> lxml==3.4.0; platform_python_implementation == 'PyPy' >> cryptography==1.5 >> paramiko # NETCONF, BGP speaker >> tinyrpc # RPC >> ncclient # OF-Config >> >> >>> >>>>> I prefer to separating the optional requirements from "test-requires", >>>>> though... >>>>> >>> >>> But I am ok in using test-requires if it makes many works. >>> >>>>>> >>>>>> And the above Dockerfile seems to faile in a case not to use >>>>>> osrg/ryu image. But we may not need that case. >>>>> >>>>> You mean the above does not have "python-pip", for example? >>> >>> >>> Yes. And I get the following error while I build ryu image file. >>> >>> $ cat /tmp/ctn_docker/bgp/ryu/Dockerfile >>> FROM ubuntu:14.04.5 >>> RUN apt-get update >>> RUN apt-get install -qy --no-install-recommends telnet tcpdump iproute2 gcc >>> python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev zlib1g-dev >>> python-pip >>> ADD ryu /root/osrg/ryu >>> RUN cd /root/osrg/ryu && pip install -r tools/pip-requires -r >>> tools/test-requires && python setup.py install >>> $ sudo docker build -t ryu /tmp/ctn_docker/bgp/ryu >>> Sending build context to Docker daemon 179.3 MB >>> Step 1 : FROM ubuntu:14.04.5 >>> ---> f2d8ce9fa988 >>> Step 2 : RUN apt-get update >>> ---> Running in 90d7f0bb0ef3 >>> Ign http://archive.ubuntu.com trusty InRelease >>> Get:1 http://archive.ubuntu.com trusty-updates InRelease [65.9 kB] >>> >>> ... >>> >>> Removing intermediate container 384b0764eeaf >>> Step 4 : ADD ryu /root/osrg/ryu >>> ---> ee57cfffe5d9 >>> Removing intermediate container 1adb83f316e9 >>> Step 5 : RUN cd /root/osrg/ryu && pip install -r tools/pip-requires -r >>> tools/test-requires && python setup.py install >>> ---> Running in 9de973780048 >>> Exception: >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in >>> main >>> status = self.run(options, args) >>> File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line >>> 262, in run >>> for req in parse_requirements(filename, finder=finder, options=options, >>> session=session): >>> File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1632, in >>> parse_requirements >>> req = InstallRequirement.from_line(line, comes_from, >>> prereleases=getattr(options, "pre", None)) >>> File "/usr/lib/python2.7/dist-packages/pip/req.py", line 173, in from_line >>> return cls(req, comes_from, url=url, prereleases=prereleases) >>> File "/usr/lib/python2.7/dist-packages/pip/req.py", line 71, in __init__ >>> req = pkg_resources.Requirement.parse(req) >>> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2667, in >>> parse >>> reqs = list(parse_requirements(s)) >>> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2605, in >>> parse_requirements >>> line, p, specs = scan_list(VERSION,LINE_END,line,p,(1,2),"version spec") >>> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2573, in >>> scan_list >>> raise ValueError("Expected "+item_name+" in",line,"at",line[p:]) >>> ValueError: ('Expected version spec in', "lxml; >>> platform_python_implementation != 'PyPy'", 'at', "; >>> platform_python_implementation != 'PyPy'") >>> >>> Storing debug log for failure in /root/.pip/pip.log >>> The command '/bin/sh -c cd /root/osrg/ryu && pip install -r >>> tools/pip-requires -r tools/test-requires && python setup.py install' >>> returned a non-zero code: 2 >>> $ >> >> The above error show the pip version is too old, I guess. >> The packaged python-pip on Ubuntu 14.04 might be old to use >> the environment marker in the requirements file. >> https://www.python.org/dev/peps/pep-0345/#environment-markers >> > > Thank you for your explanation. > >> >>> >>>>> Well, is there any case or request for using the other base image? >>>>> If any, I guess we can use the Ubuntu based image only... >>>>> >>> >>> In the default DockerImage class uses ubuntu 14.04.5. >>> I think that there may be a case try multiple ubuntu version and >>> the latest ubuntu version and use debian. >>> But I don't know whether we try those cases. >>> >>> BTW how about having a separate method like the following? >>> >>> def create_ryu(self, tagname='ryu', image=None, check_exist=False): >>> ... >>> >>> def create_ryu_osrg(self, tagname='ryu', image='osrg/ryu', >>> check_exist=False): >>> ... >> >> Hummm... I don't want to separate method for the specific case... >> > > I don't like that. But I thought that in case osrg/ryu and create from > the begining we may need a different procedure. > If we have the optional requirements file, we will not need separate > method. > >> With Ubuntu 16.04, the packaged python-pip version is enough to >> use the environment marker, so this problem will not occur. >> But, if we should consider the multiple Ubuntu version, >> I will update my patch to include python-pip. > > Pehaps when the latest ubuntu(for example 18.04) is released, we > will face the same problem. What exactly "the same problem" means? Well, if on Ubuntu 18.04, the packaged pip should be the latest or stable at the time of its release, we will never face this problem, I guess. BTW, with my v2 patch, we can remove the environment marker in the new requirement files and be free from this problem. Thanks, Iwase > I think that we will can use get-pip.py > > # apt-get install curl > # curl -k -f --retry 6 -o get-pip.py https://bootstrap.pypa.io/get-pip.py > # python get-pip.py > # pip --version > pip 9.0.1 from /usr/local/lib/python2.7/dist-packages (python 2.7) > # > >> >> Thanks, >> Iwase >> >> >>> >>> >>> Thank you, >>> fumihiko kakuma >>> >>> >>>>> Thanks, >>>>> Iwase >>>>> >>>>>> >>>>>>> + install += ' && python setup.py install' >>>>>>> c << install >>>>>>> >>>>>>> - self.cmd.sudo('rm -rf ' + workdir) >>>>>>> + self.cmd.sudo('rm -rf %s' % workdir) >>>>>>> self.cmd.execute('mkdir -p ' + workdir) >>>>>>> self.cmd.execute("echo '%s' > %s/Dockerfile" % (str(c), >>>>>>> workdir)) >>>>>>> self.cmd.execute('cp -r ../ryu %s/' % workdir) >>>>>>> -- >>>>>>> 2.7.4 >>>>>>> >>>>>>> >>>>>>> ------------------------------------------------------------------------------ >>>>>>> Developer Access Program for Intel Xeon Phi Processors >>>>>>> Access to Intel Xeon Phi processor-based developer platforms. >>>>>>> With one year of Intel Parallel Studio XE. >>>>>>> Training and support from Colfax. >>>>>>> Order your platform today. http://sdm.link/xeonphi >>>>>>> _______________________________________________ >>>>>>> Ryu-devel mailing list >>>>>>> [email protected] >>>>>>> https://lists.sourceforge.net/lists/listinfo/ryu-devel >>>>>> >>> > ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi _______________________________________________ Ryu-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ryu-devel
