Hi Chamila, Great stuff, please send a PR. We will investigate this. For me git clone works fine with authentication.
Thanks On Fri, Feb 13, 2015 at 10:01 AM, Chamila De Alwis <[email protected]> wrote: > Hi Imesh, > > I have almost completed the work to switch to gitpython. However there is > an issue with the type of the auth url, that is given to GitPython. Other > than that, public repo cloning now works with the library. > > When a url with credentials is passed, I'm getting the following error. > Investigating further. > > [2015-02-13 09:59:13,200] ERROR {eventsubscriber.py:run} - Error > processing 'ArtifactUpdatedEvent' event > Traceback (most recent call last): > File > "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/subscriber/eventsubscriber.py", > line 103, in run > handler(event_msg) > File > "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/test-classes/../10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/agent.py", > line 185, in on_artifact_updated > CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj) > File > "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/extensions/extensionhandler.py", > line 122, in on_artifact_updated_event > subscribe_run, updated = > agentgithandler.AgentGitHandler.checkout(repo_info) > File > "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py", > line 89, in checkout > git_repo = AgentGitHandler.clone(git_repo) > File > "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py", > line 192, in clone > Repo.clone_from(unicode(git_repo.repo_url, "utf-8"), > unicode(git_repo.local_repo_path, "utf-8")) > File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line > 844, in clone_from > return cls._clone(Git(os.getcwd()), url, to_path, GitCmdObjectDB, > progress, **kwargs) > File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line > 791, in _clone > v=True, **add_progress(kwargs, git, progress)) > File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 431, in > <lambda> > return lambda *args, **kwargs: self._call_process(name, *args, > **kwargs) > File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 802, in > _call_process > return self.execute(make_call(), **_kwargs) > File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 556, in > execute > **subprocess_kwargs > File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ > errread, errwrite) > File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child > raise child_exception > TypeError: execve() arg 2 must contain only strings > > > Regards, > Chamila de Alwis > Software Engineer | WSO2 | +94772207163 > Blog: code.chamiladealwis.com > > > > On Fri, Feb 13, 2015 at 9:30 AM, Imesh Gunaratne <[email protected]> wrote: > >> I analyzed the issue the NoneType error has occurred due to the EOF >> exception which has been raised while executing git clone shell command: >> >> The problem is in the following line: >> >> git_repo = AgentGitHandler.clone(git_repo) >> AgentGitHandler.log.debug("Git clone executed: [tenant-id] %s [repo-url] %s", >> git_repo.tenant_id, git_repo.repo_url) >> >> Here clone() method returns NoneType if clone fails. Anyway the root cause >> of the problem is with the Git client implementation. Will switch to >> GitPython to avoid this intermittent issue of EOF. >> >> Thanks >> >> >> On Fri, Feb 13, 2015 at 9:20 AM, Imesh Gunaratne <[email protected]> >> wrote: >> >>> I can see the tenant id attribute in received message, may be it was not >>> properly set: >>> >>> [2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} - >>> Message received: instance/notifier/ArtifactUpdatedEvent: >>> >>> {"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":" >>> https://github.com/imesh/stratos-php-applications.git >>> ","tenantId":"-1234","commitEnabled":false} >>> >>> On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <[email protected]> >>> wrote: >>> >>>> Also the following error was there in the log: >>>> >>>> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception >>>> while executing git clone command on git binary : EOF("End Of File (EOF). >>>> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion: >>>> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', ' >>>> https://github.com/imesh/stratos-php-applications.git', >>>> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer >>>> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72) >>>> \\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72), >>>> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index: >>>> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed: >>>> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile: >>>> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase: >>>> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose: >>>> 0.1\ndelayafterterminate: 0.1",) >>>> Traceback (most recent call last): >>>> File >>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py", >>>> line 200, in clone >>>> result = clone_op.expect(["Username for .*", "Password for .*", >>>> "Checking connectivity... done."]) >>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", >>>> line 1451, in expect >>>> timeout, searchwindowsize) >>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", >>>> line 1466, in expect_list >>>> timeout, searchwindowsize) >>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", >>>> line 1554, in expect_loop >>>> raise EOF(str(err) + '\n' + str(self)) >>>> EOF: End Of File (EOF). Exception style platform. >>>> <pexpect.spawn object at 0x25c17d0> >>>> >>>> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <[email protected]> >>>> wrote: >>>> >>>>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH >>>>> has been removed from it. I just fixed it, pushed the new PHP docker image >>>>> to docker hub and did another flow test and found the below error: >>>>> >>>>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error >>>>> processing 'ArtifactUpdatedEvent' event >>>>> Traceback (most recent call last): >>>>> File >>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py", >>>>> line 103, in run >>>>> handler(event_msg) >>>>> File "agent.py", line 185, in on_artifact_updated >>>>> >>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj) >>>>> File >>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py", >>>>> line 122, in on_artifact_updated_event >>>>> subscribe_run, updated = >>>>> agentgithandler.AgentGitHandler.checkout(repo_info) >>>>> File >>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py", >>>>> line 91, in checkout >>>>> git_repo.tenant_id, git_repo.repo_url) >>>>> AttributeError: 'NoneType' object has no attribute 'tenant_id' >>>>> >>>>> [1] >>>>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh >>>>> >>>>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <[email protected]> >>>>> wrote: >>>>> >>>>>> Thanks Lakmal, yes as I found basic features we need works with Git >>>>>> python. Will do that. >>>>>> Regarding the above error, seems like it has occurred due to the >>>>>> APPLICATION_PATH not being sent in payload, there was no None check or a >>>>>> log saying its not found: >>>>>> >>>>>> def clone(git_repo): >>>>>> if os.path.isdir(git_repo.local_repo_path): >>>>>> # delete and recreate local repo path if exists >>>>>> GitUtils.delete_folder_tree(git_repo.local_repo_path) >>>>>> >>>>>> >>>>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> As it looks like this error has been caused by the agent git >>>>>>>> handler while executing git clone, IMO its better if we can switch to >>>>>>>> GitPython. >>>>>>>> >>>>>>>> >>>>>>> +1, If GitPython has full functionality what we expect, lets move to >>>>>>> it. I was under impression that it has some limitation. If so please not >>>>>>> waist time to fix current one. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Devs, >>>>>>>>> >>>>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the below >>>>>>>>> error when trying to receive artifact updated event: >>>>>>>>> >>>>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error >>>>>>>>> processing 'ArtifactUpdatedEvent' event >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File >>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py", >>>>>>>>> line 103, in run >>>>>>>>> handler(event_msg) >>>>>>>>> File "agent.py", line 185, in on_artifact_updated >>>>>>>>> >>>>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj) >>>>>>>>> File >>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py", >>>>>>>>> line 122, in on_artifact_updated_event >>>>>>>>> subscribe_run, updated = >>>>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info) >>>>>>>>> File >>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py", >>>>>>>>> line 89, in checkout >>>>>>>>> git_repo = AgentGitHandler.clone(git_repo) >>>>>>>>> File >>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py", >>>>>>>>> line 190, in clone >>>>>>>>> if os.path.isdir(git_repo.local_repo_path): >>>>>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir >>>>>>>>> st = os.stat(s) >>>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType >>>>>>>>> foun >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Imesh Gunaratne >>>>>>>>> >>>>>>>>> Technical Lead, WSO2 >>>>>>>>> Committer & PMC Member, Apache Stratos >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Imesh Gunaratne >>>>>>>> >>>>>>>> Technical Lead, WSO2 >>>>>>>> Committer & PMC Member, Apache Stratos >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Lakmal Warusawithana >>>>>>> Vice President, Apache Stratos >>>>>>> Director - Cloud Architecture; WSO2 Inc. >>>>>>> Mobile : +94714289692 >>>>>>> Blog : http://lakmalsview.blogspot.com/ >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Imesh Gunaratne >>>>>> >>>>>> Technical Lead, WSO2 >>>>>> Committer & PMC Member, Apache Stratos >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Imesh Gunaratne >>>>> >>>>> Technical Lead, WSO2 >>>>> Committer & PMC Member, Apache Stratos >>>>> >>>> >>>> >>>> >>>> -- >>>> Imesh Gunaratne >>>> >>>> Technical Lead, WSO2 >>>> Committer & PMC Member, Apache Stratos >>>> >>> >>> >>> >>> -- >>> Imesh Gunaratne >>> >>> Technical Lead, WSO2 >>> Committer & PMC Member, Apache Stratos >>> >> >> >> >> -- >> Imesh Gunaratne >> >> Technical Lead, WSO2 >> Committer & PMC Member, Apache Stratos >> > > -- Imesh Gunaratne Technical Lead, WSO2 Committer & PMC Member, Apache Stratos
