Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
Excerpts from Doug Hellmann's message of 2015-10-16 16:22:55 -0400: > Excerpts from Jeremy Stanley's message of 2015-10-16 20:02:15 +: > > On 2015-10-16 15:27:30 -0400 (-0400), Doug Hellmann wrote: > > > Maybe we can combine the ideas. Define an option with a default value > > > based on an environment variable, and then projects can override it if > > > they care about having the doc build generate that link correctly > > > outside of CI. > > > > > > What do I need to do to expose the environment variable? > > > > Within the calling environment for the jobs on our workers, there is > > a $ZUUL_PROJECT variable we can use to get this information. > > However, the job is really just a shell script > > (jenkins/scripts/run-docs.sh in openstack-infra/project-config) > > where we can set any new environment variable oslosphinx cares about > > to that value quite trivially. The envvar name should ultimately > > just be something which makes sense for oslosphinx outside of the CI > > system as well. > > I realized as I went to implement this that because we run through > tox, we would also need to update every tox.ini to allow the > environment variable to be passed through. If we're going to do > that, we might as well just update the conf.py directly. I'll add > an option that can be set there. > > Doug > Here's the change: https://review.openstack.org/236091 Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
Excerpts from Jeremy Stanley's message of 2015-10-16 20:02:15 +: > On 2015-10-16 15:27:30 -0400 (-0400), Doug Hellmann wrote: > > Maybe we can combine the ideas. Define an option with a default value > > based on an environment variable, and then projects can override it if > > they care about having the doc build generate that link correctly > > outside of CI. > > > > What do I need to do to expose the environment variable? > > Within the calling environment for the jobs on our workers, there is > a $ZUUL_PROJECT variable we can use to get this information. > However, the job is really just a shell script > (jenkins/scripts/run-docs.sh in openstack-infra/project-config) > where we can set any new environment variable oslosphinx cares about > to that value quite trivially. The envvar name should ultimately > just be something which makes sense for oslosphinx outside of the CI > system as well. I realized as I went to implement this that because we run through tox, we would also need to update every tox.ini to allow the environment variable to be passed through. If we're going to do that, we might as well just update the conf.py directly. I'll add an option that can be set there. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 2015-10-16 15:27:30 -0400 (-0400), Doug Hellmann wrote: > Maybe we can combine the ideas. Define an option with a default value > based on an environment variable, and then projects can override it if > they care about having the doc build generate that link correctly > outside of CI. > > What do I need to do to expose the environment variable? Within the calling environment for the jobs on our workers, there is a $ZUUL_PROJECT variable we can use to get this information. However, the job is really just a shell script (jenkins/scripts/run-docs.sh in openstack-infra/project-config) where we can set any new environment variable oslosphinx cares about to that value quite trivially. The envvar name should ultimately just be something which makes sense for oslosphinx outside of the CI system as well. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
Excerpts from Jeremy Stanley's message of 2015-10-16 17:49:00 +: > On 2015-10-16 12:59:29 -0400 (-0400), Doug Hellmann wrote: > > Excerpts from Jeremy Stanley's message of 2015-10-16 15:46:33 +: > [...] > > > * You never know what the form of a developer's local origin URL > > > might be, so this is likely locally broken for a lot of people > > > anyway. For example, I usually clone with a .git extension on > > > the URL because I'm a pedant (there are plenty of ways this > > > assumption could go wrong, this is merely one of them). > > > > I wasn't worried about supporting the local build case for this. Do you > > think we need to? > [...] > > I suppose not, unless people testing their documentation changes > locally expect to see that link work. I agree it seems like a bit of > a corner case. Sure, it would be fine if it worked, but it doesn't seem like a priority. > > > > 1. Check some documented envvar like OPENSTACK_PROJECT_NAME and > > > expect it to contain something in the form of > > > /. This way we can explicitly set it in our > > > CI jobs to an already populated variable like ZUUL_PROJECT which > > > contains this information. > > > > That sounds like way more work than is useful for such a small > > feature. > > I mainly bring it up because the project name is known to the build > environment, if only oslosphinx had some means to be told the actual > value rather than having to separately guess it based on less > accurate mechanisms. Maybe we can combine the ideas. Define an option with a default value based on an environment variable, and then projects can override it if they care about having the doc build generate that link correctly outside of CI. What do I need to do to expose the environment variable? > > > > 2. If OPENSTACK_PROJECT_NAME is not set, check an optional > > > extension parameter in doc/source/conf.py so that projects can > > > explicitly set their source URL (this improves downstream > > > consumability of the extension). > > > > Defining a config value and updating every project to set it also seemed > > like more work than the outcome warranted. Defining the option is easy, > > but that's a lot of project patches. > [...] > > However, it does make it possible for projects to use this feature > in oslosphinx even if they're not using our hosting infrastructure. > Again, probably a corner case so not a necessary usage for us to > support I guess. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 2015-10-16 12:59:29 -0400 (-0400), Doug Hellmann wrote: > Excerpts from Jeremy Stanley's message of 2015-10-16 15:46:33 +: [...] > > * You never know what the form of a developer's local origin URL > > might be, so this is likely locally broken for a lot of people > > anyway. For example, I usually clone with a .git extension on > > the URL because I'm a pedant (there are plenty of ways this > > assumption could go wrong, this is merely one of them). > > I wasn't worried about supporting the local build case for this. Do you > think we need to? [...] I suppose not, unless people testing their documentation changes locally expect to see that link work. I agree it seems like a bit of a corner case. > > 1. Check some documented envvar like OPENSTACK_PROJECT_NAME and > > expect it to contain something in the form of > > /. This way we can explicitly set it in our > > CI jobs to an already populated variable like ZUUL_PROJECT which > > contains this information. > > That sounds like way more work than is useful for such a small > feature. I mainly bring it up because the project name is known to the build environment, if only oslosphinx had some means to be told the actual value rather than having to separately guess it based on less accurate mechanisms. > > 2. If OPENSTACK_PROJECT_NAME is not set, check an optional > > extension parameter in doc/source/conf.py so that projects can > > explicitly set their source URL (this improves downstream > > consumability of the extension). > > Defining a config value and updating every project to set it also seemed > like more work than the outcome warranted. Defining the option is easy, > but that's a lot of project patches. [...] However, it does make it possible for projects to use this feature in oslosphinx even if they're not using our hosting infrastructure. Again, probably a corner case so not a necessary usage for us to support I guess. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
Excerpts from Jeremy Stanley's message of 2015-10-16 15:46:33 +: > On 2015-10-16 15:03:57 + (+), Neil Jerram wrote: > [...] > > Surely it must be intended to be browseable? If it was intended as a > > clone URL, shouldn't it be presented in plain text, instead of as a link? > > That's my assumption as well. So given this, it's probably better to > think about our cgit interface URLs as being: > > http(s)://git.openstack.org/cgit// > > Problem here is, _html_page_context() assumes currently that the > origin in your repo will always be in the form of: > > : > > This is a bad assumption for a couple reasons: > > * Depending on how the job is constructed in our CI, the origin > may be reset to a proxied URL which isn't formed in the expected > fashion. In this case you can see it happen in a recent job > log[*], demonstrating this as the underlying source of the > unexpected URL path. OK, I didn't realize we did this. > > * You never know what the form of a developer's local origin URL > might be, so this is likely locally broken for a lot of people > anyway. For example, I usually clone with a .git extension on > the URL because I'm a pedant (there are plenty of ways this > assumption could go wrong, this is merely one of them). I wasn't worried about supporting the local build case for this. Do you think we need to? > > There are probably a few ways to go about tweaking oslosphinx to > correct this. In my mind, for the sake of reusability, it would make > most sense to do something like: > > 1. Check some documented envvar like OPENSTACK_PROJECT_NAME and > expect it to contain something in the form of > /. This way we can explicitly set it in our > CI jobs to an already populated variable like ZUUL_PROJECT which > contains this information. That sounds like way more work than is useful for such a small feature. > > 2. If OPENSTACK_PROJECT_NAME is not set, check an optional > extension parameter in doc/source/conf.py so that projects can > explicitly set their source URL (this improves downstream > consumability of the extension). Defining a config value and updating every project to set it also seemed like more work than the outcome warranted. Defining the option is easy, but that's a lot of project patches. > > 3. If there's no envvar or Sphinx config override, guess from the > origin with a somewhat more thorough regex like > '/([^/]+/[^/]$)(\.git|)$' to be able to omit leading path nodes > and any .git extension. This is also not perfect, but is at > least better than the present situation and will get these links > working correctly on docs.openstack.org for now. > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 2015-10-16 15:46:33 + (+), Jeremy Stanley wrote: [...] > a somewhat more thorough regex like '/([^/]+/[^/]$)(\.git|)$' [...] Heh, mailer regex obviously. ;) I meant '/([^/]+/[^/]+)(\.git|)$' of course (and then take search().group(1) if it succeeds). -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 2015-10-16 15:46:33 + (+), Jeremy Stanley wrote: [...] > Depending on how the job is constructed in our CI, the origin may > be reset to a proxied URL which isn't formed in the expected > fashion. In this case you can see it happen in a recent job > log[*], demonstrating this as the underlying source of the > unexpected URL path. [...] Oops, I forgot to append my footnote log URL: http://logs.openstack.org/5c/5ca6e64e3387bce287e21ce8da3ed03f9537ae7e/post/ironic-docs/a958d35/console.html#_2015-10-16_11_15_46_788 > -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 2015-10-16 15:03:57 + (+), Neil Jerram wrote: [...] > Surely it must be intended to be browseable? If it was intended as a > clone URL, shouldn't it be presented in plain text, instead of as a link? That's my assumption as well. So given this, it's probably better to think about our cgit interface URLs as being: http(s)://git.openstack.org/cgit// Problem here is, _html_page_context() assumes currently that the origin in your repo will always be in the form of: : This is a bad assumption for a couple reasons: * Depending on how the job is constructed in our CI, the origin may be reset to a proxied URL which isn't formed in the expected fashion. In this case you can see it happen in a recent job log[*], demonstrating this as the underlying source of the unexpected URL path. * You never know what the form of a developer's local origin URL might be, so this is likely locally broken for a lot of people anyway. For example, I usually clone with a .git extension on the URL because I'm a pedant (there are plenty of ways this assumption could go wrong, this is merely one of them). There are probably a few ways to go about tweaking oslosphinx to correct this. In my mind, for the sake of reusability, it would make most sense to do something like: 1. Check some documented envvar like OPENSTACK_PROJECT_NAME and expect it to contain something in the form of /. This way we can explicitly set it in our CI jobs to an already populated variable like ZUUL_PROJECT which contains this information. 2. If OPENSTACK_PROJECT_NAME is not set, check an optional extension parameter in doc/source/conf.py so that projects can explicitly set their source URL (this improves downstream consumability of the extension). 3. If there's no envvar or Sphinx config override, guess from the origin with a somewhat more thorough regex like '/([^/]+/[^/]$)(\.git|)$' to be able to omit leading path nodes and any .git extension. This is also not perfect, but is at least better than the present situation and will get these links working correctly on docs.openstack.org for now. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 16/10/15 15:51, Jeremy Stanley wrote: > On 2015-10-16 10:20:17 -0400 (-0400), Doug Hellmann wrote: >> Excerpts from Neil Jerram's message of 2015-10-16 13:10:41 +: > [...] >>> Well, it would appear that 'git config --local --get remote.origin.url' >>> on the docs build machine gives a URL whose path includes a leading /p. >>> But there isn't any /p in the Git URLs currently advertised on >>> https://git.openstack.org/cgit/. > [...] >> Is there a gerrit remote with a better URL? > I'd rather not first jump to conclusions on this. I checked an > arbitrary job workspace on one of the worker nodes and `git config > --local --get remote.origin.url` isn't returning anything with a /p > in it. We should probably step through the process to determine > where in the job that string is actually being set. Sorry, you're right. I did jump too quickly to a conclusion there. > > For starters, what is the goal of that link? Should it be a > browseable version of the repository or a clone URL? If it's > supposed to be browseable, then querying git remote is a bad idea > regardless. Surely it must be intended to be browseable? If it was intended as a clone URL, shouldn't it be presented in plain text, instead of as a link? Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 2015-10-16 10:20:17 -0400 (-0400), Doug Hellmann wrote: > Excerpts from Neil Jerram's message of 2015-10-16 13:10:41 +: [...] > > Well, it would appear that 'git config --local --get remote.origin.url' > > on the docs build machine gives a URL whose path includes a leading /p. > > But there isn't any /p in the Git URLs currently advertised on > > https://git.openstack.org/cgit/. [...] > Is there a gerrit remote with a better URL? I'd rather not first jump to conclusions on this. I checked an arbitrary job workspace on one of the worker nodes and `git config --local --get remote.origin.url` isn't returning anything with a /p in it. We should probably step through the process to determine where in the job that string is actually being set. For starters, what is the goal of that link? Should it be a browseable version of the repository or a clone URL? If it's supposed to be browseable, then querying git remote is a bad idea regardless. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
Excerpts from Neil Jerram's message of 2015-10-16 13:10:41 +: > On 16/10/15 13:53, Christian Berendt wrote: > > On 10/16/2015 01:46 PM, Neil Jerram wrote: > >> I'd be happy to submit a fix for this, but I haven't yet found which > >> project generates this URL. Can someone advise? > > The developer documentations use the oslosphinx > > theme.(http://git.openstack.org/cgit/openstack/oslosphinx). The > > mentioned URL is generated in > > https://github.com/openstack/oslosphinx/blob/master/oslosphinx/__init__.py#L20-L36. > > Probably there is something wrong. > > Well, it would appear that 'git config --local --get remote.origin.url' > on the docs build machine gives a URL whose path includes a leading /p. > But there isn't any /p in the Git URLs currently advertised on > https://git.openstack.org/cgit/. And URLs with /p added don't appear to > be valid. For example: > > neil@nj-ubuntu:~/calico/networking-calico$ git ls-remote > git://git.openstack.org/openstack/ironic | head -15 > 5ca6e64e3387bce287e21ce8da3ed03f9537ae7eHEAD > 7fea52b4d107efcd4ee1570870f6b6c1a7307c10refs/changes/00/115300/1 > 52cfde5dbefdbe619784c3302ea21a198a48be8erefs/changes/00/115300/2 > 7c765279a4eb577d03dfa4e640d008331f67626arefs/changes/00/115300/3 > 11501290b3acdd27cce4eb6a3cea50354492aa95refs/changes/00/115600/1 > f77f01c97585175e2659960c82dc576609656d38refs/changes/00/115600/10 > 34fa5519bd2923caa1b524942e3f30cb6eb85936refs/changes/00/115600/11 > b76ed28657b3dcd6bb322d565728ad485c95688arefs/changes/00/115600/2 > 34eca7eb274f54526e4e0cdb89cfeecffa75e8aarefs/changes/00/115600/3 > c7f3b2591acc143c0fbb08ae9f7cf0fbc6d252cfrefs/changes/00/115600/4 > 7589dd01adbb9a22f921c73da571038b1a662f2frefs/changes/00/115600/5 > e03459f1d9ed1a8f0530acece486834fafc7fd35refs/changes/00/115600/6 > 3e3ace091c6f813b3a074fe4b008a07b33b71bd2refs/changes/00/115600/7 > 9f2516d4e13467a1a45eb4f33884c3cec1b21b44refs/changes/00/115600/8 > 154a97bc36c0758448f2f104ca0eb53276bfda5frefs/changes/00/115600/9 > > neil@nj-ubuntu:~/calico/networking-calico$ git ls-remote > git://git.openstack.org/p/openstack/ironic > fatal: remote error: access denied or repository not exported: > /p/openstack/ironic > > I'm afraid I'm stuck, then. Any thoughts? Is there a gerrit remote with a better URL? Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 16/10/15 13:53, Christian Berendt wrote: > On 10/16/2015 01:46 PM, Neil Jerram wrote: >> I'd be happy to submit a fix for this, but I haven't yet found which >> project generates this URL. Can someone advise? > The developer documentations use the oslosphinx > theme.(http://git.openstack.org/cgit/openstack/oslosphinx). The > mentioned URL is generated in > https://github.com/openstack/oslosphinx/blob/master/oslosphinx/__init__.py#L20-L36. > Probably there is something wrong. Well, it would appear that 'git config --local --get remote.origin.url' on the docs build machine gives a URL whose path includes a leading /p. But there isn't any /p in the Git URLs currently advertised on https://git.openstack.org/cgit/. And URLs with /p added don't appear to be valid. For example: neil@nj-ubuntu:~/calico/networking-calico$ git ls-remote git://git.openstack.org/openstack/ironic | head -15 5ca6e64e3387bce287e21ce8da3ed03f9537ae7eHEAD 7fea52b4d107efcd4ee1570870f6b6c1a7307c10refs/changes/00/115300/1 52cfde5dbefdbe619784c3302ea21a198a48be8erefs/changes/00/115300/2 7c765279a4eb577d03dfa4e640d008331f67626arefs/changes/00/115300/3 11501290b3acdd27cce4eb6a3cea50354492aa95refs/changes/00/115600/1 f77f01c97585175e2659960c82dc576609656d38refs/changes/00/115600/10 34fa5519bd2923caa1b524942e3f30cb6eb85936refs/changes/00/115600/11 b76ed28657b3dcd6bb322d565728ad485c95688arefs/changes/00/115600/2 34eca7eb274f54526e4e0cdb89cfeecffa75e8aarefs/changes/00/115600/3 c7f3b2591acc143c0fbb08ae9f7cf0fbc6d252cfrefs/changes/00/115600/4 7589dd01adbb9a22f921c73da571038b1a662f2frefs/changes/00/115600/5 e03459f1d9ed1a8f0530acece486834fafc7fd35refs/changes/00/115600/6 3e3ace091c6f813b3a074fe4b008a07b33b71bd2refs/changes/00/115600/7 9f2516d4e13467a1a45eb4f33884c3cec1b21b44refs/changes/00/115600/8 154a97bc36c0758448f2f104ca0eb53276bfda5frefs/changes/00/115600/9 neil@nj-ubuntu:~/calico/networking-calico$ git ls-remote git://git.openstack.org/p/openstack/ironic fatal: remote error: access denied or repository not exported: /p/openstack/ironic I'm afraid I'm stuck, then. Any thoughts? Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
On 10/16/2015 01:46 PM, Neil Jerram wrote: > I'd be happy to submit a fix for this, but I haven't yet found which > project generates this URL. Can someone advise? The developer documentations use the oslosphinx theme.(http://git.openstack.org/cgit/openstack/oslosphinx). The mentioned URL is generated in https://github.com/openstack/oslosphinx/blob/master/oslosphinx/__init__.py#L20-L36. Probably there is something wrong. Christian. -- Christian Berendt Cloud Solution Architect Mail: bere...@b1-systems.de B1 Systems GmbH Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all][docs] Incorrect '/p' in Project Source URLs on docs.openstack.org/developer
If you go to http://docs.openstack.org/developer/openstack-projects.html, then go from there into any project page, the 'Project Source' URL in the sidebar is wrong. (Well, I say 'any', but I've only actually tried for ironic, networking-midonet and networking-calico. Those all show the following problem.) For example, at http://docs.openstack.org/developer/ironic/ the URL is http://git.openstack.org/cgit/p/openstack/ironic, but it should be http://git.openstack.org/cgit/openstack/ironic. I'd be happy to submit a fix for this, but I haven't yet found which project generates this URL. Can someone advise? Regards, Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
