Re: [Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
Hi 8--snip--- I have updated the plugin repository wiki page and tried to dump as much as possible from the ideas and requirements: http://www.qgis.org/wiki/PluginRepository Ponies!?? :-) Heheh cant wait to see how the portal is going to look with this kind of inspiration :-) I was also thinking about a few other apps to go under our django project: - python snippets - c++ snippets - style repository - symbol repository (svgs / pngs / ttfs) If we can make them all in pink that would be cool too :-) I know above are future things but it would be nice to bear in mind where we are going as we build the plugin repo app so that we can maximise code re-use. Regards Tim 8--snip--- -- Tim Sutton - QGIS Project Steering Committee Member (Release Manager) == Please do not email me off-list with technical support questions. Using the lists will gain more exposure for your issues and the knowledge surrounding your issue will be shared with all. Visit http://linfiniti.com to find out about: * QGIS programming and support services * Mapserver and PostGIS based hosting plans * FOSS Consulting Services Skype: timlinux Irc: timlinux on #qgis at freenode.net == ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
On 11/19/2010 03:00 PM, Martin Dobias wrote: On Wed, Nov 17, 2010 at 12:22 PM, Tim Sutton li...@linfiniti.com wrote: Hi On Tue, Nov 16, 2010 at 12:23 AM, Alex Mandel tech_...@wildintellect.com wrote: On 11/15/2010 01:23 PM, Borys Jurgiel wrote: So change of mind from our discussion on IRC yesterday? Should Pirmin and I still bother with the test install of Redmine? In the discussion at the hackfest (following the irc discussion) we felt that using trac would be a lower threshold to cross - a number of us know it (as admins), as do the users. Personally I'm not averse to using redmin so playing around on a test instance would be nice) but I think we should set some kind of cut off date and make a decision - Paolo and others are getting frustrated trying to manage plugin issues so it would be nice to get something in place as soon as possible. Right - trac is a good option because users know it, developers know it, some people even know how to administrate it :-) So unless other bug tracking system comes with some features that we seriously need and Trac does not have them, we should probably stick to it. Ok I think Pirmin broke it down better than I in his follow up post, but lets try to separate discussion about the management of the codebase for the qgis-django project which among other things will provide the new plugin portal, from the plugins themselves. In the case of qgis-django portal, the suggestion was to completely manage it using github infrastructure. Right, there are two separate things from my point of view: plugin repository and ticket system for plugins. While we can choose from various available bug tracking systems, the requirements for the plugin repository are quite unique and I'm not aware of any existing software that could be used for it. I have updated the plugin repository wiki page and tried to dump as much as possible from the ideas and requirements: http://www.qgis.org/wiki/PluginRepository #6 I assume this SVN integrated with the New plugin bug trac site is ok? Did you still want the ability to set permissions by folder on the svn (this is doable). Yes the idea was that each plugin author would be given a folder that only they (or team members) could write to. Its an admin overhead (for you?) but would be the idea - liberal access so that anyone can have an svn write account but they can only write to their own dir. My opinion is that we simply shouldn't provide a common svn/git repository for plugin authors. Why try to provide source code hosting for 3rd parties if there are lots of easily accessible free source code hosting services? There is Google Project Hosting (SVN/Mercurial), SourceForge (SVN/CVS/Git/Mercurial/Bazaar), BitBucket (SVN/Mercurial), GitHub and Gitorious (Git) and many others. These services allow users to create their code hosting in one minute and what is most important - with no admin overhead for QGIS project. We could just give the plugin authors some hints which hostings are the best. Additionally the free hostings usually provide bug tracking systems and other services, so we could possibly even skip the part with setting up a common bug tracking system for 3rd party plugins. Because if a plugin author is not willing to use the common tracking system, the number of tickets will increase without being ever resolved. Regards Martin I feel that a central option would help new plugin authors and authors who want collaborate on plugins. I've created a sourceforge site for my plugins in the past and attracted zero developers and only had a handful of bugs put into my trac. Aside from that, a central place lets us easily adopt abandoned plugins, and to give 1 place for reporting bugs which in theory would improve the reporting and tracking of fixes. I don't think we have to host it, but if we don't I would like to see us pick one of the 3rd party hosting services and create a collaborative collective under which people list their plugins(a qgis group). A large number of plugins right now aren't even under version control and even fewer of them have public tickets systems. So when I go to hack on a plugin(that isn't mine) I never know if I have the latest code or if it's been fixed and I don't expect an update to the release for every little fix. I also have no idea if it's still under development, or abandoned in favor of something else, merged into another plugin, etc. I actually see a bug tracker also serving as a quality check. If like you say a lot of bugs get filed for a plugin but never get solved, it gives us the opportunity to ask if the plugin is dead, and if so to change it's xml so users realize it's no longer being developed. Or to calculate some measure of the plugins likelihood to work. It's also an opt in thing, a plugin won't automatically be in the list of things to report against. As for the Trac vs. Redmine thing, yes we are specifically looking at Redmine
Re: [Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
On 11/19/2010 04:43 PM, Martin Dobias wrote: Hi Alex On Sat, Nov 20, 2010 at 12:44 AM, Alex Mandel tech_...@wildintellect.com wrote: I feel that a central option would help new plugin authors and authors who want collaborate on plugins. I've created a sourceforge site for my plugins in the past and attracted zero developers and only had a handful of bugs put into my trac. Aside from that, a central place lets us easily adopt abandoned plugins, and to give 1 place for reporting bugs which in theory would improve the reporting and tracking of fixes. I don't think we have to host it, but if we don't I would like to see us pick one of the 3rd party hosting services and create a collaborative collective under which people list their plugins(a qgis group). Picking a 3rd party hosting service seems like an option. Anyway there were such motions before, but did not really succeed: http://sourceforge.net/projects/qgiscommunitypl/ The biggest con is that we can't use osgeo logins if it's 3rd party. This to me is a big barrier to getting people using it. So picking a place a bunch of us already use would be good. We could speculate more about why that one didn't pan out (had forgotten about it) but I think the important parts are that there needs to be some buy-in (which I feel we have now that we've discussed the options), some ability for qgis core devs to admin over it (for rescue purposes), and more visibility like obvious linkage from qgis.org and in this case the new plugin site that's being built. My guess is a survey amongst current plugin and core devs would have them pick github 1st and maybe googlecode next with bitbucket/launchpad/sourceforge on the least favorite end. I also think Paolo is really excited about a central place to report and track issues. His team seems to do most of it right now and I see how other users searching the internet could really benefit from knowing where to look/getting them in search results. Then it just becomes a cultural thing to convince devs to pay attention to it. A large number of plugins right now aren't even under version control and even fewer of them have public tickets systems. So when I go to hack on a plugin(that isn't mine) I never know if I have the latest code or if it's been fixed and I don't expect an update to the release for every little fix. I also have no idea if it's still under development, or abandoned in favor of something else, merged into another plugin, etc. I just have the impression that the plugin developers mostly will not use that service. As you say, lots of plugins don't have any versioning system (or just a private one) and maybe that's because the authors do not need it for their work - more than 90% of the plugins are one-man shows. I actually see this as a potential issue going forward as more and more people rely on plugins to get stuff done and there is potential to incorporate some plugins that offer central functionality into the core. A Bus number of 1 is kinda scary and while they may be one man shows that might be because it too much effort to solicit help (I've heard this from some developers out there over the years, 'I didn't open up the code or ask for collaboration because it was too much extra work'). Lack of version control is a little frightening too (Carson lost all his original work on ftools at one point because he wasn't using version control, I think he recovered from the already released code). My goal is to better enable collaboration and encourage good open source programming practices. In the new plugin repository we would like to keep track of plugin updates - so determining whether the plugin is dead should be simple. Regards Martin Example, someone files a ticket with a patch to fix a major bug(feature) in a plugin. Said plugin appears to be dead. So someone else grabs the patch, forks the code, applies the patch and releases the plugin update. I'm open to trying github right now before we go and install stuff if we can figure out how to group stuff. I get the feeling the issue tracking won't be super central but it's really easy to try it 1st before we start installing stuff on the servers. Thanks, Alex ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
Hi On Tue, Nov 16, 2010 at 11:39 AM, Pirmin Kalberer pi...@sourcepole.com wrote: -8-snip-- I would group the aspects in 1. Package repository + Package download website Right lets call this 'the plugin portal' to be developed using django + git + github for issue tracking. 2. Plugin source code repository Correct. Proposal from some is to provide an svn with liberal access (anyone can get a committer account) with each user given write access only to their project folder. Some in the discussion have suggested we just skip this part an focus on 1 3. Using git instead of svn for this part would also be fine though the concensus seemed to be that people are more likely to be familiar with svn and that they can just create github accounts if they want to use git. 3. Bug tracking + Plugin home pages Right. Bug tracking seems to me to be where the main contention lies (between trac or redmine). Personally I dont have a strong opinion either way other than being familiar with trac and unfamiliar with redmine. For plugin home pages - isnt this something that can be provided as part of the plugin portal? Should be easy to do? I think we need to just pick a set of technologies that a) offers the best functionality as individual parts b) are easily integrated and c) that will be most easily adapted to by plugin authors. I say 'just' in my previous sentance though socially it seems difficult to actually make the choice between such a mixed bag of technologies, so maybe it should come down to 'he who is prepared to maintain it has the strongest argument'. In which case since Alex is prepared to put in the work on the insfrastructure side for the plugins versioning and issue tracker we should just let him set something up and go with it. For the qgis portal side I think everything is catered for with github and we can just get on with it. Regards Tim Borys and Martin proposed a solution for point 1, developed with Django. They did not couple the package repository with the source code repository to let plugin developers choose their SCM. Tim's announcement includes the following solutions for these aspects: 1. Django application 2. One(?) Github repository 3. One(?) Trac instance In the discussion Alex is referring to, we came to this: 1. not discussed 2. OSGeo Git server 3. Redmine We didn't go deeper into point 2 and number 3 was not confirmed by Tim. I have absolutely no problems with Tim's solution and I'm sure he will clarify open points as soon as he arrives in South Africa. Regards Pirmin -- Pirmin Kalberer Sourcepole - Linux Open Source Solutions http://www.sourcepole.com ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Tim Sutton - QGIS Project Steering Committee Member (Release Manager) == Please do not email me off-list with technical support questions. Using the lists will gain more exposure for your issues and the knowledge surrounding your issue will be shared with all. Visit http://linfiniti.com to find out about: * QGIS programming and support services * Mapserver and PostGIS based hosting plans * FOSS Consulting Services Skype: timlinux Irc: timlinux on #qgis at freenode.net == ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
Am Montag, 15. November 2010, um 22.23:59 schrieb Borys Jurgiel: So change of mind from our discussion on IRC yesterday? Should Pirmin and I still bother with the test install of Redmine? Actually I'm still a bit confused and don't want to take one of the two sides (mainly because I only looked briefly at the Redmine), but let's just write down the disadvantages and threats of the two solutions. [...] In my opinion it's not a discussion between Django and Redmine. The confusion comes from not discussing all aspects in one session (yes, people were too tired). I would group the aspects in 1. Package repository + Package download website 2. Plugin source code repository 3. Bug tracking + Plugin home pages Borys and Martin proposed a solution for point 1, developed with Django. They did not couple the package repository with the source code repository to let plugin developers choose their SCM. Tim's announcement includes the following solutions for these aspects: 1. Django application 2. One(?) Github repository 3. One(?) Trac instance In the discussion Alex is referring to, we came to this: 1. not discussed 2. OSGeo Git server 3. Redmine We didn't go deeper into point 2 and number 3 was not confirmed by Tim. I have absolutely no problems with Tim's solution and I'm sure he will clarify open points as soon as he arrives in South Africa. Regards Pirmin -- Pirmin Kalberer Sourcepole - Linux Open Source Solutions http://www.sourcepole.com ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
Hi all There has been a lot of discussion about the way forward for plugins, the plugin repository, plugin source management and plugin issue tracking / bug reporting in the last few weeks. Sitting in the same room for the last few days, we have also been able to go into more detail about our requirements. We would like to summarise our requirements and plans and get things going. So here is the synopsis: 1) We will set up a django instance on the QGIS.org virtual machine (using a python vitual environment) for building the new QGIS Plugin Repository. 2) The code for the repository will be managed via GIT and is hosted on github. If you need write access, please provide me with your github user name. Details for the repo here: https://github.com/timlinux/qgis-django 3) The management of issues for the repository itself will be done using github infrastructure. 4) For the plugins themselves, we are going to request that OSGEO SAC provide us with a trac instance dedicated for plugin bugs. It should be configured to authenticate users via LDAP so that existing OSGEO users will not need to create another account. 5) For the management of plugin source code, plugin authors will have the option of hosting their own project code (e.g. on github, sourceforge etc). 6) For plugin authors who wish to use an 'official' source repository, we will put in an OSGEO SAC request to set up an SVN instance for QGIS plugins. We will provide liberal access to that repository and existing user name/passwords can be used. Plugins hosted in this repo will be able to be maintained by QGIS developers should they need fixing and the original author is no longer available. Following this email we will post OSGEO SAC requests for the above mentioned SVN and Trac instances and get going. I have created a wiki page that we can use to document the setup and procedures for plugin-repository authors and plugin writers here: http://www.qgis.org/wiki/PluginRepository We will use the same QGIS-Django project to include future work such as reinstating the users map, creating the certification system web site and so on. If you are interested and able to collaborate with the django project development please let us know and add yourself to the wiki page. Thanks! Regards -- Tim Sutton - QGIS Project Steering Committee Member (Release Manager) == Please do not email me off-list with technical support questions. Using the lists will gain more exposure for your issues and the knowledge surrounding your issue will be shared with all. Visit http://linfiniti.com to find out about: * QGIS programming and support services * Mapserver and PostGIS based hosting plans * FOSS Consulting Services Skype: timlinux Irc: timlinux on #qgis at freenode.net == ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
On 11/15/2010 03:45 AM, Tim Sutton wrote: Hi all There has been a lot of discussion about the way forward for plugins, the plugin repository, plugin source management and plugin issue tracking / bug reporting in the last few weeks. Sitting in the same room for the last few days, we have also been able to go into more detail about our requirements. We would like to summarise our requirements and plans and get things going. So here is the synopsis: 1) We will set up a django instance on the QGIS.org virtual machine (using a python vitual environment) for building the new QGIS Plugin Repository. 2) The code for the repository will be managed via GIT and is hosted on github. If you need write access, please provide me with your github user name. Details for the repo here: https://github.com/timlinux/qgis-django 3) The management of issues for the repository itself will be done using github infrastructure. 4) For the plugins themselves, we are going to request that OSGEO SAC provide us with a trac instance dedicated for plugin bugs. It should be configured to authenticate users via LDAP so that existing OSGEO users will not need to create another account. 5) For the management of plugin source code, plugin authors will have the option of hosting their own project code (e.g. on github, sourceforge etc). 6) For plugin authors who wish to use an 'official' source repository, we will put in an OSGEO SAC request to set up an SVN instance for QGIS plugins. We will provide liberal access to that repository and existing user name/passwords can be used. Plugins hosted in this repo will be able to be maintained by QGIS developers should they need fixing and the original author is no longer available. Following this email we will post OSGEO SAC requests for the above mentioned SVN and Trac instances and get going. I have created a wiki page that we can use to document the setup and procedures for plugin-repository authors and plugin writers here: http://www.qgis.org/wiki/PluginRepository We will use the same QGIS-Django project to include future work such as reinstating the users map, creating the certification system web site and so on. If you are interested and able to collaborate with the django project development please let us know and add yourself to the wiki page. Thanks! Regards So change of mind from our discussion on IRC yesterday? Should Pirmin and I still bother with the test install of Redmine? To be clear #2 above is where the code for the django site itself will be hosted, not the source code of plugins. #3 isn't going to be kinda weird for people to report bugs to github about things not working on the website, or do we expect to just funnel email comments about it to the bug system ourselves. #6 I assume this SVN integrated with the New plugin bug trac site is ok? Did you still want the ability to set permissions by folder on the svn (this is doable). Thanks, Alex ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
So change of mind from our discussion on IRC yesterday? Should Pirmin and I still bother with the test install of Redmine? Actually I'm still a bit confused and don't want to take one of the two sides (mainly because I only looked briefly at the Redmine), but let's just write down the disadvantages and threats of the two solutions. Django: + the repository is easy to write ? can we connect a vcs as an integrated backend to the repo? (If no, let's try to sketch the workflow with external ones - are all the authors granted to upload the release to the repo or only the coordinator is, etc) ? can we handle existing osgeo accounts via ldap? ? can we write a trac plugin for syncronizing #tickets with release versions only, or with revisions in the backend vcs too? ? any other thoughts... Redmine: ? I assume, all the trac and vcs system is OFTB. Am I right? ? What with the repo itself? Is the API sufficient for uploading and downloading zips and downloading the metadata in easy-to-parse format? If not, should it be written as a (redmine) plugin? Is there anybody keen to write it? (in Ruby, as I think?) ? any other thoughts... I've probably missed many important points after the 5 unsleepy days and nights. To be clear #2 above is where the code for the django site itself will be hosted, not the source code of plugins. exactly #3 isn't going to be kinda weird for people to report bugs to github about things not working on the website, or do we expect to just funnel email comments about it to the bug system ourselves. Maybe better let's keep the repository bugtracking inside its trac. Do you see any disadvantages (except the case whole application hangs or crashes) #6 I assume this SVN integrated with the New plugin bug trac site is ok? Did you still want the ability to set permissions by folder on the svn (this is doable). I think the major problem is to grant permissions to create new folders (when creating each new plugin). Is it doable? ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Way forward: QGIS plugin repo, plugin Trac etc.
On 11/15/2010 01:23 PM, Borys Jurgiel wrote: So change of mind from our discussion on IRC yesterday? Should Pirmin and I still bother with the test install of Redmine? Actually I'm still a bit confused and don't want to take one of the two sides (mainly because I only looked briefly at the Redmine), but let's just write down the disadvantages and threats of the two solutions. Django: + the repository is easy to write ? can we connect a vcs as an integrated backend to the repo? (If no, let's try to sketch the workflow with external ones - are all the authors granted to upload the release to the repo or only the coordinator is, etc) Not out of the box as far as I know, but nothing's impossible. (might be faster to use something premade) ? can we handle existing osgeo accounts via ldap? Should not be an issue for any of the options. ? can we write a trac plugin for syncronizing #tickets with release versions only, or with revisions in the backend vcs too? ? any other thoughts... I think it would be easy to write a django app that pulls from an svn or git specified by the user, zips up the files and posts the plugin for download. The user managing a given plugin would just need to ensure the right link to their source code. What do you mean by synchronizing tickets? Any arbitrary version numbers could exist in trac, are you thinking you want that to link to a django site url? That sounds doable to me. Redmine: ? I assume, all the trac and vcs system is OFTB. Am I right? ? What with the repo itself? Is the API sufficient for uploading and downloading zips and downloading the metadata in easy-to-parse format? If not, should it be written as a (redmine) plugin? Is there anybody keen to write it? (in Ruby, as I think?) ? any other thoughts... Entirely possible if someone is up for Ruby. Probably also entirely possible as a TracPlugin which would be python. I've probably missed many important points after the 5 unsleepy days and nights. I was under the impression the choices were between: Django Frontend for hosting releases of plugins / Trac+svn backend for issues and code VS Django Frontend for hosting releases of plugins / Redmine+git backend for issues and code That way we only loosely couple the front and back which while more complex is more flexible and allows for easier opt-in components. Also means the front-end dev won't be held up by backend issues. If people stick to naming conventions it should be easy to create an easy link to jump from a given plugin on the Django site to it's page on the trac site or it's issue summary. To be clear #2 above is where the code for the django site itself will be hosted, not the source code of plugins. exactly #3 isn't going to be kinda weird for people to report bugs to github about things not working on the website, or do we expect to just funnel email comments about it to the bug system ourselves. Maybe better let's keep the repository bugtracking inside its trac. Do you see any disadvantages (except the case whole application hangs or crashes) I just thought it was odd to send people to github for issues related to a QGIS website. Where do we currently log website issues? #6 I assume this SVN integrated with the New plugin bug trac site is ok? Did you still want the ability to set permissions by folder on the svn (this is doable). I think the major problem is to grant permissions to create new folders (when creating each new plugin). Is it doable? Based on my research this is not an issue if we use the following Trac macros and plugins. When a user fills out the form to register a new plugin it creates the svn folder. http://trac-hacks.org/wiki/NewHackMacro (requires slight customization) Permission per folder can be done through the web, or it could be an all/none based on if you've been added to the approved list. http://trac-hacks.org/wiki/SvnAuthzAdminPlugin Although my read of Tim's announcement was that he no longer cared if there was folder level permissions on the svn for plugin development. Thanks, Alex ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
