Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sebastian Pipping wrote: I have created another script yesterday that can auto-merge information from gitosis.conf into repositories.xml. With that script in a Git hook setting up new Git-based Gentoo-hosted overlays requires changes at only an single place and propagates to repositories.xml and layman-global.txt. A post-update Git hook seems to be working well, code here: http://git.goodpoint.de/?p=overlays-xml-specification.git;a=blob;f=post-update.sample.gitosis-conf-merge I have sent a patch for layman to Gunnar already. Let's see if he likes it. Sebastian -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrGmz8ACgkQsAvGakAaFgDmNQCgq6IkOXgRC3SAMpUkleBwPYer xboAn2gijL3U0fbsylFIfoe280vpO48i =uM1k -END PGP SIGNATURE-
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Am Mittwoch, den 30.09.2009, 17:57 +0200 schrieb Sebastian Pipping: Fabian Groffen wrote: Point remains that it looks in-consistant, for repo, name is an attribute, while for owner it is a sub-element. Why having attributes in the first place anyway? It's closer to the original layman-global.txt which also makes the converter scripts simpler (and faster) than the element approach. I think it's just right. Then you're doing something wrong in your converter script because that would be a simple change. A simple rule of thumb is: use attributes for values with predefined contents, use elements otherwise. So, make name an element please. -- Tiziano Müller Gentoo Linux Developer Areas of responsibility: Samba, PostgreSQL, CPP, Python, sysadmin, GLEP Editor E-Mail : dev-z...@gentoo.org GnuPG FP : F327 283A E769 2E36 18D5 4DE2 1B05 6A63 AE9C 1E30 signature.asc Description: Dies ist ein digital signierter Nachrichtenteil smime.p7s Description: S/MIME cryptographic signature
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Am Mittwoch, den 30.09.2009, 18:17 +0200 schrieb Sebastian Pipping: Ciaran McCreesh wrote: Sure. Just periodically fetch the repository centrally. Have a master list of sync URLs with expected repository names, and use that to generate the full master list that includes metadata. Added bonus: you can quickly remove any repository that no longer exists. How long do you want the time frame for add-meta-data-or-get-kicked to be? If half the repos don't not make it will kicking them help anybody? Yes, then they're not maintained. And unmaintained overlays tend to contain even more broken ebuilds than others. What if that metadata format changes later or is extended by additional required entries? How about using the power of xml and version the schemas? As long as the metadata-file specifies the respective dtd/xsd/relaxng you know exactly how to validate (and parse) it and may apply a xsl-trafo if necessary to convert it to a new format on the fly. As long as you keep using xml you can then change the complete format in a new schema version. -- Tiziano Müller Gentoo Linux Developer Areas of responsibility: Samba, PostgreSQL, CPP, Python, sysadmin, GLEP Editor E-Mail : dev-z...@gentoo.org GnuPG FP : F327 283A E769 2E36 18D5 4DE2 1B05 6A63 AE9C 1E30 signature.asc Description: Dies ist ein digital signierter Nachrichtenteil smime.p7s Description: S/MIME cryptographic signature
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
On Thursday 01 October 2009, Tiziano Müller wrote: Am Mittwoch, den 30.09.2009, 18:17 +0200 schrieb Sebastian Pipping: Ciaran McCreesh wrote: Sure. Just periodically fetch the repository centrally. Have a master list of sync URLs with expected repository names, and use that to generate the full master list that includes metadata. Added bonus: you can quickly remove any repository that no longer exists. How long do you want the time frame for add-meta-data-or-get-kicked to be? If half the repos don't not make it will kicking them help anybody? Yes, then they're not maintained. And unmaintained overlays tend to contain even more broken ebuilds than others. There are plenty of unmaintained overlays. Keeping them in our index allows everyone to use what is still usable, and develoers to pick up maintenance. Removing them from the index makes it harder to do this. I guess what you want to do is keep unsuspecting users safe from these overlays, and the quality levels (i.e. graveyard in this case) are there to serve this purpose. Robert signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Tiziano Müller wrote: What if that metadata format changes later or is extended by additional required entries? How about using the power of xml and version the schemas? As long as the metadata-file specifies the respective dtd/xsd/relaxng you know exactly how to validate (and parse) it and may apply a xsl-trafo if necessary to convert it to a new format on the fly. As long as you keep using xml you can then change the complete format in a new schema version. We are version the schemas already. My point is that we would need to wait for the overlay maintainers once again to update their metadata. Transforming does not save us as we are missing new data. Sebastian
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Tiziano Müller wrote: A simple rule of thumb is: use attributes for values with predefined contents, use elements otherwise. So, make name an element please. Okay, let's give it a try. http://git.goodpoint.de/?p=overlays-xml-specification.git;a=commitdiff;h=8b10c5c7fd5b42bb5540fd686e0d97f5d11ad841 Sebastian
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Am Montag, den 28.09.2009, 20:23 +0200 schrieb Sebastian Pipping: repositories.xml = repo name=sping quality=experimental status=unofficial descriptionGentoo overlay of Sebastian Pipping/description homepagehttp://git.goodpoint.de/?p=overlay-sping.git/homepage owner type=person emailsebast...@pipping.org/email nameSebastian Pipping/name /owner source type=gitgit://git.goodpoint.de/overlay-sping.git/source feedhttp://git.goodpoint.de/?p=overlay-sping.git.git;a=atom/feed /repo = What is the reason that name is an attribute? While quality, status and type have a distinct set of allowed values, name doesn't and I'd therefore set it as an element instead. How about adding an attribute lang to description to be able to give descriptions in different languages? Cheers, Tiziano -- Tiziano Müller Gentoo Linux Developer Areas of responsibility: Samba, PostgreSQL, CPP, Python, sysadmin, GLEP Editor E-Mail : dev-z...@gentoo.org GnuPG FP : F327 283A E769 2E36 18D5 4DE2 1B05 6A63 AE9C 1E30 signature.asc Description: Dies ist ein digital signierter Nachrichtenteil smime.p7s Description: S/MIME cryptographic signature
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
On Mon, 28 Sep 2009 20:23:34 +0200 Sebastian Pipping webmas...@hartwork.org wrote: Now please ask questions and let us know what you think. Here's an alternative idea: * Move the repository information into the overlays themselves. Require overlays to provide a file containing the description, homepage, owner information etc. * After a certain amount of time, switch new versions of layman to use a new file that's generated from these. Any overlay that doesn't contain such a file is clearly unmaintained and should no longer be supported. * Also use this opportunity to clean up the repo_name vs layman name mess: any repository that gets it wrong should not be included in the new layman file. Sounds like a perfect opportunity to make sure that everything in layman's still actively maintained and developed. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Tiziano Müller wrote: Am Montag, den 28.09.2009, 20:23 +0200 schrieb Sebastian Pipping: repositories.xml = repo name=sping quality=experimental status=unofficial descriptionGentoo overlay of Sebastian Pipping/description homepagehttp://git.goodpoint.de/?p=overlay-sping.git/homepage owner type=person emailsebast...@pipping.org/email nameSebastian Pipping/name /owner source type=gitgit://git.goodpoint.de/overlay-sping.git/source feedhttp://git.goodpoint.de/?p=overlay-sping.git.git;a=atom/feed /repo = What is the reason that name is an attribute? While quality, status and type have a distinct set of allowed values, name doesn't and I'd therefore set it as an element instead. I don't see value in that change and I like name as it is. How about adding an attribute lang to description to be able to give descriptions in different languages? Done. http://git.goodpoint.de/?p=overlays-xml-specification.git;a=commitdiff;h=f7d068d772aa9961338f78d4b91b4783f43ff40f Sebastian
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
On 30-09-2009 17:36:47 +0200, Sebastian Pipping wrote: Tiziano Müller wrote: Am Montag, den 28.09.2009, 20:23 +0200 schrieb Sebastian Pipping: repositories.xml = repo name=sping quality=experimental status=unofficial descriptionGentoo overlay of Sebastian Pipping/description homepagehttp://git.goodpoint.de/?p=overlay-sping.git/homepage owner type=person emailsebast...@pipping.org/email nameSebastian Pipping/name /owner source type=gitgit://git.goodpoint.de/overlay-sping.git/source feedhttp://git.goodpoint.de/?p=overlay-sping.git.git;a=atom/feed /repo = What is the reason that name is an attribute? While quality, status and type have a distinct set of allowed values, name doesn't and I'd therefore set it as an element instead. I don't see value in that change and I like name as it is. Point remains that it looks in-consistant, for repo, name is an attribute, while for owner it is a sub-element. Why having attributes in the first place anyway? It's XML, so it may be extremely annoying to parse/use. -- Fabian Groffen Gentoo on a different level
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Ciaran McCreesh wrote: On Mon, 28 Sep 2009 20:23:34 +0200 Sebastian Pipping webmas...@hartwork.org wrote: Now please ask questions and let us know what you think. Here's an alternative idea: * Move the repository information into the overlays themselves. Require overlays to provide a file containing the description, homepage, owner information etc. I have created another script yesterday that can auto-merge information from gitosis.conf into repositories.xml. With that script in a Git hook setting up new Git-based Gentoo-hosted overlays requires changes at only an single place and propagates to repositories.xml and layman-global.txt. (The only extension needed to make it work is adding a line gentoo-is-overlay = True to Git repos that are overlays in gitosis.conf. The gentoo- prefix is meant as a namespace and gitosis upstream confirmed that this works without interfering with gitosis itself.) At the moment moving overlay meta info into the overlays does not seem like a good idea to. It would mean that any script working with overlays needs to check the repo out to get to that data. That doesn't sound like fun to me. Sounds like a perfect opportunity to make sure that everything in layman's still actively maintained and developed. Let me second that. Sebastian
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
On Wed, 30 Sep 2009 17:51:02 +0200 Sebastian Pipping webmas...@hartwork.org wrote: At the moment moving overlay meta info into the overlays does not seem like a good idea to. It would mean that any script working with overlays needs to check the repo out to get to that data. That doesn't sound like fun to me. No, you can copy the data into your master file. It's just that the original source of the data would be the repository, not the person adding the repository to the list. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Fabian Groffen wrote: Point remains that it looks in-consistant, for repo, name is an attribute, while for owner it is a sub-element. Why having attributes in the first place anyway? It's closer to the original layman-global.txt which also makes the converter scripts simpler (and faster) than the element approach. I think it's just right. It's XML, so it may be extremely annoying to parse/use. Please elaborate on that. Sebastian
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Ciaran McCreesh wrote: On Wed, 30 Sep 2009 17:51:02 +0200 Sebastian Pipping webmas...@hartwork.org wrote: At the moment moving overlay meta info into the overlays does not seem like a good idea to. It would mean that any script working with overlays needs to check the repo out to get to that data. That doesn't sound like fun to me. No, you can copy the data into your master file. It's just that the original source of the data would be the repository, not the person adding the repository to the list. If it can change, it needs to be kept in sync. I really don't think it belongs in there. Sebastian
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
On Wed, 30 Sep 2009 18:00:13 +0200 Sebastian Pipping webmas...@hartwork.org wrote: Ciaran McCreesh wrote: On Wed, 30 Sep 2009 17:51:02 +0200 Sebastian Pipping webmas...@hartwork.org wrote: At the moment moving overlay meta info into the overlays does not seem like a good idea to. It would mean that any script working with overlays needs to check the repo out to get to that data. That doesn't sound like fun to me. No, you can copy the data into your master file. It's just that the original source of the data would be the repository, not the person adding the repository to the list. If it can change, it needs to be kept in sync. I really don't think it belongs in there. Sure. Just periodically fetch the repository centrally. Have a master list of sync URLs with expected repository names, and use that to generate the full master list that includes metadata. Added bonus: you can quickly remove any repository that no longer exists. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
On Mon, Sep 28, 2009 at 08:23:34PM +0200, Sebastian Pipping wrote: Hello there! This may look like like a lot of text but it actually isn't. Please read on. Thanks. Thanks for your good work. I agree with your changes it was indeed needed. Actually, during my GSOC, I've noted layman-global.txt should be changed to have a cleanier description information and it's probably the moment to introduce this. At the moment, description is used for very short descriptions: descriptionArcon Overlay!/description or quite longer: descriptionA collection of Bazaar-related ebuilds, including release candidates of Bazaar./description even ery long: descriptionBerkano overlay provides an overlay for Gentoo Linux with ebuilds of bleeding edge software (such as live subversion builds) and other handy tools that are missing from the official portage tree. Areas of interest include multimedia and system administration./description layman is a cli so it doesn't really care of the description lenght but it's a pain to use in a GUI like PackageKit repository list. I think we should introduce a longdescription field even with a lang parameter like we already have in metadata.xml. So, description field will have to be used for short descriptions (a few words) to complete the name of the overlay like: descriptionexperimental gnome ebuilds/description I don't think DTD have abilities to restrict field value lenght but even if we will have to change a few description by hands, having a longdescription field could help. Sebastian, do you think you can add this to your set of changes ? (only the new field in the DTD at the moment). Thanks, Mounir
Re: [gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
volk...@gentoo.org wrote: I think we should introduce a longdescription field even with a lang parameter like we already have in metadata.xml. Done. http://git.goodpoint.de/?p=overlays-xml-specification.git;a=commit;h=c13a394fe1a868012548b2be5fb58359b3bc2891 Sebastian
[gentoo-dev] [rfc] layman-global.txt, repositories.xml, layman, overlays.gentoo.org
Hello there! This may look like like a lot of text but it actually isn't. Please read on. Thanks. During the last few days rbu and I have been teaming up on a proposal implementation improving on some of the issues that persist around layman-global.txt. (previously summarized in [1]) Both layman and the feed aggregator (planet) behind overlays.gentoo.org work with a database of information on repositories. These databases are not shared so changes do not propagate. That's not cool. What we have done for now: - Extended layman-global.txt format to repositories.xml format adding - A list of feed URIs .. for the planet side - Several source URIs .. for mirroring and several protocols - Owner name i.e. the name of the person to contact - quality classification (core, stable, testing, experimental, graveyard) .. to better guide users - project/person distinction .. for the planet side mainly - Created DTDs and Relax NG schemas for both the old and new format - Wrote a converter script from layman-global.txt format to repositories.xml format and back. It also adds generated file style warnings. The scripts have no dependencies other than plain Python. - Prepared a patch against layman trunk at r38 that adds support for - repositories.xml format - display of owner names (in addition to e-mail addresses) We have more or less the following future process in mind: - Collect feedback from the list (right now) and adjust the format and process as needed - Give birth to repositories.xml online - Decide about the final web location of repositories.xml - Put an initial repositories.xml on gentoo infra - Set up the repositories.xml to layman-global.txt converter script on the same machine to propagate all changes over to layman-global.txt in an automated fashion so we can keep it alive and up-to-date while still only editing its replacement - Revise layman patch until wrobel wants it upstream and put our a new layman release after - Setup script to create planet config from repositories.xml (Script creation is in progress.) - Take layman-global.txt offline in a year or so Here is a sample entry for each XML formats: layman-global.txt = overlay contact=sebast...@pipping.org name=sping src=git://git.goodpoint.de/overlay-sping.git status=unofficial type=git linkhttp://git.goodpoint.de/?p=overlay-sping.git/link descriptionGentoo overlay of Sebastian Pipping/description /overlay = repositories.xml = repo name=sping quality=experimental status=unofficial descriptionGentoo overlay of Sebastian Pipping/description homepagehttp://git.goodpoint.de/?p=overlay-sping.git/homepage owner type=person emailsebast...@pipping.org/email nameSebastian Pipping/name /owner source type=gitgit://git.goodpoint.de/overlay-sping.git/source feedhttp://git.goodpoint.de/?p=overlay-sping.git.git;a=atom/feed /repo = The code and schemas can be found at [2]. Now please ask questions and let us know what you think. Sebastian [1] http://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg35095.html [2] http://git.goodpoint.de/?p=overlays-xml-specification.git;a=summary
