Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
Hi Stefano, On 6/25/19 10:18 PM, Stefano Stabellini wrote: On Tue, 25 Jun 2019, Julien Grall wrote: The point here is that we can be flexible and creative about the way to maintain the docs on xen.git. But as a technology is certainly better than the wiki: we don't have to keep them all up-to-date with the code, but at least this way we have a chance (if we want to). If we leave them on the wiki, there is no chance. I can't see how xen.git is going to be better if "we don't have to keep them all up-to-date". That's because a contributor could add a patch at the end of a series to update one of the docs, even if the doc in question comes with no promises of being up-to-date. I think this is going the wrong direction. The goal of using xen.git is to try to keep the documentation up-to-date. Is the goal to keep the documentation fully up-to-date, or more up-to-date than what we currently have on the wiki? I don't see this as black and white. There are lot of stages of "up-to-dateness" in between. From the discussion it is pretty that we want the doc fully up-to-date. But my point here is most of the board should be trivial. The most of the non-trivial setup require non-upstream patch. While I am happy to see that on the wiki, I think xen.git should not promote such configuration at all. We are working upstream, not with unknown/untrusted stack. For some working fully upstream, I don't think xen.git should promote any distros/versions of the kernel. However, this is ok on the wiki. I would like to see the wiki disappear completely in the long term. As we are moving more content to xen.git, it is not a good idea to have two places where we keep information, for similar reasons why you suggested to use in-code comments instead of docs to document interfaces. It just takes more efforts to maintain information in two places and they tend to get out of sync with each others. If we make the wiki go away (I hope so), we'll need a place to store the Arm board-specific documents, and other tutorials. Removing the wiki is an honorable goal, however I don't think all the wiki is suitable for xen.git. The Arm board-specific documents is an example. You actually haven't addressed my concern above. If you look at the wiki, a lot of them ([1], [2], [3]) contains non-upstreamed work or non-upstreamable hack. For those containing only upstream work [4], the example is focusing on one set of distros. In the case of QEMU, I already had some people asking whether it is possible to use without U-boot. Why would we promote Ubuntu and not something else? Overall, there are so many configurations possible (kernel, u-boot, distributions) that it does not makes sense to keep track of that in xen.git. Instead, I think we should write generic doc on how to boot Xen on a U-boot/UEFI setup and inviting the users to look into more details for his board. I don't think that we should host non-upstreamable hacks in general either in xen.git. However, some boards require specific Linux kernel trees to boot that are not upstream, so there is a fine line between "non-upstreamable hack" and "following the docs and code provided with a given board". Similarly, I don't think we should have distro-specific information in xen.git (or the wiki) either but sometimes we need to pick one as an example. Otherwise, the tutorial wouldn't be complete. In the case of the board specific files, we do both these things, but because they are unavoidable, not because somebody wanted to advertise Ubuntu, or preferred to use a private Linux kernel tree over upstream. If that is the case, we should not import them into xen.git. That's not correct, it is unavoidable because we never worked with the distros to integrate Xen on Arm. So we pile up crap as we can't say "Please select your distros and install Xen from the package". Basically, we are doing the job of a distro. Instead, we should focus at better integration in distros such as Yocto, Zephyr. I also agree with you that if we are going to host only docs 100% accurate, generic, and fully maintained by us on xen.git, then we should NOT have the board specific docs there. But I would say that there is a benefit in having docs not maintained by us and potentially not fully up-to-date on xen.git that are useful to the community, like the board specific docs. It would be easier to keep them "somewhat up to date" release by release compared to the wiki, and in the future turn them into "fully up to date" docs if we get more engagement. If we leave them in the wiki I have the impression that their only possible future is to rot and die... This is not something that is going to happen anytime soon, so I think we should take Lars' suggestion and talk about it at the Summit. Sounds good to me. Cheers, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
On Tue, 25 Jun 2019, Julien Grall wrote: > > > > The point here is that we can be flexible and creative about the way to > > > > maintain the docs on xen.git. But as a technology is certainly better > > > > than the wiki: we don't have to keep them all up-to-date with the code, > > > > but at least this way we have a chance (if we want to). If we leave them > > > > on the wiki, there is no chance. > > > > > > I can't see how xen.git is going to be better if "we don't have to keep > > > them > > > all up-to-date". > > > > That's because a contributor could add a patch at the end of a series to > > update one of the docs, even if the doc in question comes with no > > promises of being up-to-date. > > I think this is going the wrong direction. The goal of using xen.git is to try > to keep the documentation up-to-date. Is the goal to keep the documentation fully up-to-date, or more up-to-date than what we currently have on the wiki? I don't see this as black and white. There are lot of stages of "up-to-dateness" in between. > > > But my point here is most of the board should be trivial. The most of the > > > non-trivial setup require non-upstream patch. While I am happy to see that > > > on > > > the wiki, I think xen.git should not promote such configuration at all. We > > > are > > > working upstream, not with unknown/untrusted stack. > > > > > > For some working fully upstream, I don't think xen.git should promote any > > > distros/versions of the kernel. However, this is ok on the wiki. > > > > I would like to see the wiki disappear completely in the long term. As > > we are moving more content to xen.git, it is not a good idea to have two > > places where we keep information, for similar reasons why you suggested > > to use in-code comments instead of docs to document interfaces. It > > just takes more efforts to maintain information in two places and they > > tend to get out of sync with each others. > > > > If we make the wiki go away (I hope so), we'll need a place to store the > > Arm board-specific documents, and other tutorials. > > Removing the wiki is an honorable goal, however I don't think all the wiki is > suitable for xen.git. The Arm board-specific documents is an example. > > You actually haven't addressed my concern above. If you look at the wiki, a > lot of them ([1], [2], [3]) contains non-upstreamed work or non-upstreamable > hack. > > For those containing only upstream work [4], the example is focusing on one > set of distros. In the case of QEMU, I already had some people asking whether > it is possible to use without U-boot. Why would we promote Ubuntu and not > something else? > > Overall, there are so many configurations possible (kernel, u-boot, > distributions) that it does not makes sense to keep track of that in xen.git. > > Instead, I think we should write generic doc on how to boot Xen on a > U-boot/UEFI setup and inviting the users to look into more details for his > board. I don't think that we should host non-upstreamable hacks in general either in xen.git. However, some boards require specific Linux kernel trees to boot that are not upstream, so there is a fine line between "non-upstreamable hack" and "following the docs and code provided with a given board". Similarly, I don't think we should have distro-specific information in xen.git (or the wiki) either but sometimes we need to pick one as an example. Otherwise, the tutorial wouldn't be complete. In the case of the board specific files, we do both these things, but because they are unavoidable, not because somebody wanted to advertise Ubuntu, or preferred to use a private Linux kernel tree over upstream. If that is the case, we should not import them into xen.git. I also agree with you that if we are going to host only docs 100% accurate, generic, and fully maintained by us on xen.git, then we should NOT have the board specific docs there. But I would say that there is a benefit in having docs not maintained by us and potentially not fully up-to-date on xen.git that are useful to the community, like the board specific docs. It would be easier to keep them "somewhat up to date" release by release compared to the wiki, and in the future turn them into "fully up to date" docs if we get more engagement. If we leave them in the wiki I have the impression that their only possible future is to rot and die... This is not something that is going to happen anytime soon, so I think we should take Lars' suggestion and talk about it at the Summit. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
> On Jun 25, 2019, at 12:34, Lars Kurth wrote: > > On 25/06/2019, 14:47, "Andrew Cooper" wrote: > >> On 25/06/2019 13:15, Lars Kurth wrote: >> On 25/06/2019, 10:03, "Julien Grall" wrote: >> > The point here is that we can be flexible and creative about the way to > maintain the docs on xen.git. But as a technology is certainly better > than the wiki: we don't have to keep them all up-to-date with the code, > but at least this way we have a chance (if we want to). If we leave them > on the wiki, there is no chance. I can't see how xen.git is going to be better if "we don't have to keep them all up-to-date". >>> >>> That's because a contributor could add a patch at the end of a series to >>> update one of the docs, even if the doc in question comes with no >>> promises of being up-to-date. >> >> I think this is going the wrong direction. The goal of using xen.git is to >> try >> to keep the documentation up-to-date. >> >> I agree with Julien and this was also not my intention. The reason why I >> brought this up now is that the in-tree docs are pretty much a mess today >> and are stale in many ways. And they look TERRIBLE and are not easily >> searchable. However, Andy's latest set of patches provide an opportunity to >> consolidate some of the in-tree docs in a nicely rendered and searchable >> format. > > So the plan here is to get a consistent and uniform set of high quality > docs. > > As fair warning, I'm intending to be fairly strict with what goes in > (quality wise), because I'm going to do my best to ensure that the > sphinx documentation doesn't devolve into the mess that wiki or the > majority of docs/ currently is. > > I wholeheartedly agree > >> I have been focussing on process related and key developer related docs, >> because who maintains them is not actually an issue in theory. Everyone >> really ought to care, because everyone is impacted by these. > > The key point is for maintainers/reviewers to be aware of whether > documentation exists for the area of code being modified, and if so, > whether the submitted patch should also patch the documentation. > > I am wondering whether this is something which could be addressed. One > possibility may be to have SUPPORT.md point to documentation. But that is > kind of assuming that SUPPORT.md works and is widely used. There may be > better or orthogonal ways to point to relevant docs (e.g. by pointing to docs > in header files and other source files). > > Reviewers tend to be fairly good at noticing patches which also need to > patch docs/misc/xen-command-line.pandoc (submitters, less so), but this > approach needs extending to the whole of the sphinx docs (which in turn > requires the docs to stay high quality so its easy for maintainers to > know what is where). > > Although this does not apply in my proposal, I think the key issue has been > that reviewers and submitters of code often don't use our documentation. The > wiki is seen as a separate thing and also has the disadvantage that it > doesn't lend itself to supporting different versions of Xen. And most of the > time, developers do not use it and neither contribute to it. > > My hope was that by hosting documentation related to contribution workflow > and other essential tasks close to other useful documentation this would > enable change. > > @Andy and others: I need to know whether you agree with my proposal and > whether anyone has other suggestions. If not already present in the release manager process checklist, we could specify documentation-related updates for each release, e.g. minimum text for new features, revisions to modified features, SUPPORT.md updates. Rich (resend with non-bulkmail address) ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
> On Jun 25, 2019, at 12:34, Lars Kurth wrote: > > > > On 25/06/2019, 14:47, "Andrew Cooper" wrote: > >>On 25/06/2019 13:15, Lars Kurth wrote: >> On 25/06/2019, 10:03, "Julien Grall" wrote: >> > The point here is that we can be flexible and creative about the way to > maintain the docs on xen.git. But as a technology is certainly better > than the wiki: we don't have to keep them all up-to-date with the code, > but at least this way we have a chance (if we want to). If we leave them > on the wiki, there is no chance. I can't see how xen.git is going to be better if "we don't have to keep them all up-to-date". >>> >>> That's because a contributor could add a patch at the end of a series to >>> update one of the docs, even if the doc in question comes with no >>> promises of being up-to-date. >> >>I think this is going the wrong direction. The goal of using xen.git is >> to try >>to keep the documentation up-to-date. >> >> I agree with Julien and this was also not my intention. The reason why I >> brought this up now is that the in-tree docs are pretty much a mess today >> and are stale in many ways. And they look TERRIBLE and are not easily >> searchable. However, Andy's latest set of patches provide an opportunity to >> consolidate some of the in-tree docs in a nicely rendered and searchable >> format. > >So the plan here is to get a consistent and uniform set of high quality >docs. > >As fair warning, I'm intending to be fairly strict with what goes in >(quality wise), because I'm going to do my best to ensure that the >sphinx documentation doesn't devolve into the mess that wiki or the >majority of docs/ currently is. > > I wholeheartedly agree > >> I have been focussing on process related and key developer related docs, >> because who maintains them is not actually an issue in theory. Everyone >> really ought to care, because everyone is impacted by these. > >The key point is for maintainers/reviewers to be aware of whether >documentation exists for the area of code being modified, and if so, >whether the submitted patch should also patch the documentation. > > I am wondering whether this is something which could be addressed. One > possibility may be to have SUPPORT.md point to documentation. But that is > kind of assuming that SUPPORT.md works and is widely used. There may be > better or orthogonal ways to point to relevant docs (e.g. by pointing to docs > in header files and other source files). > >Reviewers tend to be fairly good at noticing patches which also need to >patch docs/misc/xen-command-line.pandoc (submitters, less so), but this >approach needs extending to the whole of the sphinx docs (which in turn >requires the docs to stay high quality so its easy for maintainers to >know what is where). > > Although this does not apply in my proposal, I think the key issue has been > that reviewers and submitters of code often don't use our documentation. The > wiki is seen as a separate thing and also has the disadvantage that it > doesn't lend itself to supporting different versions of Xen. And most of the > time, developers do not use it and neither contribute to it. > > My hope was that by hosting documentation related to contribution workflow > and other essential tasks close to other useful documentation this would > enable change. > > @Andy and others: I need to know whether you agree with my proposal and > whether anyone has other suggestions. If not already present in the release manager process checklist, we could specify documentation-related updates for each release, e.g. minimum text for new features, revisions to modified features, SUPPORT.md updates. Rich ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
On 25/06/2019 17:34, Lars Kurth wrote: > > On 25/06/2019, 14:47, "Andrew Cooper" wrote: > > On 25/06/2019 13:15, Lars Kurth wrote: > > On 25/06/2019, 10:03, "Julien Grall" wrote: > > > > >>> The point here is that we can be flexible and creative about > the way to > > >>> maintain the docs on xen.git. But as a technology is certainly > better > > >>> than the wiki: we don't have to keep them all up-to-date with > the code, > > >>> but at least this way we have a chance (if we want to). If we > leave them > > >>> on the wiki, there is no chance. > > >> > > >> I can't see how xen.git is going to be better if "we don't have > to keep them > > >> all up-to-date". > > > > > > That's because a contributor could add a patch at the end of a > series to > > > update one of the docs, even if the doc in question comes with no > > > promises of being up-to-date. > > > > I think this is going the wrong direction. The goal of using > xen.git is to try > > to keep the documentation up-to-date. > > > > I agree with Julien and this was also not my intention. The reason why > I brought this up now is that the in-tree docs are pretty much a mess today > and are stale in many ways. And they look TERRIBLE and are not easily > searchable. However, Andy's latest set of patches provide an opportunity to > consolidate some of the in-tree docs in a nicely rendered and searchable > format. > > So the plan here is to get a consistent and uniform set of high quality > docs. > > As fair warning, I'm intending to be fairly strict with what goes in > (quality wise), because I'm going to do my best to ensure that the > sphinx documentation doesn't devolve into the mess that wiki or the > majority of docs/ currently is. > > I wholeheartedly agree > > > I have been focussing on process related and key developer related > docs, because who maintains them is not actually an issue in theory. Everyone > really ought to care, because everyone is impacted by these. > > The key point is for maintainers/reviewers to be aware of whether > documentation exists for the area of code being modified, and if so, > whether the submitted patch should also patch the documentation. > > I am wondering whether this is something which could be addressed. One > possibility may be to have SUPPORT.md point to documentation. But that is > kind of assuming that SUPPORT.md works and is widely used. There may be > better or orthogonal ways to point to relevant docs (e.g. by pointing to docs > in header files and other source files). > > Reviewers tend to be fairly good at noticing patches which also need to > patch docs/misc/xen-command-line.pandoc (submitters, less so), but this > approach needs extending to the whole of the sphinx docs (which in turn > requires the docs to stay high quality so its easy for maintainers to > know what is where). > > Although this does not apply in my proposal, I think the key issue has been > that reviewers and submitters of code often don't use our documentation. The > wiki is seen as a separate thing and also has the disadvantage that it > doesn't lend itself to supporting different versions of Xen. And most of the > time, developers do not use it and neither contribute to it. It is a positive feedback cycle. Noone uses our documentation because its terrible, so the documentation stays in a terrible state. As soon as the docs start to improve, the awareness will improve. That said, there is still a responsibility for maintainers to be aware of the documentation for their own areas, and to ensure the documentation is maintained to the same high standard as we expect from our code. > > My hope was that by hosting documentation related to contribution workflow > and other essential tasks close to other useful documentation this would > enable change. > > @Andy and others: I need to know whether you agree with my proposal and > whether anyone has other suggestions. Absolutely. "how to submit changes" is one of the most cited wiki pages, and having that in sphinx will cause loads of people to be aware that we do have other docs in sphinx as well. The fact that everything is properly indexed is a massive benefit here. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
On 25/06/2019, 14:47, "Andrew Cooper" wrote: On 25/06/2019 13:15, Lars Kurth wrote: > On 25/06/2019, 10:03, "Julien Grall" wrote: > > >>> The point here is that we can be flexible and creative about the way to > >>> maintain the docs on xen.git. But as a technology is certainly better > >>> than the wiki: we don't have to keep them all up-to-date with the code, > >>> but at least this way we have a chance (if we want to). If we leave them > >>> on the wiki, there is no chance. > >> > >> I can't see how xen.git is going to be better if "we don't have to keep them > >> all up-to-date". > > > > That's because a contributor could add a patch at the end of a series to > > update one of the docs, even if the doc in question comes with no > > promises of being up-to-date. > > I think this is going the wrong direction. The goal of using xen.git is to try > to keep the documentation up-to-date. > > I agree with Julien and this was also not my intention. The reason why I brought this up now is that the in-tree docs are pretty much a mess today and are stale in many ways. And they look TERRIBLE and are not easily searchable. However, Andy's latest set of patches provide an opportunity to consolidate some of the in-tree docs in a nicely rendered and searchable format. So the plan here is to get a consistent and uniform set of high quality docs. As fair warning, I'm intending to be fairly strict with what goes in (quality wise), because I'm going to do my best to ensure that the sphinx documentation doesn't devolve into the mess that wiki or the majority of docs/ currently is. I wholeheartedly agree > I have been focussing on process related and key developer related docs, because who maintains them is not actually an issue in theory. Everyone really ought to care, because everyone is impacted by these. The key point is for maintainers/reviewers to be aware of whether documentation exists for the area of code being modified, and if so, whether the submitted patch should also patch the documentation. I am wondering whether this is something which could be addressed. One possibility may be to have SUPPORT.md point to documentation. But that is kind of assuming that SUPPORT.md works and is widely used. There may be better or orthogonal ways to point to relevant docs (e.g. by pointing to docs in header files and other source files). Reviewers tend to be fairly good at noticing patches which also need to patch docs/misc/xen-command-line.pandoc (submitters, less so), but this approach needs extending to the whole of the sphinx docs (which in turn requires the docs to stay high quality so its easy for maintainers to know what is where). Although this does not apply in my proposal, I think the key issue has been that reviewers and submitters of code often don't use our documentation. The wiki is seen as a separate thing and also has the disadvantage that it doesn't lend itself to supporting different versions of Xen. And most of the time, developers do not use it and neither contribute to it. My hope was that by hosting documentation related to contribution workflow and other essential tasks close to other useful documentation this would enable change. @Andy and others: I need to know whether you agree with my proposal and whether anyone has other suggestions. Regards Lars ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
On 25/06/2019 13:15, Lars Kurth wrote: > On 25/06/2019, 10:03, "Julien Grall" wrote: > > >>> The point here is that we can be flexible and creative about the way > to > >>> maintain the docs on xen.git. But as a technology is certainly better > >>> than the wiki: we don't have to keep them all up-to-date with the > code, > >>> but at least this way we have a chance (if we want to). If we leave > them > >>> on the wiki, there is no chance. > >> > >> I can't see how xen.git is going to be better if "we don't have to > keep them > >> all up-to-date". > > > > That's because a contributor could add a patch at the end of a series to > > update one of the docs, even if the doc in question comes with no > > promises of being up-to-date. > > I think this is going the wrong direction. The goal of using xen.git is > to try > to keep the documentation up-to-date. > > I agree with Julien and this was also not my intention. The reason why I > brought this up now is that the in-tree docs are pretty much a mess today and > are stale in many ways. And they look TERRIBLE and are not easily searchable. > However, Andy's latest set of patches provide an opportunity to consolidate > some of the in-tree docs in a nicely rendered and searchable format. So the plan here is to get a consistent and uniform set of high quality docs. As fair warning, I'm intending to be fairly strict with what goes in (quality wise), because I'm going to do my best to ensure that the sphinx documentation doesn't devolve into the mess that wiki or the majority of docs/ currently is. > I have been focussing on process related and key developer related docs, > because who maintains them is not actually an issue in theory. Everyone > really ought to care, because everyone is impacted by these. The key point is for maintainers/reviewers to be aware of whether documentation exists for the area of code being modified, and if so, whether the submitted patch should also patch the documentation. Reviewers tend to be fairly good at noticing patches which also need to patch docs/misc/xen-command-line.pandoc (submitters, less so), but this approach needs extending to the whole of the sphinx docs (which in turn requires the docs to stay high quality so its easy for maintainers to know what is where). ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
On 25/06/2019, 10:03, "Julien Grall" wrote: >>> The point here is that we can be flexible and creative about the way to >>> maintain the docs on xen.git. But as a technology is certainly better >>> than the wiki: we don't have to keep them all up-to-date with the code, >>> but at least this way we have a chance (if we want to). If we leave them >>> on the wiki, there is no chance. >> >> I can't see how xen.git is going to be better if "we don't have to keep them >> all up-to-date". > > That's because a contributor could add a patch at the end of a series to > update one of the docs, even if the doc in question comes with no > promises of being up-to-date. I think this is going the wrong direction. The goal of using xen.git is to try to keep the documentation up-to-date. I agree with Julien and this was also not my intention. The reason why I brought this up now is that the in-tree docs are pretty much a mess today and are stale in many ways. And they look TERRIBLE and are not easily searchable. However, Andy's latest set of patches provide an opportunity to consolidate some of the in-tree docs in a nicely rendered and searchable format. I have been focussing on process related and key developer related docs, because who maintains them is not actually an issue in theory. Everyone really ought to care, because everyone is impacted by these. What happens today for many of these type of docs and/or processes is that: a) We have discussion about a process / working practice on the list until we come to a conclusion b) Then we take it and copy it to the wiki Why not merge this into one activity Both of you are interested in Arm docs, but this is something I will let you fight out. Maybe you want to chat about this some more at the summit >> But my point here is most of the board should be trivial. The most of the >> non-trivial setup require non-upstream patch. While I am happy to see that on >> the wiki, I think xen.git should not promote such configuration at all. We are >> working upstream, not with unknown/untrusted stack. >> >> For some working fully upstream, I don't think xen.git should promote any >> distros/versions of the kernel. However, this is ok on the wiki. > > I would like to see the wiki disappear completely in the long term. As > we are moving more content to xen.git, it is not a good idea to have two > places where we keep information, for similar reasons why you suggested > to use in-code comments instead of docs to document interfaces. It > just takes more efforts to maintain information in two places and they > tend to get out of sync with each others. > > If we make the wiki go away (I hope so), we'll need a place to store the > Arm board-specific documents, and other tutorials. Removing the wiki is an honorable goal, however I don't think all the wiki is suitable for xen.git. The Arm board-specific documents is an example. Removing the wiki was not my aim. The wiki is useful in some cases, but not in others. Lars ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
Hi Stefano, On 25/06/2019 01:02, Stefano Stabellini wrote: On Mon, 24 Jun 2019, Julien Grall wrote: Hi Stefano, On 6/24/19 9:23 PM, Stefano Stabellini wrote: On Mon, 24 Jun 2019, Julien Grall wrote: Hi, On 24/06/2019 19:03, Stefano Stabellini wrote: On Mon, 24 Jun 2019, Lars Kurth wrote: I think we all agree by now that maintaining up-to-date docs on the wiki and keeping them in sync with code changes is hard. I see moving things from the wiki to xen.git as a great improvement. We have a few Xen on ARM docs that are worth importing from the wiki: https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions I agree for this but ... And all the board specific docs linked from it, such as: https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64 https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels https://wiki.xenproject.org/wiki/HiKey960 ... I think it is a pretty bad idea to import board specific docs. There are a lot of way to build components for a given board and I am worry of the overheard for the maintainers to look/maintain the documentation. It also brings the question of the acceptance/removal of a board documentation. That problem can be solved by specifying an appropriate maintenance model for those documents. Instead we should provide generic guidance/troubleshoot to the user. Anything board specific could be maintain on the wiki by someone caring about the board without having us to gate it. If we move the docs to xen.git it doesn't immediately imply that the REST maintainers need to "gate" them. We could make the existing curators of those pages the maintainers for those files, for example. We can come up with mode ideas. We could even leave them unmaintained. I don't think I want to add a random person as a maintainer in xen.git. So at best we would need a new role. This is a good point, taking into account the current governance model. We could use R: for that? I am not entirely sure, this still mean "THE REST" will be in charge of it. Anyway, this is not the biggest problem here. The point here is that we can be flexible and creative about the way to maintain the docs on xen.git. But as a technology is certainly better than the wiki: we don't have to keep them all up-to-date with the code, but at least this way we have a chance (if we want to). If we leave them on the wiki, there is no chance. I can't see how xen.git is going to be better if "we don't have to keep them all up-to-date". That's because a contributor could add a patch at the end of a series to update one of the docs, even if the doc in question comes with no promises of being up-to-date. I think this is going the wrong direction. The goal of using xen.git is to try to keep the documentation up-to-date. But my point here is most of the board should be trivial. The most of the non-trivial setup require non-upstream patch. While I am happy to see that on the wiki, I think xen.git should not promote such configuration at all. We are working upstream, not with unknown/untrusted stack. For some working fully upstream, I don't think xen.git should promote any distros/versions of the kernel. However, this is ok on the wiki. I would like to see the wiki disappear completely in the long term. As we are moving more content to xen.git, it is not a good idea to have two places where we keep information, for similar reasons why you suggested to use in-code comments instead of docs to document interfaces. It just takes more efforts to maintain information in two places and they tend to get out of sync with each others. If we make the wiki go away (I hope so), we'll need a place to store the Arm board-specific documents, and other tutorials. Removing the wiki is an honorable goal, however I don't think all the wiki is suitable for xen.git. The Arm board-specific documents is an example. You actually haven't addressed my concern above. If you look at the wiki, a lot of them ([1], [2], [3]) contains non-upstreamed work or non-upstreamable hack. For those containing only upstream work [4], the example is focusing on one set of distros. In the case of QEMU, I already had some people asking whether it is possible to use without U-boot. Why would we promote Ubuntu and not something else? Overall, there are so many configurations possible (kernel, u-boot, distributions) that it does not makes sense to keep track of that in xen.git. Instead, I think we should write generic doc on how to boot Xen on a U-boot/UEFI setup and inviting the users to look into more details for his board. Cheers, [1] https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-X [2] https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Stout [3] https://wiki.xenproject.org/wiki/HiKey [4] https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
On Mon, 24 Jun 2019, Julien Grall wrote: > Hi Stefano, > > On 6/24/19 9:23 PM, Stefano Stabellini wrote: > > On Mon, 24 Jun 2019, Julien Grall wrote: > > > Hi, > > > > > > On 24/06/2019 19:03, Stefano Stabellini wrote: > > > > On Mon, 24 Jun 2019, Lars Kurth wrote: > > > > I think we all agree by now that maintaining up-to-date docs on the wiki > > > > and keeping them in sync with code changes is hard. I see moving things > > > > from the wiki to xen.git as a great improvement. We have a few Xen on > > > > ARM docs that are worth importing from the wiki: > > > > > > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions > > > > > > I agree for this but ... > > > > > > > > > > > And all the board specific docs linked from it, such as: > > > > > > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64 > > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels > > > > https://wiki.xenproject.org/wiki/HiKey960 > > > > > > ... I think it is a pretty bad idea to import board specific docs. There > > > are a lot of way to build components for a given board and I am worry of > > > the overheard for the maintainers to look/maintain the documentation. It > > > also brings the question of the acceptance/removal of > > > a board documentation. > > > > That problem can be solved by specifying an appropriate maintenance > > model for those documents. > > > > > > > Instead we should provide generic guidance/troubleshoot to the user. > > > Anything board specific could be maintain on the wiki by someone caring > > > about the board without having us to gate it. > > > > If we move the docs to xen.git it doesn't immediately imply that the > > REST maintainers need to "gate" them. We could make the existing > > curators of those pages the maintainers for those files, for example. We > > can come up with mode ideas. We could even leave them unmaintained. > > I don't think I want to add a random person as a maintainer in xen.git. So at > best we would need a new role. This is a good point, taking into account the current governance model. We could use R: for that? > > The point here is that we can be flexible and creative about the way to > > maintain the docs on xen.git. But as a technology is certainly better > > than the wiki: we don't have to keep them all up-to-date with the code, > > but at least this way we have a chance (if we want to). If we leave them > > on the wiki, there is no chance. > > I can't see how xen.git is going to be better if "we don't have to keep them > all up-to-date". That's because a contributor could add a patch at the end of a series to update one of the docs, even if the doc in question comes with no promises of being up-to-date. > But my point here is most of the board should be trivial. The most of the > non-trivial setup require non-upstream patch. While I am happy to see that on > the wiki, I think xen.git should not promote such configuration at all. We are > working upstream, not with unknown/untrusted stack. > > For some working fully upstream, I don't think xen.git should promote any > distros/versions of the kernel. However, this is ok on the wiki. I would like to see the wiki disappear completely in the long term. As we are moving more content to xen.git, it is not a good idea to have two places where we keep information, for similar reasons why you suggested to use in-code comments instead of docs to document interfaces. It just takes more efforts to maintain information in two places and they tend to get out of sync with each others. If we make the wiki go away (I hope so), we'll need a place to store the Arm board-specific documents, and other tutorials. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
Hi Stefano, On 6/24/19 9:23 PM, Stefano Stabellini wrote: On Mon, 24 Jun 2019, Julien Grall wrote: Hi, On 24/06/2019 19:03, Stefano Stabellini wrote: On Mon, 24 Jun 2019, Lars Kurth wrote: I think we all agree by now that maintaining up-to-date docs on the wiki and keeping them in sync with code changes is hard. I see moving things from the wiki to xen.git as a great improvement. We have a few Xen on ARM docs that are worth importing from the wiki: https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions I agree for this but ... And all the board specific docs linked from it, such as: https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64 https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels https://wiki.xenproject.org/wiki/HiKey960 ... I think it is a pretty bad idea to import board specific docs. There are a lot of way to build components for a given board and I am worry of the overheard for the maintainers to look/maintain the documentation. It also brings the question of the acceptance/removal of a board documentation. That problem can be solved by specifying an appropriate maintenance model for those documents. Instead we should provide generic guidance/troubleshoot to the user. Anything board specific could be maintain on the wiki by someone caring about the board without having us to gate it. If we move the docs to xen.git it doesn't immediately imply that the REST maintainers need to "gate" them. We could make the existing curators of those pages the maintainers for those files, for example. We can come up with mode ideas. We could even leave them unmaintained. I don't think I want to add a random person as a maintainer in xen.git. So at best we would need a new role. The point here is that we can be flexible and creative about the way to maintain the docs on xen.git. But as a technology is certainly better than the wiki: we don't have to keep them all up-to-date with the code, but at least this way we have a chance (if we want to). If we leave them on the wiki, there is no chance. I can't see how xen.git is going to be better if "we don't have to keep them all up-to-date". But my point here is most of the board should be trivial. The most of the non-trivial setup require non-upstream patch. While I am happy to see that on the wiki, I think xen.git should not promote such configuration at all. We are working upstream, not with unknown/untrusted stack. For some working fully upstream, I don't think xen.git should promote any distros/versions of the kernel. However, this is ok on the wiki. Cheers, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
On Mon, 24 Jun 2019, Julien Grall wrote: > Hi, > > On 24/06/2019 19:03, Stefano Stabellini wrote: > > On Mon, 24 Jun 2019, Lars Kurth wrote: > > I think we all agree by now that maintaining up-to-date docs on the wiki > > and keeping them in sync with code changes is hard. I see moving things > > from the wiki to xen.git as a great improvement. We have a few Xen on > > ARM docs that are worth importing from the wiki: > > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions > > I agree for this but ... > > > > > And all the board specific docs linked from it, such as: > > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64 > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels > > https://wiki.xenproject.org/wiki/HiKey960 > > ... I think it is a pretty bad idea to import board specific docs. There > are a lot of way to build components for a given board and I am worry of > the overheard for the maintainers to look/maintain the documentation. It > also brings the question of the acceptance/removal of > a board documentation. That problem can be solved by specifying an appropriate maintenance model for those documents. > Instead we should provide generic guidance/troubleshoot to the user. > Anything board specific could be maintain on the wiki by someone caring > about the board without having us to gate it. If we move the docs to xen.git it doesn't immediately imply that the REST maintainers need to "gate" them. We could make the existing curators of those pages the maintainers for those files, for example. We can come up with mode ideas. We could even leave them unmaintained. The point here is that we can be flexible and creative about the way to maintain the docs on xen.git. But as a technology is certainly better than the wiki: we don't have to keep them all up-to-date with the code, but at least this way we have a chance (if we want to). If we leave them on the wiki, there is no chance. > We could try to revive a discussion we had a couple of years ago where > someone is responsible for a given board to test and document it. Today > it is informal, but I have been pushing for it recently for new boards. Yes, this is a good idea and fit nicely with the maintenance model I was suggesting. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
On Mon, 24 Jun 2019, Lars Kurth wrote: > Hi all, > > since Andy created https://xenbits.xen.org/docs/sphinx-unstable/ and > https://xenbits.xen.org/docs/sphinx-unstable-staging/ (sources in > docs/hypervisor-guide, docs/guest-guide, docs/admin-guide) I was wondering > whether it would not make sense to migrate some key documents in the wiki > (and possible some docs elsewhere in the tree) into the new structure and > refresh and update the documentation in the process. I am volunteering to do > some of the leg-work. > > I started looking into what some other projects do and the following seem to > be sensible and lightweight examples of Information Architecture to structure > the content: > * https://docs.openstack.org/infra/manual/developers.html > * > https://github.com/kubernetes/community/blob/master/contributors/devel/README.md > > Both are fairly lightweight and seem to be sensible guides. But before we get > to that level of detail, I thought it makes sense to look at what we have and > candidates for moving/improving/creation. However, it is clear that we need > two broad categories under docs/hypervisor-guide (Hypervisor developer > documentation): note that I don’t much care about the actual labels > * The process of developing and contributing code > * Setting up your dev environment, coding, and debugging > > Documentation which may be worth moving or creating > > Text files in tree (which are close to RST file format) > * Xen.git:CONTRIBUTING – it seems to me that this is a candidate for moving > (with a note in the original file which outlines where to look for the > source/rendered output) – Even committers occasionally don’t seem to be aware > of some of the licensing related common practices we have. Giving some of > that content a more prominent place in a new more user friendly and modern > looking docset seems sensible > * Xen.git:INSTALL may be a good candidate which could live in the admin guide > and/or developer guide > * Xen.git:CODING_STYLE and Xen.git:tools/libxl/CODING_STYLE - Note that in > the community call discussion Jan raised the point that we tend to not > document precedents for many things which are coding style related. Maybe we > can get a bit better > * Xen.git:MAINTAINERS should stay as it is, but should probably be referred > to appropriately in the docset > > Content from the wiki (the idea would be to redirect those pages in the wiki > to the new locations) > * https://wiki.xenproject.org/wiki/Asking_Developer_Questions - could do with > a refresh. Possibly there is also a tie in with > https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01518.html > * https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source - there seems to > be some overlap with Xen.git/INSTALL which may be worth cleaning up > * https://wiki.xenproject.org/wiki/Xen_Project_Repositories > * https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches - This has > become a bit unwieldy and could do with some clean-up >* Slight overlap with Xen.git:CONTRIBUTING (around DCO and Sign-off) >* Making good patches probably needs some work and maybe should be broken > out. It should include good examples and setting expectations of what is > deemed good and bad around areas where we have higher standards than other > projects (such as commit messages, explaining rationale for a change, > technical debt, ...). It should probably also cover things such as Design > Documentation and where to find templates, highlight existing documentation > and where to find it/update it, the same with text, SUPPORT.md (aka add a new > feature), etc. >* Setting up git send-email: should probably go into a section related to > the development environment set-up and just be referred to >* Using git send-email alone - we should nuke this section and focus on > the next section >* Simplest workflow: Git format-patch, > add_maintainers.pl/get_maintainer.pl and git send-email - I would build the > bulk of the doc around this, but maybe move the > add_maintainers.pl/get_maintainer.pl file into a separate document under a > Xen specific development tools section and just refer to it > * Sending patches manually - we should nuke this section and focus on the > next section > * I would move the bulk of this into a Contribution Workflow section, > which gives an overall workflow and just highlight the reroll count. We > should define the tags and conventions such as RFC somewhere in an > introductory section of this document >* All the QEMU, Linux, ... stuff can either stay on the Wiki or could be > broken out > * https://wiki.xenproject.org/wiki/Reporting_Bugs_against_Xen_Project - strip > all the XAPI stuff. NBot sure whether > https://wiki.xenproject.org/wiki/XenParavirtOpsHelp is still applicable: nuke > if not > > Key content that is missing > * An overview for testing, which should include >- OSSTEST >- XTF
[Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process
Hi all, since Andy created https://xenbits.xen.org/docs/sphinx-unstable/ and https://xenbits.xen.org/docs/sphinx-unstable-staging/ (sources in docs/hypervisor-guide, docs/guest-guide, docs/admin-guide) I was wondering whether it would not make sense to migrate some key documents in the wiki (and possible some docs elsewhere in the tree) into the new structure and refresh and update the documentation in the process. I am volunteering to do some of the leg-work. I started looking into what some other projects do and the following seem to be sensible and lightweight examples of Information Architecture to structure the content: * https://docs.openstack.org/infra/manual/developers.html * https://github.com/kubernetes/community/blob/master/contributors/devel/README.md Both are fairly lightweight and seem to be sensible guides. But before we get to that level of detail, I thought it makes sense to look at what we have and candidates for moving/improving/creation. However, it is clear that we need two broad categories under docs/hypervisor-guide (Hypervisor developer documentation): note that I don’t much care about the actual labels * The process of developing and contributing code * Setting up your dev environment, coding, and debugging Documentation which may be worth moving or creating Text files in tree (which are close to RST file format) * Xen.git:CONTRIBUTING – it seems to me that this is a candidate for moving (with a note in the original file which outlines where to look for the source/rendered output) – Even committers occasionally don’t seem to be aware of some of the licensing related common practices we have. Giving some of that content a more prominent place in a new more user friendly and modern looking docset seems sensible * Xen.git:INSTALL may be a good candidate which could live in the admin guide and/or developer guide * Xen.git:CODING_STYLE and Xen.git:tools/libxl/CODING_STYLE - Note that in the community call discussion Jan raised the point that we tend to not document precedents for many things which are coding style related. Maybe we can get a bit better * Xen.git:MAINTAINERS should stay as it is, but should probably be referred to appropriately in the docset Content from the wiki (the idea would be to redirect those pages in the wiki to the new locations) * https://wiki.xenproject.org/wiki/Asking_Developer_Questions - could do with a refresh. Possibly there is also a tie in with https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01518.html * https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source - there seems to be some overlap with Xen.git/INSTALL which may be worth cleaning up * https://wiki.xenproject.org/wiki/Xen_Project_Repositories * https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches - This has become a bit unwieldy and could do with some clean-up * Slight overlap with Xen.git:CONTRIBUTING (around DCO and Sign-off) * Making good patches probably needs some work and maybe should be broken out. It should include good examples and setting expectations of what is deemed good and bad around areas where we have higher standards than other projects (such as commit messages, explaining rationale for a change, technical debt, ...). It should probably also cover things such as Design Documentation and where to find templates, highlight existing documentation and where to find it/update it, the same with text, SUPPORT.md (aka add a new feature), etc. * Setting up git send-email: should probably go into a section related to the development environment set-up and just be referred to * Using git send-email alone - we should nuke this section and focus on the next section * Simplest workflow: Git format-patch, add_maintainers.pl/get_maintainer.pl and git send-email - I would build the bulk of the doc around this, but maybe move the add_maintainers.pl/get_maintainer.pl file into a separate document under a Xen specific development tools section and just refer to it * Sending patches manually - we should nuke this section and focus on the next section * I would move the bulk of this into a Contribution Workflow section, which gives an overall workflow and just highlight the reroll count. We should define the tags and conventions such as RFC somewhere in an introductory section of this document * All the QEMU, Linux, ... stuff can either stay on the Wiki or could be broken out * https://wiki.xenproject.org/wiki/Reporting_Bugs_against_Xen_Project - strip all the XAPI stuff. NBot sure whether https://wiki.xenproject.org/wiki/XenParavirtOpsHelp is still applicable: nuke if not Key content that is missing * An overview for testing, which should include - OSSTEST - XTF - The GitLab CI * Outcome from the Minimum Standards and Best Practices discussion at https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01518.html depending on outcome. I was thinking about a
