Re: [Warzone-dev] Splitting the repository into source data sections?
On Mon, Oct 6, 2008 at 10:41 AM, Giel van Schijndel [EMAIL PROTECTED] wrote: Why do we even need the videos under revision control? People asked the same question about the data/ stuff. It quickly became clear that not having data under version control invited to total chaos. Just image when someone updates one or more videos through better re-encoding, and we have multiple versions floating around. Or someone adds more subtitles, or whatever. IMHO, we need one, authoritative source for all our stuff, and I do not feel confident in anything short of a version controlled repository. A tarball on some FTP/HTTP server is just not the same. - Per ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Per Inge Mathisen schreef: On Mon, Oct 6, 2008 at 1:23 AM, Giel van Schijndel [EMAIL PROTECTED] wrote: Per Inge Mathisen schreef: BTW - what will happen to existing, checked out working copies if the current contents of trunk/ is moved? I don't want my existing working copies to die a sudden death. If you have any changes you in any of the directories (subdirectories included) that will be moved you'll find svn update to fail halfway (i.e. fail at anything 0% progress and 100% progress). In other words, pretty much all existing checked out working copies will stop working. I would prefer not doing the reorganization, then. Too much pain. Can we make a new directory videos/ side by side with originals/ and trunk/ to store the FMVs instead? Why do we even need the videos under revision control? Then we can reorganize once we get a better revision control tool (subversion 1.5 or git or whatever). Subversion 1.5 is released for quite a long time already and wouldn't solve the above problem it's only got some better merge tracking. As for Git, unlike Subversion, Git isn't capable of only cloning/checking out a subdirectory of the repository, it checks out the entire directory tree. Instead Git promotes the use of multiple repositories if you want different trees. -- Giel signature.asc Description: OpenPGP digital signature ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Per Inge Mathisen schreef: On Mon, Oct 6, 2008 at 10:41 AM, Giel van Schijndel [EMAIL PROTECTED] wrote: Why do we even need the videos under revision control? People asked the same question about the data/ stuff. It quickly became clear that not having data under version control invited to total chaos. Just image when someone updates one or more videos through better re-encoding, and we have multiple versions floating around. Or someone adds more subtitles, or whatever. IMHO, we need one, authoritative source for all our stuff, and I do not feel confident in anything short of a version controlled repository. A tarball on some FTP/HTTP server is just not the same. Ok, point taken. However I'd like to suggest that we use a *separate* repository for those videos. I would personally prefer a git repository. Because of its speed with cloning and checking out different revisions (even with large data sets). -- Giel signature.asc Description: OpenPGP digital signature ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
On Mon, Oct 6, 2008 at 11:02 AM, Giel van Schijndel [EMAIL PROTECTED] wrote: Ok, point taken. However I'd like to suggest that we use a *separate* repository for those videos. I would personally prefer a git repository. Because of its speed with cloning and checking out different revisions (even with large data sets). I am fine with that. - Per ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
On Sun, Oct 5, 2008 at 7:50 AM, bugs buggy [EMAIL PROTECTED] wrote: With the upcoming release of the FMV patch in trunk, *before* we add the fmvs into the repo, I would like to confirm the data structure. trunk/data (only pumpkin official stuff, + derivatives of their work?) trunk/code or trunk/source trunk/FMVs (since it is optional?) trunk/tools (for editors, utilities, and whatever else) trunk/3rdparty (*If* we go this route. Though, I rather not.) I would prefer to have tools as part of the code/ directory, since they are (or should be) closely linked to the rest of the code. trunk/FMVs - trunk/video ? Otherwise, I think this is a good idea, since they are so big. trunk/3rdparty - trunk/mods ? I am in favour of putting the best and well-licensed mods in the repository, and giving their maintainers write access. This makes it easier for us to package them as well as keeping track of its history. The name 3rdparty makes it sound like a storage area for external code, or something. - Per ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Am Sonntag, 5. Oktober 2008 07:50:39 schrieb bugs buggy: With the upcoming release of the FMV patch in trunk, *before* we add the fmvs into the repo, I would like to confirm the data structure. trunk/data (only pumpkin official stuff, + derivatives of their work?) trunk/code or trunk/source trunk/FMVs (since it is optional?) - trunk/videos or trunk/movies, *if* we need this. Though I think it would be ok to have it as a subdir of trunk/data? (Well, ok, I am a bit unsure. :P More to download if you dont want it, but it seems logical, you only download it once, the structure does not affect later packaging, ...) --Devu signature.asc Description: This is a digitally signed message part. ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Per Inge Mathisen schreef: On Sun, Oct 5, 2008 at 7:50 AM, bugs buggy [EMAIL PROTECTED] wrote: With the upcoming release of the FMV patch in trunk, *before* we add the fmvs into the repo, I would like to confirm the data structure. trunk/data (only pumpkin official stuff, + derivatives of their work?) trunk/code or trunk/source trunk/FMVs (since it is optional?) trunk/tools (for editors, utilities, and whatever else) trunk/3rdparty (*If* we go this route. Though, I rather not.) I would prefer to have tools as part of the code/ directory, since they are (or should be) closely linked to the rest of the code. Agreed. I would also prefer to use source instead of code. trunk/FMVs - trunk/video ? Otherwise, I think this is a good idea, since they are so big. I agree on the video directory name, though I'm not sure if we should put the videos in the repository at all... trunk/3rdparty - trunk/mods ? I am in favour of putting the best and well-licensed mods in the repository, and giving their maintainers write access. This makes it easier for us to package them as well as keeping track of its history. The name 3rdparty makes it sound like a storage area for external code, or something. Perhaps we should just put the mods in another repository? That solves the issue of giving maintainers write access without having to give write access to the Warzone source as well... Thus I suggest we use this directory layout for our own repository: trunk/data trunk/pkg trunk/source trunk/source/build_tools trunk/source/doc trunk/source/icons trunk/source/lib trunk/source/m4 trunk/source/macosx trunk/source/makerules trunk/source/po trunk/source/src trunk/source/tools trunk/source/win32 Then use a layout like this for the mod repository: mod/branches mod/tags mod/trunk Then restrict access to the mod subdirectories to only allow maintainers of that mod to write to that part of the repo. E.g. I was thinking of setting up the mod repository in a similar fashion to how trac-hacks.org is set up. trac-hacks.org allows you to create a new project, it'll then automatically add the directory structure to the repository and grant the creating user write access. -- Giel signature.asc Description: OpenPGP digital signature ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
On Sunday, 5 October 2008 at 1:50, bugs buggy wrote: With the upcoming release of the FMV patch in trunk, *before* we add the fmvs into the repo, I would like to confirm the data structure. Question: Why do you want to add those to SVN? I can understand the rest of the data (well, perhaps not the music, but that is not that large), but the FVMs? Those are large, won't really be modified and are not necessary for the game. trunk/data (only pumpkin official stuff, + derivatives of their work?) trunk/code or trunk/source trunk/FMVs (since it is optional?) And even separated from the rest? What's the difference to just having a separate archive then? And when you don't have the movies in SVN, what exactly are the benefits of separating source and data? Packaging? Were there any complaints about that? The Debian packaging system easily allows the creation of separate packages from one source archive, and I'd imagine any other deserving that name does as well. Redownloading changed textures? How often are those changed? I think Buginator complained about the bandwidth required comparing revisions before and after a big change. But if the src and data are separated, that won't magically make those old big changes go away, and introduce another one. And this problem seems easier solved by having the complete SVN history locally, with a tool like git-svn, hgsvn (for Mercurial) or SVK (I guess there are more, those are just the ones I know of). Those were the reasons listed in this thread, and both don't seem very convincing to me. Are there other advantages I have missed? ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
On Sun, Oct 5, 2008 at 2:43 PM, Giel van Schijndel [EMAIL PROTECTED] wrote: I would prefer to have tools as part of the code/ directory, since they are (or should be) closely linked to the rest of the code. Agreed. I would also prefer to use source instead of code. This is imprecise, because there are sources for other things than code, and this directory contains only code. - Per ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Am Sonntag, 5. Oktober 2008 16:47:47 schrieb Per Inge Mathisen: On Sun, Oct 5, 2008 at 2:43 PM, Giel van Schijndel [EMAIL PROTECTED] wrote: I would prefer to have tools as part of the code/ directory, since they are (or should be) closely linked to the rest of the code. Agreed. I would also prefer to use source instead of code. This is imprecise, because there are sources for other things than code, and this directory contains only code. sourcecode? :P signature.asc Description: This is a digitally signed message part. ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Dennis Schridde schreef: Am Sonntag, 5. Oktober 2008 14:43:40 schrieb Giel van Schijndel: Thus I suggest we use this directory layout for our own repository: trunk/data trunk/pkg Why put pkg here? pkg will need to be have access to both the source and data... -- Giel signature.asc Description: OpenPGP digital signature ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Per Inge Mathisen schreef: On Sun, Oct 5, 2008 at 2:43 PM, Giel van Schijndel [EMAIL PROTECTED] wrote: I would prefer to have tools as part of the code/ directory, since they are (or should be) closely linked to the rest of the code. Agreed. I would also prefer to use source instead of code. This is imprecise, because there are sources for other things than code, and this directory contains only code. Using the directory layout I proposed it'll also contain the translations (po), documentation (doc), icons (icons) and all build systems. I suppose you could interpret the build systems as being part of the code, I'm not sure how that'll apply to the other mentioned items though. -- Giel signature.asc Description: OpenPGP digital signature ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Christian Ohm schreef: On Sunday, 5 October 2008 at 1:50, bugs buggy wrote: With the upcoming release of the FMV patch in trunk, *before* we add the fmvs into the repo, I would like to confirm the data structure. Question: Why do you want to add those to SVN? I can understand the rest of the data (well, perhaps not the music, but that is not that large), but the FVMs? Those are large, won't really be modified and are not necessary for the game. I don't like the idea of adding the videos to our repo either... trunk/data (only pumpkin official stuff, + derivatives of their work?) trunk/code or trunk/source trunk/FMVs (since it is optional?) And even separated from the rest? What's the difference to just having a separate archive then? And when you don't have the movies in SVN, what exactly are the benefits of separating source and data? Packaging? Were there any complaints about that? The Debian packaging system easily allows the creation of separate packages from one source archive, and I'd imagine any other deserving that name does as well. Packaging by external parties is IMO a non-issue. If they have *real* issues with whatever directory layout we provide then *they* should bring that up on *this* mailing list. Considering that no such thing has been brought to this list I'd consider packaging a void argument. Redownloading changed textures? How often are those changed? I think Buginator complained about the bandwidth required comparing revisions before and after a big change. AFAIK that is the only reason. But if the src and data are separated, that won't magically make those old big changes go away, and introduce another one. No, it will definitely not solve the issues for, e.g., the PNG shrinkage revisions. It will indeed introduce a new problem (of larger scale) with the revision that performs the directory layout migration as boundary. The only advantage I can see is that, after said layout migration, huge changes to the data (such as aforementioned PNG shrinkage) will have a much less significant impact on the code when you need to jump between revisions. When put in that light changing the directory layout doesn't sound very appealing to me anymore. In fact it sounds like the wrong solution to this problem to me. IMO a better solution would be to require changes of large amounts of binary data to go through this mailing list first. Here changes would be defined as having svn status give one of these values in the first column: A, M or R. Whatever large means would have to be decided upon. And this problem seems easier solved by having the complete SVN history locally, with a tool like git-svn, hgsvn (for Mercurial) or SVK (I guess there are more, those are just the ones I know of). Yup, especially when using the history of the repo heavily I prefer a distributed solution (I'm mainly using git-svn myself). Those were the reasons listed in this thread, and both don't seem very convincing to me. Are there other advantages I have missed? The only one I know of is faster history browsing in light of large changes. -- Giel signature.asc Description: OpenPGP digital signature ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
BTW - what will happen to existing, checked out working copies if the current contents of trunk/ is moved? I don't want my existing working copies to die a sudden death. - Per ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Per Inge Mathisen schreef: BTW - what will happen to existing, checked out working copies if the current contents of trunk/ is moved? I don't want my existing working copies to die a sudden death. If you have any changes you in any of the directories (subdirectories included) that will be moved you'll find svn update to fail halfway (i.e. fail at anything 0% progress and 100% progress). -- Giel signature.asc Description: OpenPGP digital signature ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
On 9/25/08, bugs buggy [EMAIL PROTECTED] wrote: On 9/23/08, Per Inge Mathisen [EMAIL PROTECTED][EMAIL PROTECTED] wrote: One idea: How about using just one repository, but shaped like this: trunk/ branches/ tags/ then a next level (example from inside trunk): trunk/code/ trunk/data/ trunk/originals/ so if you wanted to download the latest of all, you would do svn up on trunk, while if you wanted to just grab a new code branch, you'd do svn up on trunk/code only. I do not see what you gain from using separate repositories, instead of separate directories inside a single repository. Am I missing something? I think it is a feature that with a single repository we would still have one global revision ID for everything, so you could say, test revision NNN, instead of test code revision NNN with data revision MMM. What you suggest would be fine. I just wasn't sure if it is better (or allowed) to have one big repository or not. If we do have a size limit, it is better to find out from GNA now, rather than running into this later. With the upcoming release of the FMV patch in trunk, *before* we add the fmvs into the repo, I would like to confirm the data structure. trunk/data (only pumpkin official stuff, + derivatives of their work?) trunk/code or trunk/source trunk/FMVs (since it is optional?) trunk/tools (for editors, utilities, and whatever else) trunk/3rdparty (*If* we go this route. Though, I rather not.) The main reason I don't think we should have 3rd party content in the repo is: a) they need access to the repo in order to fix/modify/change it. b) if we include a mod or two, we are obligated to stick all the other mods that people make. c) license issues. d) we got lots of other good content that *isn't* in the repo (lots of nice music available on the forums), and it doesn't seem fair to include only some stuff, and not other stuff. e) filling up the repo without a real good reason with the extra data. Therefore, I think it is best that we remove all 3rd party mods that we have, and instead of the repo, stick them where they belong, like http://download.gna.org/warzone/ or perhaps a dedicated page on the main site. This would even out the playing field, and not clog up the repo with extra files. ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
On 9/23/08, Per Inge Mathisen [EMAIL PROTECTED] wrote: One idea: How about using just one repository, but shaped like this: trunk/ branches/ tags/ then a next level (example from inside trunk): trunk/code/ trunk/data/ trunk/originals/ so if you wanted to download the latest of all, you would do svn up on trunk, while if you wanted to just grab a new code branch, you'd do svn up on trunk/code only. I do not see what you gain from using separate repositories, instead of separate directories inside a single repository. Am I missing something? I think it is a feature that with a single repository we would still have one global revision ID for everything, so you could say, test revision NNN, instead of test code revision NNN with data revision MMM. What you suggest would be fine. I just wasn't sure if it is better (or allowed) to have one big repository or not. If we do have a size limit, it is better to find out from GNA now, rather than running into this later. ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Am Dienstag, 23. September 2008 06:08:37 schrieb bugs buggy: I was thinking it might be a good time to either split the repository into dedicated sections, or have multiple repositories. Can someone state the pros and cons for that? (esp. the pros...) What led to that idea? One should be only used for source, and the other only for data. The question remains, do we stick with GNA, even though they still are having some issue? What issues do they have? I was told by non-Gna-members that the SVN server had been replaced? (Sadly my communications with the project itself can somehow not be established... :( ) Is having multiple repositories allowed on GNA? I would have to investigate whether multiple repositories for one project are possible via Savana. But having multiple projects for one thing on Gna is afaik not an issue (this is not an official statement, will have to ask for that, too :P ). I am building this view on the statement Large software distributions are not allowed; they should be split into separate projects. as found on https://gna.org/register/. Should we take the data to a new host to test the waters? Or is having data not allowed as a 'project' on like source forge, or whatever? I do not know the SourceForge rules and I do not know anyone of their staff. Furthermore SF is not free, they have loads of ads, even more flash and their servers are not really faster either... So if we find anything else, I'd rather go with that instead of SF... To bring this up again: Why exactly can we not host our repository on wz2100.net? That sounds like the most effective and simple solution to me... (In case we move away from Gna for whatever reason...) We'd lack the project management functions as found in Savana, but then we are not that many people and adding a new SVN access every few months doesn't sound too bothersome. Permission to do that can even be handed out to other project members, so our admin (*waves to Kamaze*) does not have to be bothered with it. --Devu signature.asc Description: This is a digitally signed message part. ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Am Dienstag, 23. September 2008 12:00:18 schrieb Freddie Witherden: Hi Dennis, Hello Fred and the rest of the Warzone 2100 Resurrection Project! On 23 Sep 2008, at 08:28, Dennis Schridde wrote: Am Dienstag, 23. September 2008 06:08:37 schrieb bugs buggy: I was thinking it might be a good time to either split the repository into dedicated sections, or have multiple repositories. Can someone state the pros and cons for that? (esp. the pros...) What led to that idea? Easier for distributors/packagers (separate packages). We are not all forced to re-download ~20MiB of images when someone decides to re- compress the tiles/textures. One idea I just got: We had that issue with originals and works-in-game data. Maybe use different repositories for those? data, data-for-game, game? The data-for-game repos could be autogenerated from the data repository regularly. (Cronjob, buildbot, ...) So data-creators would work on the originals in the data repository, and would not have to care about converting every piece themselves. We also would not require everyone to regenerate the for-game data themselves, since they can use the data-for-game repos if they want. That repos would also be what we finaly create the releases from. One should be only used for source, and the other only for data. The question remains, do we stick with GNA, even though they still are having some issue? What issues do they have? I was told by non-Gna-members that the SVN server had been replaced? (Sadly my communications with the project itself can somehow not be established... :( ) It works fine for me. Is having multiple repositories allowed on GNA? I would have to investigate whether multiple repositories for one project are possible via Savana. But having multiple projects for one thing on Gna is afaik not an issue (this is not an official statement, will have to ask for that, too :P ). I am building this view on the statement Large software distributions are not allowed; they should be split into separate projects. as found on https://gna.org/register/. Thanks, would really appreciate it if you could. I am currently trying to somehow connect with the rest of the team, but it seems the mailinglists are all moderated (or malfunctioning) so it might take a few days... To bring this up again: Why exactly can we not host our repository on wz2100.net? That sounds like the most effective and simple solution to me... (In case we move away from Gna for whatever reason...) We'd lack the project management functions as found in Savana, but then we are not that many people and adding a new SVN access every few months doesn't sound too bothersome. Permission to do that can even be handed out to other project members, so our admin (*waves to Kamaze*) does not have to be bothered with it. What if Kamaze gets hit by a bus? Shared root access? Or goes nuts and rm -rf's everything? I know already 3 mirrors of the SVN tree (Giel, me, Launchpad). I also mirror the download area on Gna (to prevent issues like the rm-rf-hack which hit us last time). We can do the same for the website and databases, I think. Something I forgot? Greetings, Devurandom signature.asc Description: This is a digitally signed message part. ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Hello WZRP, hello Per! Am Dienstag, 23. September 2008 13:05:58 schrieb Per Inge Mathisen: One idea: How about using just one repository, but shaped like this: trunk/ branches/ tags/ then a next level (example from inside trunk): trunk/code/ trunk/data/ trunk/originals/ so if you wanted to download the latest of all, you would do svn up on trunk, while if you wanted to just grab a new code branch, you'd do svn up on trunk/code only. Sound good, too. I do not see what you gain from using separate repositories, instead of separate directories inside a single repository. Am I missing something? I think it is a feature that with a single repository we would still have one global revision ID for everything, so you could say, test revision NNN, instead of test code revision NNN with data revision MMM. data/trunk data/branches/X ... Would work too, and is equivalent to the split-repos approach. (Unless SVN shows bad performance with lots of data/revisions in one repos.) (Sidenote: For Git (git-svn) this would mean to mirror the data/originals/code into different repositories, but that should not be an issue, I think.) --Devu signature.asc Description: This is a digitally signed message part. ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
Re: [Warzone-dev] Splitting the repository into source data sections?
Am Dienstag, den 23.09.2008, 14:51 +0200 schrieb Dennis Schridde: Hello WZRP, hello Per! Am Dienstag, 23. September 2008 13:05:58 schrieb Per Inge Mathisen: One idea: How about using just one repository, but shaped like this: trunk/ branches/ tags/ then a next level (example from inside trunk): trunk/code/ trunk/data/ trunk/originals/ so if you wanted to download the latest of all, you would do svn up on trunk, while if you wanted to just grab a new code branch, you'd do svn up on trunk/code only. Sound good, too. Hi all i like per's idea (in an artist's point of view) trunk/data trunk/code originals/ ... looks somehow better than trunk/ trunk/data/ originals/ ... and imo better than code/trunk code/branches code/tags data/trunk data/branches data/tags originals/ and you simply can checkout trunk/ and have everything you need, if you only want to update source, go in code and update it.. i'm not for seperate repositories, code works with data and it should be consistent each revision. The data-for-game repos could be autogenerated from the data repository regularly. (Cronjob, buildbot, ...) So data-creators would work on the originals in the data repository, and would not have to care about converting every piece themselves. i would like to keep originals for work-in-progress, unfinished things, models don't have to be updated very often. with the (future) pie converter it isn't that difficult to do it every time regards elio ___ Warzone-dev mailing list Warzone-dev@gna.org https://mail.gna.org/listinfo/warzone-dev
