Hi Markus, On Wed, Dec 02, 2015 at 01:58:17PM +0200, Markus Lehtonen wrote: > On 01/12/15 19:35, "git-buildpackage on behalf of Guido Günther" > <[email protected] on behalf of [email protected]> > wrote: > > >Hi Markus, > >On Mon, Nov 30, 2015 at 09:51:24AM +0200, Markus Lehtonen wrote: > >> Hi, > >> > >> > >> > >> On 27/11/15 17:38, "Guido Günther" <[email protected]> wrote: > >> > >> >Hi Markus, > >> >On Fri, Nov 27, 2015 at 04:43:56PM +0200, Markus Lehtonen wrote: > >> >[..snip..] > >> >> > > and I think we > >> >> > > need to rethink the strategy for this submodule. For the deb > >> >> > > component > >> >> > > tests I'm keeping everything in HEAD and set up everything from > >> >> > > there. The rpm tests need different commits to function which > >> >> > > causes/caused > >> >> > > some breakackge here when pulling in more component tests so I > >> >> > > think we > >> >> > > would be better of needing only a single commit like in the deb > >> >> > > case. Would that be doable? > >> >> > > >> >> > I haven't figured out any better way – although I admit that this is > >> >> > not > >> >> > the most elegant solution. The other branches in the testdata > >> >> > repository > >> >> > basically represent various test packages (i.e. test repositories can > >> >> > be > >> >> > constructed from those). I didn't find any nice and easy way to > >> >> > serialize the test repositories (to construct them at test time) > >> >> > either. > >> >> > And additional git submodules (per-testpackage submodule) don't work > >> >> > either because the test repositories usually require multiple branches > >> >> > (upstream, packaging, pristine-tar) and this is not supported in git > >> >> > submodules which work on single HEAD commit. Better ideas are welcome, > >> >> > of course. > >> >> > >> >> I played around with one idea I had earlier and came up with a solution > >> >> to the problem of storing test package git repositories. Please see the > >> >> 'feature/component-tests' branch in my Github repository > >> >> https://github.com/marquiz/git-buildpackage-rpm > >> >> > >> >> Now, all test data is available in submodule 'HEAD' - test package git > >> >> repositories are serialized as patches (and some additional data) in > >> >> separate subdiretories. The branch also contains two patches for > >> >> buildpackage-rpm (cherry-picked from the 'feature/buildpackage-rpm' > >> >> branch) that are needed for the buildpackage-rpm unit tests to pass. It > >> >> also contains one "optional" patch that makes names of the per-testcase > >> >> temporary directories more user friendly. > >> > > >> >Awesome! This makes the rpm and deb component tests use the same > >> >"strategy" (the rpm ones being far more complete). Running the tests > >> >fails on me with: > >> > > >> >nosetests -x > >> >ERROR: Basic test of native pkg > >> >---------------------------------------------------------------------- > >> >Traceback (most recent call last): > >> > File > >> > "/var/scratch/src/git-buildpackage/git-buildpackage/tests/component/rpm/test_buildpackage_rpm.py", > >> > line 116, in test_native_build > >> > self.init_test_repo('gbp-test-native') > >> > File > >> > "/var/scratch/src/git-buildpackage/git-buildpackage/tests/component/rpm/__init__.py", > >> > line 58, in init_test_repo > >> > dirname = os.path.basename(cls.orig_repos[pkg_name].path) > >> >KeyError: 'gbp-test-native' > >> > > >> >though. This is the first error. I have the full log attached. > >> > >> Oh, sorry, there was erroneous glob in my patch. I pushed a fixed version > >> into my branch. It should work now, please test. > > > > > >Down to one failure now after merging features/component-tests > > > >.....................E................................................................................................................................................................................................................... > >====================================================================== > >ERROR: Create an upstream zip archive > >---------------------------------------------------------------------- > >Traceback (most recent call last): > > File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest > > self.test(*self.arg) > > File > > "/var/scratch/src/git-buildpackage/git-buildpackage/tests/04_test_submodules.py", > > line 136, in test_create_zip_archives > > '', '', '', 'zip') > > File > > "/var/scratch/src/git-buildpackage/git-buildpackage/gbp/scripts/common/buildpackage.py", > > line 82, in git_archive_submodules > > CatenateZipArchive(main_archive)(submodule_archive) > > File > > "/var/scratch/src/git-buildpackage/git-buildpackage/gbp/command_wrappers.py", > > line 299, in __call__ > > Command.__call__(self, [target]) > > File > > "/var/scratch/src/git-buildpackage/git-buildpackage/gbp/command_wrappers.py", > > line 146, in __call__ > > raise CommandExecFailed(self._format_err()) > >CommandExecFailed: Couldn't append "/tmp/tmpKHcFcV/submodule.zip" to > >"/tmp/tmpKHcFcV/main.zip": execution failed: [Errno 2] No such file or > >directory > >-------------------- >> begin captured logging << -------------------- > >gbp: ERROR: Couldn't append "/tmp/tmpKHcFcV/submodule.zip" to > >"/tmp/tmpKHcFcV/main.zip": execution failed: [Errno 2] No such file or > >directory > >--------------------- >> end captured logging << --------------------- > > > >---------------------------------------------------------------------- > > Hmm, this must be because you do not have zipmerge installed. I changed the > "common/buildpackage: support for different archive formats" > patch in my branch so that zipmerge is now part of Build-Depends
Great, this fixes it. > >> >Two further things: > >> > > >> >* please add gbp_*/ to .gitignore in the data repo so we don't get any > >> > untracked content after running the tests. > >> > >> Hmm, no gbp_* temporary directories should be created under the data > >> (tests/component/rpm/data) directory. They should be created in the > >> cwd of the user which should be topdir of gbp. > > > ># git status > >On branch locals/rpm-testdata > >Changes not staged for commit: > > (use "git add <file>..." to update what will be committed) > > (use "git checkout -- <file>..." to discard changes in working directory) > > (commit or discard the untracked or modified content in submodules) > > > > modified: tests/component/rpm/data (untracked content) > > > >cd tests/component/rpm/data/ > >$ git status > >HEAD detached at 09a446d > >Untracked files: > > (use "git add <file>..." to include in what will be committed) > > > > gbp_TestGbpRpm_smDdsW/ > > gbp_TestGbpRpm_wKF6Z3/ > > gbp_TestPqRpm_UOAsQW/ > > > >nothing added to commit but untracked files present (use "git add" to track) > > > >Adding gbp_* to .gitignore in that repo would fix this. > > I think those must be leftovers from my previous erroneous patch. The > tests should not contaminate the testdata submodule - if it does > something is wrong. Thus, I would not like to add that to .gitignore > as it may shadow some problems. With the current patches there's not gbp_* around anymore as well. > >> >* It seems manage.py uses lots of code from gbp. Wouldn't it be better > >> > to reuse this. We can do this at a later time though, having the test > >> > data for the component tests correct is much more important. > >> > >> Yeah, there's a lot of similar code - doing similar things, variable names > >> etc (although it's technically written from scratch). A lot of it is > >> copied from my previous 'bootstrap.py' from the same repository. I > >> specifically wrote it independently, in order to cut the circular > >> dependency between the data repository and gbp api. The amount of > >> overlapping code is not that big IMO, although there is more than I > >> probably anticipated, initially. We can revisit this later, if needed. > >> > >> > >> >> If you're not going to merge this in the near future, please update the > >> >> rpm test data submodule into a valid commit. Currently, it points into > >> >> some local commit of yours or something as I pointed out in my earlier > >> >> email. > >> > > >> >I'd be happy to pull this in in general. There are some details with > >> > > >> > common/buildpackage: support for different archive formats > >> > > >> >which I need to work out but that can be done afterwards as well. > >> > >> Is there something specific? At least it shouldn't affect (deb > >> )buildpackage at all because zip archives are not supported there. > > > >Mostly misreading the code on my end too early in the morning. I'll have > >to run the performance tests to see if git_archive_submodules regresses > >once merged but that should be about it. > > OK. Merged now. I fixed the case where we don't have the submodules checked out too. Cheers -- Guido _______________________________________________ git-buildpackage mailing list [email protected] http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage
