I appreciate the effort going in to describing multi-repro workflows, however they still sound alarmingly complex and error-prone to me.
The bug that triggered this thread was in fact the failure of the v8 presubmit to find its buildtools/ dir. This could be remedied with a searchpath for buildtools: try local, then go up, right? Then we could support both independent v8 development and chromium/v8 development. Nick Verne On Tue, Dec 20, 2016 at 7:02 AM, Primiano Tucci <[email protected]> wrote: > As long as your chromium's DEPS and the version of the full v8 checkout > are consistent I think you can mount --bind ~/v8/src ~/chrome/v8/src > I never tried this in v8 but was using for a similar case (back in the > days when I was building webview from the android tree but uploading cls > from the chromium tree). > To be clear this will break for sure the ability to gclient sync from > chromium (because the mounted v8/src will have no .git folder, as in a > standalone v8 checkout src is just a folder of the top-level v8 project), > so you'll have to remember to unmount and remount before gclient syncing. > However should give you the ability to use git cl from the v8 folder and > all the standalone v8 checkout, but still build and test your changes from > both the chrome and v8 tree without having to do commit, push and pull > sequences. > All this unless the DEPS of standalone v8 changes the layout of folders > within v8/src, in which case all this might not work. In other words, just > try, you might be just lucky if this hack works :) > > On Mon, Dec 19, 2016 at 6:13 PM Jakob Kummerow <[email protected]> > wrote: > >> The main point of using git-new-workdir is actually easy synchronization >> between the two workdirs, which also addresses the "upload often" concern. >> I have my IDE pointed at my main V8 checkout and do all my edits there. >> From a shell that's cd'ed to that same checkout I can upload as early and >> as often as I want. In order to test the current state of things, I "git >> commit -a --amend" in the main repo, switch to the shell that's cd'ed to >> the Chromium checkout, and "git checkout -f <branch>" there to pull in the >> recently-committed changes, then compile and test. So the overhead compared >> to a standard one-repo-one-workspace scenario is minimal, just the "git >> commit"/"git checkout" pair of commands. >> >> It just so happened that I got a chance to try out the builtin >> git-worktree after writing my previous mail. Turns out it has one >> difference to git-new-workdir in that it doesn't allow the same branch to >> be checked out in two workdirs (which makes sense for safety reasons). The >> workaround is simple: have a branch for Chromium's V8 checkout, and instead >> of "git checkout -f <branch>" do "git reset --hard <branch>". To clarify: >> commits in the main checkout go to branch B1, the secondary checkout is at >> branch B2, and B2 keeps getting hard-reset to B1. >> >> It's probably possible to skip the directory moving and just keep the >> custom V8 directory in the Chromium checkout, but I haven't tried this. >> "gclient sync" should check out a given revision, so all it takes to get >> back to testing your own stuff is "git checkout B2" (+"git reset --hard B1" >> as needed). >> >> On Mon, Dec 19, 2016 at 12:38 PM, Aaron Gable <[email protected]> >> wrote: >> >> That's not actually it -- gclient *is* able to check out DEPS of the v8 >> subrepo. The problem is that, when embedded in src.git, v8 doesn't *want* to >> check out its DEPS, because many of them are shared with the rest of >> chrome, so it uses the versions supplied there. If chromium were >> restructured so that v8 was supposed to supply all of its own dependencies, >> then this would be much simpler. The src.git/DEPS file would simply specify >> "recursedeps = ['v8']" and all of v8's deps would be checked out. >> >> If you want to change the way this works, talk to the OWNERS of >> src.git/DEPS :) >> >> On Sun, Dec 18, 2016 at 3:42 PM Matt Giuca <[email protected]> wrote: >> >> Thanks for the detailed replies. >> >> I have to say neither of these workflows seem particularly nice for >> extended development. I prefer Jeremy's (that's the one I set up for Nick) >> because it doesn't involve moving directories around. (Also use of >> git-worktree seems to just be a disk saving exercise, not actually helping >> simplify the workflow.) Neither fetch time nor disk usage are the issue; >> the issue is developer time when doing an edit/build/test/upload loop. >> >> The problem is both Jakob and Jeremy's workflow involves a "final upload" >> -- what about iterative development? I like to upload early and often, and >> the upload is the hard part of this workflow since you need to: >> 1. Make sure your main repo does *not* have the branch checked out (since >> that will block push). >> 2. From the chrome repo: git push. >> 3. From the main repo: git checkout <branch>. >> 4. From the main repo: git cl upload. >> >> Not to mention how hard it is to manage multiple stacked branches in this >> workflow (for example, for git push to work from the chrome repo, the >> branch upstream needs to be the remote branch, so you can't have a local >> branch hierarchy). >> >> I really can't understand why doing all your work from inside a sub-repo >> of the chrome repo isn't a supported workflow. Literally the only thing >> missing is gclient being able to check out DEPS of the v8 repo. That's the >> only difference between the v8 inside chrome and the "main" standalone v8 >> repo, and it would surely make development much easier? >> >> On Sat, 17 Dec 2016 at 06:56 Paweł Hajdan, Jr. <[email protected]> >> wrote: >> >> +infra-dev,machenbach >> >> On Fri, Dec 16, 2016 at 8:18 PM, Jeremy Roman <[email protected]> >> wrote: >> >> My workflow has been to have two repositories (a standalone one and a >> Chromium one), but with the standalone one added as a remote of the >> chromium one. >> >> Mine are /src/chromium/src and /src/v8/v8 >> >> $ cd /src/chromium/src/v8 >> $ git remote add standalone /src/v8/v8 >> >> Then I can fetch revisions from the standalone repo, check them out, >> push/pull, etc., and do the final upload from the standalone supported repo. >> >> Works well enough for me. >> >> >> On Fri, Dec 16, 2016 at 12:36 PM, Jakob Kummerow <[email protected]> >> wrote: >> >> Working on V8 from within a Chromium checkout's src/v8/ directory is not >> a supported workflow. There are various ways to hack it, but The Official >> Way is to get a separate V8 checkout with "fetch v8" (which should be >> fairly quick, so the only cost is a bit of disk space). >> >> For testing local changes, I have a secondary workdir of my standalone V8 >> checkout in my Chromium checkout (which I created with git-new-workdir, >> apparently "git worktree" is how you'd do it now), which I rename as needed >> (because symlinking used to confuse ninja, not sure if it still does): >> # setup >> $ cd /work/v8/ >> $ git worktree add /work/chromium/src/v8-custom >> >> # implement new feature >> $ git checkout -b my-feature >> $ ... >> $ git commit -am 'yay new feature' >> >> # test feature >> $ cd /work/chromium/src >> $ git pull && gclient sync # let "gclient sync" see the official "v8" >> directory >> $ mv v8 v8-official >> $ mv v8-custom v8 >> $ (cd v8; git checkout -f my-feature) >> $ ninja ... >> >> # cleanup when done >> $ mv v8 v8-custom >> $ mv v8-official v8 >> >> # land it >> $ cd /work/v8/ >> $ git cl upload >> >> You can iterate on the patch with: >> $ cd /work/v8/ >> $ edit... >> $ git commit -a --amend >> $ cd /work/chromium/src >> $ (cd v8; git checkout -f my-feature) >> $ ninja ... >> >> On Fri, Dec 16, 2016 at 1:10 AM, Daniel Cheng <[email protected]> >> wrote: >> >> I can't say I recommend this, but for limited development, I just upload >> with --bypass-hooks and let the v8_presubmit bot make sure I didn't do >> anything bad. >> It's probably still worth pulling down an actual v8 checkout; I found >> running the v8 tests to be a bit painful without an actual v8 checkout. >> >> >> Daniel >> >> On Thu, Dec 15, 2016 at 8:56 PM Trent Apted <[email protected]> wrote: >> >> I encountered something similar with WebRTC. It has a helpful message >> when you try to `git cl upload` (basically: you *must* create a separate >> checkout): >> >> Creating CLs from this location is not supported! Please make sure the >> current >> working directory is the parent directory of this directory. >> If you're working with a Chromium checkout, you'll have to create a full >> WebRTC >> checkout and upload a CL from that. See >> https://webrtc.org/native-code/development/ for instructions. >> >> >> >> V8 has instructions here - https://github.com/v8/v8/wiki/Using%20Git - >> that seem up-to-date. (tl;dr: `fetch v8` should work) >> >> You might be able to save some electrons with something like >> >> $ mkdir v8 # at the level above your chromium/.gclient >> $ gclient-new-workdir.py chromium/src/v8 v8/v8 >> $ cd v8/v8 >> $ git checkout master >> $ cd .. >> $ cat -> .gclient >> solutions = [ >> { >> "url": "https://chromium.googlesource.com/v8/v8.git";, >> "managed": False, >> "name": "v8", >> "deps_file": "DEPS", >> "custom_deps": {}, >> }, >> ] >> ^D >> >> That will symlink all the git objects for v8 so they're not >> re-downloaded, and share disk blocks. But you'll still get dupes of v8's >> DEPS. >> >> Getting much more creative probably risks wedging your chromium checkout >> into a weird state :/ >> >> On 16 December 2016 at 15:18, Matt Giuca <[email protected]> wrote: >> >> Hi, >> >> TL;DR: How do you work on V8 (build, test and upload CLs) from within the >> v8 subrepo of a Chromium checkout (without having to make a separate v8 >> checkout)? >> >> I was helping Nick get started on V8 development within Chromium and we >> couldn't find any way to run the presubmit from within the v8 directory >> that's checked out inside the chromium repo. >> >> That is, we would like to be able to do: >> ~/chrome/src$ cd v8/src >> ~/chrome/src/v8/src$ git cl presubmit >> >> However, if you do that, you get: >> >> Traceback (most recent call last): >> File "/usr/local/google/home/mgiuca/chrome/depot_tools/git_cl.py", >> line 5654, in <module> >> sys.exit(main(sys.argv[1:])) >> File "/usr/local/google/home/mgiuca/chrome/depot_tools/git_cl.py", >> line 5636, in main >> return dispatcher.execute(OptionParser(), argv) >> File "/usr/local/google/home/mgiuca/chrome/depot_tools/subcommand.py", >> line 252, in execute >> return command(parser, args[1:]) >> File "/usr/local/google/home/mgiuca/chrome/depot_tools/git_cl.py", >> line 4222, in CMDpresubmit >> change=cl.GetChange(base_branch, None)) >> File "/usr/local/google/home/mgiuca/chrome/depot_tools/git_cl.py", >> line 1685, in RunHook >> gerrit_obj=self._codereview_impl.GetGerritObjForPresubmit()) >> File >> "/usr/local/google/home/mgiuca/chrome/depot_tools/presubmit_support.py", >> line 1247, in DoPresubmitChecks >> results += executer.ExecPresubmitScript(presubmit_script, filename) >> File >> "/usr/local/google/home/mgiuca/chrome/depot_tools/presubmit_support.py", >> line 1157, in ExecPresubmitScript >> result = eval(function_name + '(*__args)', context) >> File "<string>", line 1, in <module> >> File "<string>", line 287, in CheckChangeOnCommit >> File "<string>", line 260, in _CommonChecks >> File "<string>", line 101, in _CheckUnwantedDependencies >> ImportError: No module named checkdeps >> >> Turns out this is because the v8 directory is missing the buildtools >> subdirectory which is checked out by its DEPS. Basically, when gclient >> checks out all the subrepos of chrome (including v8) it doesn't recursively >> check out the DEPS' DEPS. So we can't submit v8 from there and maybe some >> other things are broken too. >> >> If you make a separate checkout of v8 (outside of chrome), and run >> gclient sync, it's fine. But then you can't test your changes in Chrome, >> resulting in tedious copying of git patch data back and forth between the >> two checkouts of v8. It would be much better if you could just do the full >> v8 edit/build/test/upload workflow from within the v8 checkout inside >> chrome. >> >> I don't want gclient to be recursive by default. I just want to be able >> to set it up so it fetches v8. >> >> We found this old (looks obsolete) documentation describing exactly how >> to do this: >> http://www.chromium.org/developers/how-tos/chromium- >> modularization#Advanced_Usage >> >> - It gives the example of v8 but then says "The V8 example is >> somewhat simple since V8 does not itself have other dependencies." That's >> no longer true since V8 *does* have other dependencies. This example >> doesn't get what we want. >> - Then it goes on to say how to do webkit (no longer applicable) and >> get webkit's dependencies. But we tried setting up .gclient with v8 >> similar >> to how webkit is set up there, and it didn't work. I won't go into the >> details of how it didn't work (can discuss on request). >> >> How do you do this? I'd imagine most v8 engineers have a workflow that >> solves this problem. There must be a way to have gclient check out the DEPS >> of v8 without needing to make a separate v8 checkout. >> >> Thanks >> >> Matt & Nick >> >> -- >> -- >> Chromium Developers mailing list: [email protected] >> View archives, change email options, or unsubscribe: >> http://groups.google.com/a/chromium.org/group/chromium-dev >> >> >> -- >> -- >> Chromium Developers mailing list: [email protected] >> View archives, change email options, or unsubscribe: >> http://groups.google.com/a/chromium.org/group/chromium-dev >> >> -- >> -- >> v8-dev mailing list >> [email protected] >> http://groups.google.com/group/v8-dev >> --- >> You received this message because you are subscribed to the Google Groups >> "v8-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> -- >> Chromium Developers mailing list: [email protected] >> View archives, change email options, or unsubscribe: >> http://groups.google.com/a/chromium.org/group/chromium-dev >> >> >> -- >> -- >> Chromium Developers mailing list: [email protected] >> View archives, change email options, or unsubscribe: >> http://groups.google.com/a/chromium.org/group/chromium-dev >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "infra-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> To view this discussion on the web visit https://groups.google.com/a/ >> chromium.org/d/msgid/infra-dev/CAHqYdcbG_NbSusovoNBX4S1gT3C_akWkF- >> RMFAJU%3D8QPw-CkgA%40mail.gmail.com >> <https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAHqYdcbG_NbSusovoNBX4S1gT3C_akWkF-RMFAJU%3D8QPw-CkgA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "infra-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> To view this discussion on the web visit https://groups.google.com/a/ >> chromium.org/d/msgid/infra-dev/CAKSzg3TQB%3D9jvCJ%3D7BPxmHm2Lx- >> 0DKu5PzQCqDB9QP8JuWmi6A%40mail.gmail.com >> <https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAKSzg3TQB%3D9jvCJ%3D7BPxmHm2Lx-0DKu5PzQCqDB9QP8JuWmi6A%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- -- v8-dev mailing list [email protected] http://groups.google.com/group/v8-dev --- You received this message because you are subscribed to the Google Groups "v8-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
