> -----Original Message----- > From: Bert Huijben [mailto:b...@qqmail.nl] > Sent: zaterdag 14 mei 2011 14:36 > To: Greg Stein > Cc: dev@subversion.apache.org > Subject: RE: svn commit: r1102912 - in /subversion/trunk/subversion: > include/private/svn_wc_private.h libsvn_client/prop_commands.c > libsvn_wc/externals.c libsvn_wc/wc_db.c tests/libsvn_wc/db-test.c > > They would have to live in a layer below base if we want them in nodes > (op depth -1 or MAXINT?)
Or in Working, as that would be free. (The only valid presence would be BASE-DELETED, which would be calculated via shadowing) > And there are only a few places where we do want to see them. Certainly > not as just another node in a directory, but only accessible directly > via its path. They are 'external' and really from a different > world/origin and unrelated for all wc handling except conflicts. On almost every command we only want to process file externals if they are the explicit target of the operation (never via depth or as children of a parent) add - No blame -Only as explicit target cat -Only as explicit target changelist - No checkout - No cleanup - n/a commit -Only as explicit target copy -Only as explicit target delete - Not supported diff -Only as explicit target export -Only as explicit target or via definition as external import - No info -Only as explicit target list - n/a lock -Only as explicit target log - Only explicit merge - Only explicit mergeinfo - Only explicit mkdir - n/a move - No prop* -Only as explicit target resolve - Only as explicit target revert - (unknown) status - Reported as unversioned but/external? switch -n/a unlock -Only as explicit target update -Only as explicit target > They don't belong in BASE, and keeping them there brings all kinds of > problems. Just look at that list of issues and that isn't anywhere near > complete. Moving them to working or a reserved depth (+ applying special presence) would fix this, but then we would still have to check for their presence in all loops over the children of a directory. A special TOP depth (>=WORKING) would allow reusing some NODES functions that help on installing files, but this wouldn't work over obstructing working copies. A depth < BASE would not allow supporting externals as the same path of not-present, excluded and absent nodes, where we do support file externals. > Moving them out of BASE would also allow handling conflicts between > them and a node that really lives at that place in the directory. > (Currently unsolvable in 1.7 as we can't have two op depth 0 nodes at > one local_abspath. This would also apply to a not already used depth != BASE. > But even with op depth -1 we need more work to fix the other > limitations of file externals. Different repositories and file > externals in subdirs are not going to work from nodes without adding > checks EVERYWHERE. This applies to all storage in nodes. Libsvn_client and many parts of libsvn_wc outside wc_db just assume that all nodes in a directory are from one and the same repository. > With externals 100% in EXTERNALS that problem would be solved > everywhere and we only need some additional checks in libsvn_client > while libsvn_wc can just assume that a directory has its own children. It looks like with one more helper the current test suite works with file externals completely moved to the EXTERNALS table. The reason I took this approach instead of storing them in a new way in NODES is that this allows adding file externals scenarios later while making the behavior of libsvn_wc more stable, by guaranteeing that a directory without switched children always has a complete set of children. (Required for stable merges, etc.). If we store them in NODES all working copy code has to know about them and we have to check everywhere, while only a small subset of operations wants to look at them as working copy files. For the rest they are just unversioned obstructions. Like how we handle directory externals. Currently all major tree updaters (update, merge, the adm crawler) are specialized to skip file externals and where we don't skip them we most likely didn't find the problems yet. (Status is the only exception). Enabling file externals from different repositories is impossible without major rework which is far more involving than moving file externals to EXTERNALS. But I look forward to hearing your solution to the file-externals problems, which resolves all these current problems and allows plugging the other limitations of file externals... without adding conditions everywhere where we use NODES. [continuing in your original reply] > > Bert Huijben (Cell phone) From: Greg Stein > Sent: zaterdag 14 mei 2011 14:01 > To: Bert Huijben > Cc: dev@subversion.apache.org > Subject: Re: svn commit: r1102912 - in /subversion/trunk/subversion: > include/private/svn_wc_private.h libsvn_client/prop_commands.c > libsvn_wc/externals.c libsvn_wc/wc_db.c tests/libsvn_wc/db-test.c > On Sat, May 14, 2011 at 04:00, Bert Huijben <b...@qqmail.nl> wrote: > > > > > >> -----Original Message----- > >> From: Greg Stein [mailto:gst...@gmail.com] > >> Sent: zaterdag 14 mei 2011 7:06 > >> To: dev@subversion.apache.org > >> Subject: Re: svn commit: r1102912 - in /subversion/trunk/subversion: > >> include/private/svn_wc_private.h libsvn_client/prop_commands.c > >> libsvn_wc/externals.c libsvn_wc/wc_db.c tests/libsvn_wc/db-test.c > >> > >> On Fri, May 13, 2011 at 18:23, <rhuij...@apache.org> wrote: > >> > Author: rhuijben > >> > Date: Fri May 13 22:22:59 2011 > >> > New Revision: 1102912 > >> > > >> > URL: http://svn.apache.org/viewvc?rev=1102912&view=rev > >> > Log: > >> > Make the svn_client_proplist3 funtion capable of reading properties > from > >> > new style file externals. > >> > >> Huh? I thought file externals had NODES and ACTUAL_NODE rows? > >> Shouldn't the properties be available there? What are we doing > >> *special casing* externals like this? > > > > See the document you asked me to write: notes/wc-ng/externals > > > > They live in NODES now, but not after we bump to format 29. (They > can/will > > probably survive in ACTUAL_NODES as they share the same path anyway) > > > > After we bump to format 29 file externals will be "just like normal > > externals". They won't be part of the parent working copy NODES tree. > (And > > can come from different repositories; can be placed in subdirs; etc. etc.). > > > > But of course they will still share the parent wc's db and pristine store as > > they can't have their own administrative area. > > I just read it, and I think that I disagree with the approach. > > It now seems that we are going to have to special-check *everywhere* > to see if a node exists as a file external. Every single place we look > for a node, will now require a check to see if a file external lives > at that spot. Where would that be? No, we don't want to look at them instead of a node. In all normal operations we want the nodes in the working copy, not the unversioned and external files that share the same path namespace. Externals are 'external'. Currently we copy all nodes in a directory, except file-externals. When merging we apply non inheritable mergeinfo on the directory.. and then mergeinfo on all nodes inside except the file external. When we are crawling a working copy for revision information or switches (the adm crawler; svnversion; etc.) we skip file externals. > Special casing doesn't seem like a good approach. We already do that *everywhere*. I would like to get rid of that, by only checking for file externals where we need that support. And that would (in most cases) be inside libsvn_client, where we process externals. Not in libsvn_wc where we manage a working copy and not the not working-copy files inside it. > Couldn't you have used a new presence value in the database to note > these? The NODES schema already allows for different repositories > (repos_id). All the data would live in NODES just like every other > item. No need to go and look in a different table (which already looks > much like NODES). I don't think we can do that without severe performance penalties. Every additional db read transaction in a loop is a slowdown and you are asking for a lot of these here. We would have to start checking repos_id everywhere. And then we would still have the problem that switched nodes and file 'externals' are the same thing. Moving them to a different layer might help... But it also makes NODES db operations slower, while we usually don't want to see them anywhere. (We should default to not seeing them, instead of to seeing them as a normal node) > I'd rather see file externals look more like regular nodes, instead of > some ephemeral thing. I would like to hear your solution. And a timescale on when we can get this major blocker for 1.7 done. I don't think we can release with the current status of file externals ignored in the adm crawler, update, etc. We need a solution in this space. I don't think EXTERNALS is the ultimate solution to this problem... But it works and it is easy to maintain and extend in the future. We can always move the data to somewhere else, but at least we don't pollute NODES like we did with svn_wc_entry_t. (Currently file externals break +- every NODES validation rule that we have: BASE nodes with only WORKING directories....) Bert > > Cheers, > -g
