Re: [fossil-users] Symlink trouble
On 4/8/2015 2:44 PM, David Mason wrote: Here is another problem with symlinks: [add files in a subdirectory] [commit] [move subdirectory outside of repository] [create symlink to subdirectory with same name as original] [Fossil doesn't notice anything happened] Same problem if you move the directory into a nested working directory. Continuing from the previous example (hence the first couple of commands to revert to the original situation): Fossil operates on files, not directories. If it can stat all the files and they all have the same contents as in the repository, it doesn't realize there's a change. You don't even need directories for this: $ f new tmp.fossil $ mkdir tmp $ cd tmp $ f open ../tmp.fossil $ f settings allow-symlinks 1 $ echo hello a $ echo hello b $ f addremove $ f commit -m 1 $ ln -sf a b $ f changes (shows nothing) $ f commit -m 2 (complains that nothing has changed) $ echo goodbye a $ f changes (shows both a and b as having been edited) $ f commit -m 2 $ f artifact current (shows a and b are files with the same checksum) I tossed in an allow-symlinks just to show that this bug is there even when symlinks are explicitly enabled. Removing the allow-symlinks line gives the same failure. To fix (assuming we want to go down this rabbit hole), we must teach Fossil more about where symlinks may lurk. Each path component (subdirectories and files) could be a symlink and must be checked. It's not enough to blindly open files and read their contents. -- Andy Goth | andrew.m.goth/at/gmail/dot/com signature.asc Description: OpenPGP digital signature ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
On 4/9/2015 12:00 PM, David Mason wrote: I use symlinks a lot. I *really* wish fossil handled them properly. This is one of my biggest beefs about fossil. Fossil's driving requirement is to support the development of SQLite, and its applicability anywhere else is just a bonus. Since SQLite does not require symlinks, it is unsurprising that Fossil did not originally support symlinks. Symlink support was added by Dmitry Chestnykh in 2011, done to his satisfaction. It's only natural as time progresses that other users with other use cases flush out omissions in the initial symlink implementation, just like with anything else. In this thread we have identified a few such shortcomings, and it makes sense that we fix them, provided we do so in a way that does not interfere with Fossil's driving requirement. The other big one is that if I set some property (in this case allow-symlinks true) and I also set the corresponding .fossil-setting I get a warning when I do almost anything. I keep intending to do something about these annoyances, but never find the time. I know where and how to fix this warning such that it only shows in the case of discrepancy. You can also look at the andygoth-versioned-open branch to see where the code is, since it's adjacent my largest change. -- Andy Goth | andrew.m.goth/at/gmail/dot/com signature.asc Description: OpenPGP digital signature ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
Thanks for the examples, Andy. I use symlinks a lot. I *really* wish fossil handled them properly. This is one of my biggest beefs about fossil. The other big one is that if I set some property (in this case allow-symlinks true) and I also set the corresponding .fossil-setting I get a warning when I do almost anything. Even if they are saying the same thing (i.e., yes, allow). I definitely should get a warning if they disagree, but the current behaviour is annoying. (And the reason I do the set version is to work around some of points Andy was making at the beginning of this thread. And the reason I do the .fossil-settings version is to document the status in the file-system, or if I were ever to want to put the working directory into another fossil, I want the symlinks handled properly.) I keep intending to do something about these annoyances, but never find the time. ../Dave ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
Andy Goth wrote: My andygoth-versioned-open branch (just checked in) addresses this problem and seems to fix the symlink issue. However, the Fossil coding style is rather alien to me, particularly the way it leaks memory on purpose, so the way I'm doing things may not be the best. Please have a look, and feel free to ask questions and make suggestions and further changes. I've made some tweaks on the branch. Here are the highlights: 1. By changing the return code checking for historical_version_of_file(), which apparently returns greater than zero on success. 2. Set noWarn based on the historical version of that file, if it exists. 3. Unrelated: Removed superfluous slash in the .fossil-settings path used by print_setting(). -- Joe Mistachkin ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
I don't know if it's just me, or if there's a school of thought regarding this, but if this is a case of maintaining symlinks to publish as part of a distribution, I usually relegate their management to a script that will be part of a release generation process (with repository != release in mind). Are the problematic uses of symlinks different from that? On Apr 7, 2015 11:14 PM, Joe Mistachkin sql...@mistachkin.com wrote: Andy Goth wrote: My andygoth-versioned-open branch (just checked in) addresses this problem and seems to fix the symlink issue. However, the Fossil coding style is rather alien to me, particularly the way it leaks memory on purpose, so the way I'm doing things may not be the best. Please have a look, and feel free to ask questions and make suggestions and further changes. I've made some tweaks on the branch. Here are the highlights: 1. By changing the return code checking for historical_version_of_file(), which apparently returns greater than zero on success. 2. Set noWarn based on the historical version of that file, if it exists. 3. Unrelated: Removed superfluous slash in the .fossil-settings path used by print_setting(). -- Joe Mistachkin ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
On 4/8/2015 1:14 AM, Joe Mistachkin wrote: I've made some tweaks on the branch. Thank you, I appreciate it. I hesitated to do much more than move existing code around since I don't know how strong the stylistic preferences are. For example, one thing I wanted to do was treat pointers as booleans, e.g. if(pointer) rather than if(pointer!=0), but the precedent seemed to go against me. I'd like to do refactoring but really don't want to step on toes, especially regarding the memory leak policy: we're already relying on the ultimate garbage collector. Penultimate I mean, since power failure is the ultimate. :^) Here are the highlights: 1. By changing the return code checking for historical_version_of_file(), which apparently returns greater than zero on success. I'm pretty sure it returns its final argument if there's a failure, or panics on failure if its final argument is zero. So it should be sufficient to check if its return value doesn't match its final argument. However, it's not clear what it returns on success (you say greater than zero, but you also say apparently suggesting you don't know the full specification). In my tests I found it returned 1, so when its final argument was 1 it was impossible to distinguish between success and failure. So I went with -1 as the final argument. Sorry, not going to dig into the code just this second, so I don't recall what the names of the parameters. 2. Set noWarn based on the historical version of that file, if it exists. I thought about doing this but figured it didn't matter all that much for the open operation, but I will gladly accept your enhancement. 3. Unrelated: Removed superfluous slash in the .fossil-settings path used by print_setting(). Baseline issue, but again thanks. Also I recall you made it not try to update the cached value of allow-symlinks if there aren't any check-ins in the repository. This is fine, though now that we're forcing the initial empty check-in again, I'm not sure of the benefit. Maybe you're just dodging a NULL dereference in the case of a repository made with an older Fossil. Or maybe you could shun the initial empty check-in and rebuild, but the rebuild might make another initial empty check-in. Not sure, but still this is a good check to add. -- Andy Goth | andrew.m.goth/at/gmail/dot/com ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
On Wed, Apr 8, 2015 at 12:17 PM, Ron W ronw.m...@gmail.com wrote: On Wed, Apr 8, 2015 at 1:58 PM, Scott Robison sc...@casaderobison.com wrote: Or whatever your team dictates. :) In our case, we are required to follow industry guidelines, except where compelling technical issues require a deviation. And such deviations must be documented. Also, use of NULL is considered more indicative of the intent, therefore, more readable. I agree the rationale for using it is valid, and particularly the idea that if(p!=NULL) works whether or not your environment is standards compliant. So I'm not saying you're wrong to use that particular construct, just observing that 0 is not wrong in a standards compliant compiler as it will do the right thing. There are many ways to do the same task. Each has its pros cons. I live mainly in C++ land, so I avoid NULL. I do like the nullptr constant of C++11 and later, since there is no way for it to be accidentally converted to an integer. I happen to like the if(p) syntax as succinct terminology for if(p is valid) (or if(!p) for if(p is not valid)), though. None of them are perfect, and good reasons can be given for any of them IMO. Recently I've been working in C++ code where the programmer liked to use if(TRUE==someBOOL) which I hate on so many levels: 1. If you have to use BOOL due to the Windows API, fine, but limit your use of BOOL to that, use bool the rest of the time. 1a. Same for TRUE FALSE vs true false. 2. I dislike Yoda conditionals of the form constant op variable. I realize there are reasons why people use them, to avoid accidental assignment in a conditional, but the modern compilers I work with diagnose these problems for me so that I can have clearer code without the risk of accidental assignment. 3. If someBOOL returns a true value that happens to not be exactly equal to 1 (the definition of TRUE) then the statement will fail when perhaps maybe it shouldn't have? 3a. If you are really depending on the value to be 0 1 vs zero not zero, don't call it a BOOL. 4. I dislike someBOOL op constant statements. Just say someBOOL or !someBOOL. But I digress. Sorry. Carry on. SDR ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
On 4/8/2015 1:33 AM, bch wrote: I don't know if it's just me, or if there's a school of thought regarding this, but if this is a case of maintaining symlinks to publish as part of a distribution, I usually relegate their management to a script that will be part of a release generation process (with repository != release in mind). Are the problematic uses of symlinks different from that? I prefer your approach, however I did not get to pick in this instance since I am trying to import an existing repository from ClearCase, actually a snapshot, and it uses symlinks. Furthermore I think some of the symlinks are used stupidly, but once again I don't get to pick. -- Andy Goth | andrew.m.goth/at/gmail/dot/com ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
(email to reporter of problem several years ago, copying list so discussion can continue here) I made a fix to Fossil opening a repository containing symlinks. It's currently on a branch. For details, see this thread: http://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg20112.html And here's code: http://www.fossil-scm.org/index.html/timeline?r=andygoth-versioned-open -- Andy Goth | andrew.m.goth/at/gmail/dot/com ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
On 4/8/15, Andy Goth andrew.m.g...@gmail.com wrote: On 4/8/2015 1:33 AM, bch wrote: I don't know if it's just me, or if there's a school of thought regarding this, but if this is a case of maintaining symlinks to publish as part of a distribution, I usually relegate their management to a script that will be part of a release generation process (with repository != release in mind). Are the problematic uses of symlinks different from that? I prefer your approach, however I did not get to pick in this instance since I am trying to import an existing repository from ClearCase, actually a snapshot, and it uses symlinks. Furthermore I think some of the symlinks are used stupidly, but once again I don't get to pick. 1) It's nice to hear that others are like this 2) That you imported a (in our opinion) poorly-laid-out project is a good point to remember -- it's not always greenfield repositories that we work with. Thanks for the reminder. Cheers, -- Andy Goth | andrew.m.goth/at/gmail/dot/com ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
On Wed, Apr 8, 2015 at 11:13 AM, Ron W ronw.m...@gmail.com wrote: FWIW, the coding rules I work under require us to write if(pointer!=NULL) because the invalid pointer is a compiler-dependent value. I've actually used a compiler where NULL was not 0. Instead it was 0x. Presumably this was partially because this the address of the last byte of the reset vector (as well as being the highest addressable byte) and because the erased/unwritten value of a byte of Flash ROM is 0xFF. (At for every Flash ROM device I've ever (directly) used.) For any standard compliant C compiler, the zero comparison is legal regardless of the bit pattern representation of a null point on the given platform. The 1990 ISO standard (virtually identical to the 1989 ANSI standard) includes the following (from section 6.2.2.3 Pointers): An integral constant expression with the value 0, or such an expression cast to type void *, is called a null pointer constant. If a null pointer constant is assigned to or compared for equality to a pointer. the constant is converted to a pointer of that type. Such a pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function. Two null pointers, converted through possibly different sequences of casts to pointer types, shall compare equal. So even though the internal bit pattern representation might not be zero, the standard guarantees zero can be used as the null pointer constant value. C++ changes the rules slightly, in that void* types won't auto cast like in C, so only 0 was a valid null pointer constant (prior to the introduction of nullptr). I don't include NULL as a null pointer constant as it is only a macro that expands to a null pointer constant. So unless one is dealing with a non-standard compiler, if(p) and if(p!=0) and if(p!=NULL) are all semantically identically. One could argue that if(p!=NULL) is the safest of the available options, as it would work on all compilers that define NULL. If you're dealing with a non-standard compiler (which would include pre-standard compilers), use what you have to use for the platform. Otherwise use whatever you feel most comfortable with. Or whatever your team dictates. :) -- Scott Robison ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
On Wed, Apr 8, 2015 at 1:58 PM, Scott Robison sc...@casaderobison.com wrote: Or whatever your team dictates. :) In our case, we are required to follow industry guidelines, except where compelling technical issues require a deviation. And such deviations must be documented. Also, use of NULL is considered more indicative of the intent, therefore, more readable. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
Here is another problem with symlinks: Last login: Tue Apr 7 20:11:50 on ttys004 : ~ ; cd /tmp : /tmp ; fs init foo.fossil project-id: d24564a4337e8c50f77a20ee355e2f76a9b84b78 server-id: aa025469a22046732337b7fa075c7c4e85f45c0a admin-user: dmason (initial password is d5a283) : /tmp ; cd foo : /tmp/foo ; fs open ../foo.fossil project-name: unnamed repository: /private/tmp/foo/../foo.fossil local-root: /private/tmp/foo/ config-db:/Users/dmason/.fossil project-code: d24564a4337e8c50f77a20ee355e2f76a9b84b78 checkins: 0 : /tmp/foo ; mkdir foo : /tmp/foo ; touch foo/bar foo/blat : /tmp/foo ; fs add foo ADDED foo/bar ADDED foo/blat : /tmp/foo ; fs ci -m foo New_Version: d01c99b9316832532f4acdf7b1afb06e9c071e43 : /tmp/foo ; mv foo ../foob : /tmp/foo ; ln -s ../foob foo : /tmp/foo ; fs stat repository: /private/tmp/foo/../foo.fossil local-root: /private/tmp/foo/ config-db:/Users/dmason/.fossil checkout: d01c99b9316832532f4acdf7b1afb06e9c071e43 2015-04-08 19:20:41 UTC leaf: open tags: trunk comment: foo (user: dmason) : /tmp/foo ; ls -l total 8 lrwxr-xr-x 1 dmason wheel 7 8 Apr 15:21 foo - ../foob : /tmp/foo ; fs stat repository: /private/tmp/foo/../foo.fossil local-root: /private/tmp/foo/ config-db:/Users/dmason/.fossil checkout: d01c99b9316832532f4acdf7b1afb06e9c071e43 2015-04-08 19:20:41 UTC leaf: open tags: trunk comment: foo (user: dmason) EDITED foo/bar : /tmp/foo ; As you can see if a directory is replaced with a link to a directory outside of the working directory it doesn't recognize it. And it thinks that files in that directory (which is outside the WD) are still to be tracked. This bit me when I was moving several projects into fossil, and I recognized a common subdirectory (TeX to be exact), so I did something like the above. Took a while to figure out what went wrong and to fix it. Same problem if you move the directory into a nested working directory. Continuing from the previous example (hence the first couple of commands to revert to the original situation): : /tmp/foo ; rm foo;mv ../foob foo : /tmp/foo ; cd .. : /tmp ; fossil init bar.fossil project-id: a52834ead71c0589bfe88c2874c45fb05818d3d4 server-id: 9b79fad662c144ea54dca3ab3f5b5fc522030c0d admin-user: dmason (initial password is d4f221) : /tmp ; cd foo : /tmp/foo ; fs stat repository: /private/tmp/foo/../foo.fossil local-root: /private/tmp/foo/ config-db:/Users/dmason/.fossil checkout: d01c99b9316832532f4acdf7b1afb06e9c071e43 2015-04-08 19:20:41 UTC leaf: open tags: trunk comment: foo (user: dmason) EDITED foo/bar : /tmp/foo ; mkdir bar : /tmp/foo ; cd bar : /tmp/foo/bar ; fs open --nested ../../bar.fossil project-name: unnamed repository: /private/tmp/foo/bar/../../bar.fossil local-root: /private/tmp/foo/bar/ config-db:/Users/dmason/.fossil project-code: a52834ead71c0589bfe88c2874c45fb05818d3d4 checkins: 0 : /tmp/foo/bar ; cd .. : /tmp/foo ; ls bar foo : /tmp/foo ; mv foo bar : /tmp/foo ; ln -s bar/foo . : /tmp/foo ; fs stat repository: /private/tmp/foo/../foo.fossil local-root: /private/tmp/foo/ config-db:/Users/dmason/.fossil checkout: d01c99b9316832532f4acdf7b1afb06e9c071e43 2015-04-08 19:20:41 UTC leaf: open tags: trunk comment: foo (user: dmason) EDITED foo/bar : /tmp/foo ; cd bar : /tmp/foo/bar ; fs ext foo/bar foo/blat : /tmp/foo/bar ; So now we have the same files in 2 repositories. Not the expected (nor, I suspect, intended) behaviour. Moving and symlinking *within* a repo seems to work properly. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
What Scott says, abbreviated from the C FAQ: http://c-faq.com/null/ptrtest.html FWIW, I always use if(p) to verify a pointer is valid. ../Dave ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Symlink trouble
My naïve user expectation is checking in .fossil-settings/allow-symlinks with contents 1 is all I need to do to get symlinks to work in a Fossil repository. It does make it possible to check in symlinks. However, it doesn't help when opening a new checkout. In this situation, symlinks are created as ordinary files containing the link target, just like on Windows. The problem is db_open_repository() uses db_get_boolean() to get the value of allow-symlinks. db_get_boolean() uses db_get() uses db_get_versioned() uses blob_read_from_file() which reads on-disk files. Of course, on-disk files don't exist until after the checkout is opened, so no versioned settings can be read right at the start of db_open_repository(). By the time it's possible to read the versioned settings, it's too late, and the symlinks have already been demoted to normal files. Things get really nasty when the symlinks point to executable files. Performing a commit after the open demoted the symlinks now results in execute permission cleared changes, and the new manifest records that the former symlinks are now just non-executable files. db_get_versioned() bails out if !g.localOpen since it knows it's not going to be able to read the files anyway. But instead of bailing out when not open, I think it would be better for it to pull the value out of the repository instead of an on-disk file, analogous to: fossil cat -R repo.fossil .fossil-settings/allow-symlinks My andygoth-versioned-open branch (just checked in) addresses this problem and seems to fix the symlink issue. However, the Fossil coding style is rather alien to me, particularly the way it leaks memory on purpose, so the way I'm doing things may not be the best. Please have a look, and feel free to ask questions and make suggestions and further changes. Test case: f new test.fossil mkdir -p test/.fossil-settings cd test f open ../test.fossil echo 1 .fossil-settings/allow-symlinks ln -s asdf fdsa f addremove -dotfiles f commit -m test f close cd .. rm -rf test mkdir test cd test f open ../test.fossil ls -l fdsa should show a symlink, not a regular file. See also: https://groups.google.com/forum/#!topic/fossil-users/L6yrc2cQfGE -- Andy Goth | andrew.m.goth/at/gmail/dot/com signature.asc Description: OpenPGP digital signature ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Symlink trouble
On 4/8/2015 12:15 AM, Andy Goth wrote: My naïve user expectation is checking in .fossil-settings/allow-symlinks with contents 1 is all I need to do to get symlinks to work in a Fossil repository. It does make it possible to check in symlinks. However, it doesn't help when opening a new checkout. In this situation, symlinks are created as ordinary files containing the link target, just like on Windows. Oh, forgot to mention that everything works just fine if you don't use versioned settings and instead use the [fossil settings] command directly. However, this is inconvenient because every user of a repository has to run the same command. -- Andy Goth | andrew.m.goth/at/gmail/dot/com signature.asc Description: OpenPGP digital signature ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
