Re: [fossil-users] enhanced-symlink branch

2017-10-16 Thread Andy Goth 

On 10/16/17 21:13, Warren Young wrote:

On Oct 16, 2017, at 7:28 PM, Andy Goth  wrote:

I don't have the luxury of Cygwin because my end users won't have it.


You can just distribute the DLL, then.


The two programs that would need Cygwin are Fossil itself and my
application.  My application is written in Tcl and is bundled into a
Starpack using a precompiled basekit.  Said basekit is used both as the
interpreter stub (placed inside the Starpack) and the standalone
interpreter used to run the build script that actually creates the
first Starpack.  Only in the second role does it actually need to know
anything about symlinks because they are used to define the layout of
the Starpack virtual filesystem.  Should I get a custom Cygwin-capable
basekit, I'd be adding a dependency on Cygwin that complicates my
install procedure without providing any runtime benefit.  Or I could
have two basekits, but that gives me an extra multi-megabyte blob
providing no capability over and above what I have right now.

Let me say that again.  What I have works right now, and it's been
working ever since the early days of my project.  I see zero reason to
incorporate Cygwin simply to avoid having to use the existing Fossil
emulation of symlinks which already works fine for me.

I'm curious, how far back does the Cygwin DLL work?  I need to support
all the way back to Windows 2000.  Actually there's one Windows 98
system, but until they fix the network on it I can't support it anyway.
Going the other direction, I think the newest system I have to support
is Windows 7, although I do test on Windows 8.1.


It's already like pulling teeth (don't get me started) to get them to
allow fossil.exe to be on the install CD


You can have an EXE but not a DLL linked to that EXE?  I suppose
dynamic linkage does increase the attack surface area, but it’s a
pretty thin argument.


The end user is simultaneously paranoid about security and ignorant
about security.  Please don't get me started.  The whole reason I was
given this project was to deal with this fact, and thus the slightest
thing I do that they perceive as complicated (even if you and I know
it's not) will cause me to fail in my task because they decline to adopt
my solution and instead persist in manually doing the things that got
them in trouble in the first place, and we are being held responsible
for the consequences of their actions.  It's not a happy situation.
Let's not discuss it further.


I'm not the one doing the reinventing.


Except you are: there are at least 4 other existing implementations of
symlinks for Windows, and you’ve just built a fifth.


No I'm not.  Fossil symlinks in Windows have always been ordinary files
containing the names of their targets.  That wasn't me.  I'm merely
exploiting this longstanding behavior because it's straightforward and
does what I need.


No, the current behavior for native Windows Fossil is to create
ordinary files containing the name of their target (no trailing
newline).


I see why people keep trying to fix the problem, then.


Right, it's fiddly and takes great care to use properly.  Changing it is
far outside the scope of my current effort.  Others have already put
forward alternatives on branches, but I'm not using them.  I'm sticking
with the bog-standard Fossil trunk here.  I'm already taking heat for
using Fossil in the first place, even though the alternative CM system
I'm expected to use flat out doesn't work in the required development
environment.  For me to not only use Fossil but also some odd branch
thereof would really push everyone over the edge.

My work is focused on reflecting the list of symlinks back into the
filesystem in the form of a manifest file.  Granted, I could also have
rewritten my code to parse the manifest file actually named "manifest"
(not manifest.*) to find every line with the "l" permission, but I
already wrote my application to read a newline-delimited list of
filenames that does nothing more than list the symlinks, so I went with
that simple format.


Native Windows EXEs can call Cygwin EXEs, and vice versa.  If the Tcl
code doesn’t need to follow Fossil-created symlinks, Tcl needn’t be
built under Cygwin.


The Tcl code does need to follow the symlinks.  The part of it that
actually gets exposed to symlinks, I wrote to first look aside at the
symlink list file so it knows which files to treat as such.  Had I used
Cygwin Fossil and Cygwin Tcl, that would all have been handled for me,
at the cost of adding a runtime dependency on Cygwin and needing custom
binaries of both that I can't be sure will work on all the required
platforms, with a benefit experienced only during the Starpack build
process that only developers will do.


Though if you wanted, the Cygwin package repository should have a wide
variety of Tcl stuff.


Not the Starpack basekits though.  Getting those to work right on
Windows 2000 was enough of a chore.  I don't expect Cygwin would make
that

Re: [fossil-users] enhanced-symlink branch

2017-10-16 Thread Warren Young 
On Oct 16, 2017, at 7:28 PM, Andy Goth  wrote:
> 
> I don't have the luxury of Cygwin because my end users won't have it.

You can just distribute the DLL, then.

In cases where your users *may* have Cygwin installed, it is considered polite 
to install the DLL only when it doesn’t already exist on the target system, so 
that the preexisting one is used where present.  If you don’t, you create two 
independent Cygwin worlds on the machine, which can lead to interop problems 
when one Cygwin program calls another that is using a different DLL.

> It's already like pulling teeth (don't get me started) to get them to
> allow fossil.exe to be on the install CD

You can have an EXE but not a DLL linked to that EXE?  I suppose dynamic 
linkage does increase the attack surface area, but it’s a pretty thin argument.

I’m not aware of any independent audit of Cygwin, but it’s got to be at least 
as well community-tested as Fossil itself, having a much larger audience.

> I'm not the one doing the reinventing.

Except you are: there are at least 4 other existing implementations of symlinks 
for Windows, and you’ve just built a fifth.

Yes, I get your point, Microsoft invented 3 of those before you got there, and 
they all suck in some ways — *.lnk, MKLINK, and WSL — but fixing that just 
leads to the XKCD “Standards” problem:

   https://xkcd.com/927/

At least if you go with Cygwin, you’re just making use of an existing standard, 
which is widely recognized.

>> 1. Just copy files on checkout, and try to do the right thing with
>> changes to “linked” files on checkin.  (Is this not the current
>> behavior for native Windows builds?)
> 
> No, the current behavior for native Windows Fossil is to create ordinary
> files containing the name of their target (no trailing newline).

I see why people keep trying to fix the problem, then.

>> 2. Use Cygwin...https://cygwin.com/licensing.html
> 
> That licensing update is good to know, but as I said elsewhere in my
> email, the code is developed and deployed using Tcl Starpacks, and on
> Windows I prefer to use the standard builds thereof rather than make
> Cygwin versions.

Native Windows EXEs can call Cygwin EXEs, and vice versa.  If the Tcl code 
doesn’t need to follow Fossil-created symlinks, Tcl needn’t be built under 
Cygwin.

Though if you wanted, the Cygwin package repository should have a wide variety 
of Tcl stuff.

>> 4. Fake symlinks somehow, like Cygwin does in its default mode.
>> (Cygwin’s symlink handling is complex; thus #2.  See:
>> https://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks)
>> Ranked still lower because it’s reinventing a perfectly good existing
>> wheel.
> 
> Faking symlinks is exactly what I did all along.

My point #4 is a broad category, but my choice to treat all of the solutions 
that fall into it as a single option doesn’t mean all options in the category 
are equal.  Using one of the existing options is better than inventing a new 
one, particularly if it’s widely recognized.

By that reckoning, I’d rank *.lnk above Cygwin symlinks in many regards.  Why 
wouldn’t that work?

>> Why is your way superior to all of these?
> 
> I'm simply refining option #4.

No, you’re adding a new thing to the #4 bag.  Not the same thing as selecting 
something already in the #4 bag.

>> My parents had a mechanical typewriter that didn’t even have a “1”
>> key: you were expected to use the “l” key.  I don’t remember if it
>> also lacked a 0 key.
> 
> Mine didn't have "!" (no "1" so where would they put it?) because it's
> easy enough to type period-backspace-apostrophe.

I found one on eBay: it has a 0 and no !.  :) 

>> There must be a better way to enable this: “2”, capital “L”, etc.
> 
> I think "s" is the best choice if it's not deemed important to use the
> same letter as the manifest file uses.

Doesn’t that scheme mean you can’t have normal manifests plus symlink 
manifests?  Maybe you should go with “2” so that “3” = 1 | 2, giving both files.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] enhanced-symlink branch

2017-10-16 Thread Andy Goth 
On 10/16/2017 5:44 PM, Arseniy Terekhin wrote:
> I want to share my thought about symlinking in fossil.
>
>    dir "my_empy_dir"
>    dir "my_empy_dir2"
>    ln "existing_file_in_repo" "new_link"
>    ln_hard "existing_directory_in_repo" "new_hard_link"
>    attr_executable "bin/*.sh"
>    attr_hidden "*.cab"
> 
> That setting can be applied whenever "empty-dirs" is currently applied.

Everything you suggest can be placed in a makefile or other such script,
no need to change Fossil to handle it.

-- 
Andy Goth | 



signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] enhanced-symlink branch

2017-10-16 Thread Warren Young 
On Oct 16, 2017, at 4:44 PM, Arseniy Terekhin  wrote:
> 
> What should "ln" do on FAT16/32 and exFAT filesystems? Well, I'd just expect 
> fossil to not bother creating any links and maybe give a warning.

Or, write two copies of the file, then on checkin either: a) take the later of 
the two files as the new truth; or b) ignore changes to the linked-to file, and 
accept changes only to the linked-from file.

See, not so simple.

> On Windows XP+ I'd expect native NTFS links.

That’s even more complex, because Microsoft in their infinite wisdom chose to 
make symlink creation a privileged action by default, requiring Admin privs out 
of the box.  And I don’t mean UAC here, I mean “Run as administrator.”

There’s some kind of policy setting to change this, but then you’d be requiring 
that every Fossil user make that change if they want to allow symlinks.

And then on top of all that, the MKLINK cmd.exe command takes its arguments 
backwards relative to ln, cp, COPY…

It’s like they didn’t want you to use this feature, that they put it in only to 
stop people asking for it.

You know, at this point I think I’d accept *.lnk files.  At least Explorer 
copes with them, and it would avoid the conundrum from the first part of this 
email.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] enhanced-symlink branch

2017-10-16 Thread Arseniy Terekhin 
I want to share my thought about symlinking in fossil.
I like that fossil manages files only and not other things like
directories*, permissions, symlinks and maybe ACLs. There is "empty-dirs"
setting and I'm okay with it, since it can be stored as file. And I think
all these things are OS specific and should be managed identically. I mean,
I expect  them to be managed in different setting groups. Or maybe create
one group, since there would be too many settings. Where content could be
something like this:

   dir "my_empy_dir"
   dir "my_empy_dir2"
   ln "existing_file_in_repo" "new_link"
   ln_hard "existing_directory_in_repo" "new_hard_link"
   attr_executable "bin/*.sh"
   attr_hidden "*.cab"

That setting can be applied whenever "empty-dirs" is currently applied.
What should "ln" do on FAT16/32 and exFAT filesystems? Well, I'd just
expect fossil to not bother creating any links and maybe give a warning. On
Windows XP+ I'd expect native NTFS links.

That's all. Hope I was clear, it's 1:30AM here.


On Mon, Oct 16, 2017 at 10:49 PM, Andy Goth  wrote:

> On 10/14/2017 5:16 PM, Andy Goth wrote:
> > Please review the enhanced-symlink branch.  I can't test it properly
> > this weekend because I don't have Windows anywhere at home.
>
> Tested on Windows 7, works just the way my project needs it to work.
> The manifest.symlinks file is created when the "l" flag is present in
> the manifest setting.  On Windows (not Unix), its contents determine
> what files are and are not considered to be symlinks.  Editing the file
> influences the changes, status, and commit commands.
>
> I'd like to merge enhanced-symlink to trunk, but honestly I am afraid to
> just do so even though it passes all my tests because there's been zero
> mailing list discussion, aside from talking about Windows symlink
> functionality present on other branches which were themselves never
> merged to trunk.
>
> I still do not know my original change was moved off trunk.  It didn't
> break compatibility, and it did provide a significant part of the needed
> functionality: listing symlinks in a way I can see them in Windows,
> short of parsing the manifest (another question I asked about that never
> got any replies on the mailing list).  I asked for clarification for my
> code being moved off trunk but never got any.
>
> Now I see my most recent trunk check-in has also been moved off trunk,
> despite also not being a compatibility issue, with no explanation to
> either mailing list, my private email, or the check-in comment.  I'm
> talking about http://fossil-scm.org/index.html/info/eb4dda482056b1db.
>
> Was there some policy change about committing to trunk that I missed out
> on?  Before I spend any more time working on Fossil I need to know that
> I'm not doing something unwelcome.
>
> --
> Andy Goth | 
>
>
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>


-- 
I remain,
Arseniy Terekhin
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] enhanced-symlink branch

2017-10-16 Thread Warren Young 
On Oct 14, 2017, at 4:16 PM, Andy Goth  wrote:
> 
> Please review the enhanced-symlink branch.

Aside from a manifest.symlinks file appearing at the project root, what should 
we see?  How do we test it?

For example, if I declare a file to be a symlink in the repository by editing 
that file, what is supposed to happen on “fossil up”?  What if the so-named 
file is already under management? 

I ask for the benefit of others, not myself, because in the uncommon instances 
where I run Fossil on Windows, it’s almost always under Cygwin or WSL.  In the 
exceedingly rare exceptions, I use one of the official binaries; I’ve never 
built a non-Cygwin fossil.exe.

I did go ahead and build your branch on Linux and under Cygwin, and all I was 
able to see is that “fossil set manifest l” caused the manifest.symlinks file 
to appear, and “fossil unset manifest” made it disappear.

> On Windows, this new file can be edited to change which files are and are not 
> considered to be symlinks. Thus, it becomes possible to create and unlink 
> symlinks on Windows.

If all this branch did is gave Fossil the power to declare where the symlinks 
are, I’d say it’s probably a harmless change.

The fact that manually adjusting the manifest apparently changes Fossil’s 
behavior worries me.  It feels like reinventing part of Unix, poorly.

It also feels like reversing a causality arrow: shouldn’t manifests merely 
declare separately-verifiable facts?  You can’t change the contents of a cargo 
container by rewriting its manifest; why do you call your file a manifest if it 
doesn’t behave the same way?

That in turn begs the question of why you have rejected the other proposed ways 
of handling symlinks on Windows?  Let me see if I can enumerate them all:

1. Just copy files on checkout, and try to do the right thing with changes to 
“linked” files on checkin.  (Is this not the current behavior for native 
Windows builds?)

2. Use Cygwin or WSL.  Fob the headache of how to map semantics off onto those 
subsystems.  You are aware that Red Hat has relicensed Cygwin to avoid the need 
to buy out the redistribution restriction for executables linked to Cygwin, 
yes?  (https://cygwin.com/licensing.html)  Cygwin Fossil symlinks work sanely.

3. Use native NT symlinks.  (One of the available branches does this.)  This 
does mean you either require Admin privs or must set the user policy 
thingamabob that allows normal users to create native symlinks, which is why I 
rank this lower than the Cygwin/WSL option.

4. Fake symlinks somehow, like Cygwin does in its default mode.  (Cygwin’s 
symlink handling is complex; thus #2.  See: 
https://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks)  Ranked still 
lower because it’s reinventing a perfectly good existing wheel.

Have I missed any?

Why is your way superior to all of these?

> If the manifest setting's value is "1" (one) not "l" (ell), the 
> manifest.symlinks file is disabled.

That’s a poor choice of letter, then.  It’s inherently confusing.

My parents had a mechanical typewriter that didn’t even have a “1” key: you 
were expected to use the “l” key.  I don’t remember if it also lacked a 0 key.

There must be a better way to enable this: “2”, capital “L”, etc.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] enhanced-symlink branch

2017-10-16 Thread Richard Hipp 
On 10/16/17, Andy Goth  wrote:
>
> I'd like to merge enhanced-symlink to trunk,

I'm not comfortable with that.  My reasons are long and inarticulate.
I need to write them up, but i"m a little busy right now.  For the
moment, let's just leave the symlink stuff on a branch.
-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] enhanced-symlink branch

2017-10-16 Thread Andy Goth 
On 10/14/2017 5:16 PM, Andy Goth wrote:
> Please review the enhanced-symlink branch.  I can't test it properly
> this weekend because I don't have Windows anywhere at home.

Tested on Windows 7, works just the way my project needs it to work.
The manifest.symlinks file is created when the "l" flag is present in
the manifest setting.  On Windows (not Unix), its contents determine
what files are and are not considered to be symlinks.  Editing the file
influences the changes, status, and commit commands.

I'd like to merge enhanced-symlink to trunk, but honestly I am afraid to
just do so even though it passes all my tests because there's been zero
mailing list discussion, aside from talking about Windows symlink
functionality present on other branches which were themselves never
merged to trunk.

I still do not know my original change was moved off trunk.  It didn't
break compatibility, and it did provide a significant part of the needed
functionality: listing symlinks in a way I can see them in Windows,
short of parsing the manifest (another question I asked about that never
got any replies on the mailing list).  I asked for clarification for my
code being moved off trunk but never got any.

Now I see my most recent trunk check-in has also been moved off trunk,
despite also not being a compatibility issue, with no explanation to
either mailing list, my private email, or the check-in comment.  I'm
talking about http://fossil-scm.org/index.html/info/eb4dda482056b1db.

Was there some policy change about committing to trunk that I missed out
on?  Before I spend any more time working on Fossil I need to know that
I'm not doing something unwelcome.

-- 
Andy Goth | 



signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users