Re: [fossil-users] filename contains illegal characters

2012-12-01 Thread Stefan Bellon
On Sat, 01 Dec, Richard Hipp wrote:
> On Sat, Dec 1, 2012 at 4:02 AM, Stefan Bellon 
> wrote:
> 
> > That's the part I am interested in and I think this should go into
> > trunk.
> 
> Added to trunk here: http://www.fossil-scm.org/fossil/info/647bb7b79f

Silly me, I just looked at the comment and not the code. Now I'm a
happy bunny. :-)

Greetings,
Stefan

-- 
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-12-01 Thread Jan Nijtmans
2012/12/1 Stefan Bellon :
> sbellon@slim$ fossil status | egrep "(checkout|tags)"
> checkout: 99ab9fb47cfb20106564012b7678ca5d579060af 2012-12-01 04:49:38 UTC
> tags: trunk
> sbellon@slim$ grep "not contain any of these characters" src/file.c
> ** *  Does not contain any of these characters in the path: "\*[]?"

Comment was not updated accordingly. Done now. Thanks!

Regards,
  Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-12-01 Thread Richard Hipp
On Sat, Dec 1, 2012 at 4:02 AM, Stefan Bellon  wrote:

> ** *  Does not contain any of these characters in the path: "\*[]?"
>
> That's the part I am interested in and I think this should go into
> trunk.
>


Added to trunk here: http://www.fossil-scm.org/fossil/info/647bb7b79f

-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-12-01 Thread Stefan Bellon
On Sat, 01 Dec, Jan Nijtmans wrote:

> I agree with Richard on this. The most important part of the patch is
> already on trunk now: allowing '[', ']', '"', '*' and '?' in
> filenames.

Really? That's the part I am interested in, but I do not see that
merged into trunk:

sbellon@slim$ fossil status | egrep "(checkout|tags)"
checkout: 99ab9fb47cfb20106564012b7678ca5d579060af 2012-12-01 04:49:38 UTC
tags: trunk
sbellon@slim$ grep "not contain any of these characters" src/file.c 
** *  Does not contain any of these characters in the path: "\*[]?"

That's the part I am interested in and I think this should go into
trunk. It makes the code even simpler and not more complex because it
removes an exceptional case which is questionable anyway.

Regarding the Unicode part of the patch, I agree that this may need
some further attention.

> The added value of the remaining part is minimal: It would
> make the win32 build of fossil do the same as the cygwin build of
> fossil. So, you have the functionality already: Just build fossil.exe
> for cygwin, it does exactly the same.

For some reasons, cygwin is no option here.

> That's not sufficient excuse to rush it into 1.25. And missing is
> still a test-case description of exactly what is the added value:
> What can you do with the changed version what cannot be
> done with the trunk version.

Well ... having filenames with [ ] " * and ? in the repository and in
checkouts if the underlying OS supports it.

Greetings,
Stefan

-- 
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-12-01 Thread Jan Nijtmans
2012/12/1 Richard Hipp :
>> I would very much like to see this going into the upcoming 1.25
>> release. Is it scheduled to get merged back into trunk?
>
>
> No.  This adds complication to a code base that is complicated enough
> already.  It is vitally important to keep Fossil as simple as possible, in
> order to promote reliability and longevity. There are no plans to merge this
> branch.

I agree with Richard on this. The most important part of the patch is
already on trunk now: allowing '[', ']', '"', '*' and '?' in filenames. The
added value of the remaining part is minimal: It would make the
win32 build of fossil do the same as the cygwin build of fossil.
So, you have the functionality already: Just build fossil.exe
for cygwin, it does exactly the same.

That's not sufficient excuse to rush it into 1.25. And missing is
still a test-case description of exactly what is the added value:
What can you do with the changed version what cannot be
done with the trunk version.

Thanks!
   Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-30 Thread Richard Hipp
On Fri, Nov 30, 2012 at 5:43 PM, Stefan Bellon  wrote:

> On Tue, 27 Nov, Jan Nijtmans wrote:
>
> > This version allows all characters in filenames that
> > are permissible on UNIX, except '\'. And those files
> > can be checked out using Cygwin and Windows as
> > well, using Cygwin's solution: In the file system
> > translate those characters to 'safe' characters in
> > the range U+F000 to U+F0FF (which should never
> > itself occur in valid fossil cards)
> >
> > I cannot find any security-risks with this. The only
> > possible security risk I found is unrelated to this
> > change, and it is already fixed in [897dfa48b4]
> > (Thanks, Richard!)
> >
> > Anyone who thinks this is a bad idea? Please, speak
> > up now, otherwise Richard might approve this!
>
> I would very much like to see this going into the upcoming 1.25
> release. Is it scheduled to get merged back into trunk?
>

No.  This adds complication to a code base that is complicated enough
already.  It is vitally important to keep Fossil as simple as possible, in
order to promote reliability and longevity. There are no plans to merge
this branch.


>
> Greetings,
> Stefan
>
> --
> Stefan Bellon
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-30 Thread Stefan Bellon
On Tue, 27 Nov, Jan Nijtmans wrote:

> This version allows all characters in filenames that
> are permissible on UNIX, except '\'. And those files
> can be checked out using Cygwin and Windows as
> well, using Cygwin's solution: In the file system
> translate those characters to 'safe' characters in
> the range U+F000 to U+F0FF (which should never
> itself occur in valid fossil cards)
> 
> I cannot find any security-risks with this. The only
> possible security risk I found is unrelated to this
> change, and it is already fixed in [897dfa48b4]
> (Thanks, Richard!)
> 
> Anyone who thinks this is a bad idea? Please, speak
> up now, otherwise Richard might approve this!

I would very much like to see this going into the upcoming 1.25
release. Is it scheduled to get merged back into trunk?

Greetings,
Stefan

-- 
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-27 Thread Jan Nijtmans
2012/11/25 Jan Nijtmans :

> New test version in [d3bee356ba]. It changes the
> "filename contains illegal characters" fatal into
> a warning, which should make your situation more
> managable: Even though files containing '\' still
> cannot be handled, they don't stop the
> "fossil addremove" anymore from adding other
> files.

New version, including the "unicode-glob" setting
in [71cd228b0e].

This version allows all characters in filenames that
are permissible on UNIX, except '\'. And those files
can be checked out using Cygwin and Windows as
well, using Cygwin's solution: In the file system
translate those characters to 'safe' characters in
the range U+F000 to U+F0FF (which should never
itself occur in valid fossil cards)

I cannot find any security-risks with this. The only
possible security risk I found is unrelated to this
change, and it is already fixed in [897dfa48b4]
(Thanks, Richard!)

Anyone who thinks this is a bad idea? Please, speak
up now, otherwise Richard might approve this!

Regards,
Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-25 Thread Jan Nijtmans
2012/11/22 Jan Nijtmans :
> 2012/11/22 Stefan Bellon :
>> On Thu, 22 Nov, Jan Nijtmans wrote:
>>
>>> Stefan, could you try out [e6a1910fa8]?
>>
>> This sounds very cool. :-)

New test version in [d3bee356ba]. It changes the
"filename contains illegal characters" fatal into
a warning, which should make your situation more
managable: Even though files containing '\' still
cannot be handled, they don't stop the
"fossil addremove" anymore from adding other
files.

Please allow some time to Richard considering this.
The currently ticketing-rework has clearly much
more priority than this.

Regards,
  Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-22 Thread Jan Nijtmans
2012/11/22 Stefan Bellon :
> On Thu, 22 Nov, Jan Nijtmans wrote:
>
>> Stefan, could you try out [e6a1910fa8]?
>
> This sounds very cool. :-)
>
> However I still get:
>
> $ ~/tmp/fossil addremove
> [...]
> /home/bellonsn/tmp/fossil: filename contains illegal characters: str\i\ng.h
...
> Am I missing something?

No, you are not missing anything. The backslash is still forbidden, but all
other characters should be allowed with this change.

It 'could' be possible to do something about backslash
as well, but that will need some more clever thinking. I
would like this change to be reviewed and accepted first,
before trying to do more:  Adapting '\' as well could
impose security risks I cannot foresee, so it deserves
a separate discussion/review.

Regards,
Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-22 Thread Stefan Bellon
On Thu, 22 Nov, Jan Nijtmans wrote:

> Stefan, could you try out [e6a1910fa8]?

This sounds very cool. :-)

However I still get:

$ ~/tmp/fossil addremove
[...]
/home/bellonsn/tmp/fossil: filename contains illegal characters: str\i\ng.h

This is with:

$ ~/tmp/fossil version
This is fossil version 1.24 [e6a1910fa8] 2012-11-22 09:32:23 UTC

I am on a x86_64 Debian GNU/Linux box. And I did the following to
build this version of fossil:

$ mkdir ~/tmp && cd ~/tmp
$ fossil clone http://fossil-scm.org/ fossil.fossil
$ fossil open fossil.fossil e6a1910fa8
$ ./configure
$ make

Am I missing something?

Greetings,
Stefan

-- 
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-22 Thread Jan Nijtmans
2012/11/20 Richard Hipp :
> I think that there needs to be two separate versions of
> fossil_utf8_to_unicode(), or perhaps a parameter on one function, so that
> some strings can get the conversion of problematic characters to the unicode
> private-use range, while others go unaltered.

Done now in [e6a1910fa8]. Only conversions involving filenames handle the
problematic characters, all other Unicode<->utf-8 conversions do not.
I think it's ready to be merged to trunk. See below for the tests I did.

> Some uses are less clear.  For example, at
> http://www.fossil-scm.org/fossil/artifact/33d79f5b0a2?ln=850 we are
> converting an entire command-line, which does likely contain some filenames,
> but also might contain other text where *?"<>| are legitimate characters.

Thinking more on it: fossil_system() should not do this special conversion:
It only translates from utf-8 to unicode, while the started fossil instance
will just do the conversion the other way around. The two conversions
just need to be symmetrical, which means that fossil_system() must
not do any special magic and surely not make any assumption on
how the command line looks like.

Stefan, could you try out [e6a1910fa8]? I did the following tests with
it, and all work fine:
- Compiled  [e6a1910fa8] on Linux (Ubuntu 12.4, 64-bit), cygwin and
win32 (with mingw-w64)) Created a repository with 'strange' filenames
containing characters like ':', '<', '[', '"' and even newline.
- With fossil  [e6a1910fa8] everything works fine: I can checkout all
  files from the repository just fine on all platforms. On Cygwin/Windows,
  the correct filename translation is done: It looks strange from Windows
  Explorer, but from the cygwin shell everything looks as expected.
- With fossil 1.24, the files from the 'offending' commit cannot be
  checked-out (we already knew that), but also the commit which
  contained the 'invalid' filenames is simply not visible at all!
  That's nice (something I didn't expect)! So, after this change is on
  trunk, it operates well with older fossil versions: 'offending' commits
  are completely invisible for older-fossil users on the timeline, even on
  the "branches" page. They wouldn't even know the commit is there,
  until they upgrade fossil and do a "fossil all rebuild". I'm impressed!

I cannot think of any reason any more why [e6a1910fa8] could
be bad for fossil. All my tests have good results. I don't see
any security risk doing this. I couldn't make fossil crash on
this, even not with older fossil versions.

Richard, what more needs to be done to get this on trunk,
so all discussions on what characters should be valid
can finally mute  The new (proposed) rule is then: The
only forbidden character in filenames is the backslash
(and NUL and  '/', of course). If a platform has more invalid
characters in filenames, fossil translates that to another
(unicode) character which is safe.
For Windows, the cygwin-algorithm is used for this,
so fossil works flawlessly together with the cygwin shell.

Feedback welcome!

Regards,
 Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-20 Thread Stefan Bellon
On Tue, 20 Nov, Richard Hipp wrote:

> Some uses are less clear.  For example, at
> http://www.fossil-scm.org/fossil/artifact/33d79f5b0a2?ln=850 we are
> converting an entire command-line, which does likely contain some
> filenames, but also might contain other text where *?"<>| are
> legitimate characters.  I think doing the conversion will work OK for
> every case where Fossil calls fossil_system(), but we need to be
> careful here.

I think you are mixing two separate aspects here. One aspect is the
encoding of the input string, the other aspect is the set of acceptable
characters for a certain use case (e.g. allowed filenames on Windows).

What about reflecting those two aspects as separate functionality, i.e.
the encoding conversion should apply to *all* input data regardless of
whether read from a file, supplied as command line argument or even
read via an environment variable (however what encoding should be
assumed is quite a different matter). Then the check regarding allowed
character set can be applied on the already converted internal unicode
representation.

But perhaps I am missing some implementation details I am not aware of.

Greetings,
Stefan

-- 
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-20 Thread Jan Nijtmans
2012/11/20 Richard Hipp :
> I think that there needs to be two separate versions of
> fossil_utf8_to_unicode(), or perhaps a parameter on one function, so that
> some strings can get the conversion of problematic characters to the unicode
> private-use range, while others go unaltered.

I guess you are completely right! My patch just does it the
'quick-and-dirty' way. It needs more eyes to be sure it
is done correctly. Just see it as a proof-of-concept.

Feel free to improve things on the branch! I will continue
to try it as well.

Thanks! That was a quick response!

Regards,
 Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-20 Thread Richard Hipp
On Tue, Nov 20, 2012 at 8:59 AM, Jan Nijtmans wrote:

> 2012/11/19 Stefan Bellon :
> > I think this restriction
> >
> >   *  Does not contain any of these characters in the path: "\*[]?"
> >
> > is too arbitrary. If you try to create a file or directory in the
> > Windows explorer and type e.g. a " then you get a bubble help with the
> > information that you must not use any of the following characters
> >
> >\ / : * ? " < > |
>
> An EXPERIMENTAL fix is committed now in [82ce90f91c]. It
> handles the invalid characters in the same way as cygwin
> does: 
> In cygwin it is no problem at all to use such characters in
> filenames, except for the backslash. I found ':' problematic
> as well, as it is used in UNIX as path separator (path1:path2)
> and on Windows as drive separator (C:\foo).
>

I think that there needs to be two separate versions of
fossil_utf8_to_unicode(), or perhaps a parameter on one function, so that
some strings can get the conversion of problematic characters to the
unicode private-use range, while others go unaltered.  Clearly, any string
that is used as a filename gets converted.  But, for example, the password
for the "fossil winsrv create" command should not get the character
conversions, because characters like * ? " < > and | can legitimately occur
in passwords.

Some uses are less clear.  For example, at
http://www.fossil-scm.org/fossil/artifact/33d79f5b0a2?ln=850 we are
converting an entire command-line, which does likely contain some
filenames, but also might contain other text where *?"<>| are legitimate
characters.  I think doing the conversion will work OK for every case where
Fossil calls fossil_system(), but we need to be careful here.

Perhaps two functions:

 fossil_filename_to_unicode()   // Does the conversion
 fossil_utf8_to_unicode() // Omits the conversion

Or else a extra boolean parameter to fossil_utf8_to_unicode() that
enables/disables the conversion.



>
> The advantage of this method is that using a cygwin-compiled fossil
> will have exactly the same result as using a win32 version of fossil.
> The cygwin shell will show all filenames as they are expected to
> be. From Windows explorer they might look strange, but
> nothing goes wrong.
>
> The only remaining problematic characters are '\' and ':', all
> others are - on Windows - translated to a safe Unicode range.
> At this moment I don't see a way to work around the two
> remaining problematic characters.
>
> I think this deserves a lot more testing. I will
>
> Regards,
>  jan Nijtmans
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-20 Thread Jan Nijtmans
2012/11/19 Stefan Bellon :
> I think this restriction
>
>   *  Does not contain any of these characters in the path: "\*[]?"
>
> is too arbitrary. If you try to create a file or directory in the
> Windows explorer and type e.g. a " then you get a bubble help with the
> information that you must not use any of the following characters
>
>\ / : * ? " < > |

An EXPERIMENTAL fix is committed now in [82ce90f91c]. It
handles the invalid characters in the same way as cygwin
does: 
In cygwin it is no problem at all to use such characters in
filenames, except for the backslash. I found ':' problematic
as well, as it is used in UNIX as path separator (path1:path2)
and on Windows as drive separator (C:\foo).

The advantage of this method is that using a cygwin-compiled fossil
will have exactly the same result as using a win32 version of fossil.
The cygwin shell will show all filenames as they are expected to
be. From Windows explorer they might look strange, but
nothing goes wrong.

The only remaining problematic characters are '\' and ':', all
others are - on Windows - translated to a safe Unicode range.
At this moment I don't see a way to work around the two
remaining problematic characters.

I think this deserves a lot more testing. I will

Regards,
 jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-19 Thread Stefan Bellon
On Mon, 19 Nov, Richard Hipp wrote:

> I made the decision to disallow "problematic" characters in Fossil
> filenames 6 years ago.  In theory, Fossil could allow these
> characters in filenames.  But doing so would enhance the potential
> for bugs and would greatly complicate testing of the code.  And I
> didn't see any value in doing so.  Simpler is better.

Yes, agreed. The question is just whether restricting characters is
"simpler" or whether those special Windows exclude rules are rather
"more complicated". See below for the reasoning.

> The code that checks for valid filenames is here:
> 
> http://www.fossil-scm.org/fossil/artifact/91e4570f9462?ln=479-513

I think this restriction

  *  Does not contain any of these characters in the path: "\*[]?"

is too arbitrary. If you try to create a file or directory in the
Windows explorer and type e.g. a " then you get a bubble help with the
information that you must not use any of the following characters

   \ / : * ? " < > |

So either all of those should get rejected by fossil's pathname filter
or none at all.

But if all of those get rejected, there may be other operating systems
out there which have other restrictions (e.g. RISC OS does not allow
for a full stop in a filename because . and / are swapped in RISC OS'
file system). The above restricted characters are just the ones for
Windows.

In fact you get into the same kind of trouble with casing as well. On
case-sensitive file systems you can add files called

  foobar.txt
  FooBar.txt
  FOOBAR.TXT

and so on. But you cannot check them out on a case-insensitive file
system (like on Windows) because they would end up in the same file. If
you wanted to make sure this cannot happen, you'd have to forbid adding
differently cased versions of a filename on e.g. UNIX just because it
could cause trouble checking out on Windows.

That leads me back to my original statement that I think that "simple"
means allowing all filenames. I think the "checkout" code that creates
files should be able of handling such failures (warn the user but
continue) instead having the "add" code decide whether a filename is
"nice enough".

> You are welcomed to modify this in any way you see fit.  If you want
> to submit a patch, we will consider adding it.

I'm not yet familiar with fossil's code and what consequences a change
at this code part could mean for other areas, i.e. which assumptions on
filenames in fossil's internal data structures exist elsewhere.
Therefore I'm a bit reluctant to perform such a change myself.

Greetings,
Stefan

-- 
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-19 Thread Richard Hipp
On Mon, Nov 19, 2012 at 7:39 AM, Stefan Bellon  wrote:

> On Mon, 19 Nov, Richard Hipp wrote:
>
> > The \ character is problematic because Windows cannot contain that
> > character in filenames (except as a separator between directory and
> > file names).  If you were allowed to check this file into Fossil,
> > then the repository could not be checked out on Windows.
>
> But the same is true for
>
>   str"i"ng.h
>
> This file can be added on GNU/Linux to fossil but cannot be checked out
> on Windows.
>
> This seems to me an inconsistency. Either support all allowed filenames
> per OS and skip them on OSes that cannot support them, or go with the
> set of filenames that all OSes can support (at least the mainstream
> ones).
>
> But being able to add str"i"ng.h but not str\i\ng.h seems strange to me.
>

I made the decision to disallow "problematic" characters in Fossil
filenames 6 years ago.  In theory, Fossil could allow these characters in
filenames.  But doing so would enhance the potential for bugs and would
greatly complicate testing of the code.  And I didn't see any value in
doing so.  Simpler is better.

The code that checks for valid filenames is here:

http://www.fossil-scm.org/fossil/artifact/91e4570f9462?ln=479-513

You are welcomed to modify this in any way you see fit.  If you want to
submit a patch, we will consider adding it.



>
> Greetings,
> Stefan
>
> --
> Stefan Bellon
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-19 Thread Jan Nijtmans
2012/11/19 Stefan Bellon :
> This seems to me an inconsistency. Either support all allowed filenames
> per OS and skip them on OSes that cannot support them, or go with the
> set of filenames that all OSes can support (at least the mainstream
> ones).
>
> But being able to add str"i"ng.h but not str\i\ng.h seems strange to me.

Agreed. I tried it in cygwin, and there I can add files with special
characters just fine. But cygwin uses a trick to accomplish that:


So, a solution could be, when checking out files on Windows which
contain invalid characters there, translate those characters to
characters in the range U+F000 - U+F0FF.

This way, files with quotes can be inserted in fossil just fine. When
using the cygwin shell, everything works fine. When using another
shell, the filename might look strange (it looks like:  "string.h")
but apart from that everything could be made to work fine.

Regards,
 Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-19 Thread Stefan Bellon
On Mon, 19 Nov, Richard Hipp wrote:

> The \ character is problematic because Windows cannot contain that
> character in filenames (except as a separator between directory and
> file names).  If you were allowed to check this file into Fossil,
> then the repository could not be checked out on Windows.

But the same is true for

  str"i"ng.h

This file can be added on GNU/Linux to fossil but cannot be checked out
on Windows.

This seems to me an inconsistency. Either support all allowed filenames
per OS and skip them on OSes that cannot support them, or go with the
set of filenames that all OSes can support (at least the mainstream
ones).

But being able to add str"i"ng.h but not str\i\ng.h seems strange to me.

Greetings,
Stefan

-- 
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-19 Thread Richard Hipp
On Mon, Nov 19, 2012 at 6:37 AM, Stefan Bellon  wrote:

> Hi!
>
> On Mon, 19 Nov, Jan Nijtmans wrote:
>
> > This sounds like a bug in the SVN ->fossil converter, not
> > in fossil itself.
>
> Just to avoid any misunderstanding: I am not using any converter. I
> just did
>
> $ cd svn_working_copy
> $ fossil init myproject.fossil
> $ fossil open --keep myproject.fossil
> $ fossil addremove
>
> And then got the message.
>
> > Quotes are not illegal in fossil, but the backslash character is.
>
> Ok, so then this *IS* the problem. I am trying to add a filename called
>
>   str\i\ng.h
>
> to the repository.
>

The \ character is problematic because Windows cannot contain that
character in filenames (except as a separator between directory and file
names).  If you were allowed to check this file into Fossil, then the
repository could not be checked out on Windows.



>
> So there's no way around than to omit this file from the fossil
> repository and generated it on the fly with some Makefile?
>
> Greetings,
> Stefan
>
> --
> Stefan Bellon
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-19 Thread Stefan Bellon
Hi!

On Mon, 19 Nov, Jan Nijtmans wrote:

> This sounds like a bug in the SVN ->fossil converter, not
> in fossil itself.

Just to avoid any misunderstanding: I am not using any converter. I
just did

$ cd svn_working_copy
$ fossil init myproject.fossil
$ fossil open --keep myproject.fossil
$ fossil addremove

And then got the message.

> Quotes are not illegal in fossil, but the backslash character is.

Ok, so then this *IS* the problem. I am trying to add a filename called

  str\i\ng.h

to the repository.

So there's no way around than to omit this file from the fossil
repository and generated it on the fly with some Makefile?

Greetings,
Stefan

-- 
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2012-11-19 Thread Jan Nijtmans
2012/11/19 Stefan Bellon :
> Now I was trying to convert an existing SVN repository to fossil and
> stumbled across this problem:
>
> ADDED str'i'ng.h
> ADDED str"i"ng.h
> ADDED str"ing.h
> fossil: filename contains illegal characters: str\i\ng.h

This sounds like a bug in the SVN ->fossil converter, not
in fossil itself. Quotes are not illegal in fossil, but the
backslash character is.

Proof:

$ fossil add "str'i'ng.h"
ADDED  win/str'i'ng.h
$ fossil status
...
ADDED  str'i'ng.h

See, no problem to add it at all.

Regards,
 Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2010-06-29 Thread Michal Suchanek
On 29 June 2010 05:31, Michael Richter  wrote:
> On 29 June 2010 02:18, Eric  wrote:
>>
>> [] are there for the same reason as * and ?:
>> ~ $ ls -d p[lu]*
>> play  public_html
>
> Ah.  I was unaware of that expansion.  I always used something like p{l,u}*
> in those situations.
>

Why this choice?

The only characters a filename cannot not contain are / and \0. On
windows there are some more and there are some reserved filenames
carried over from DOS. . and .. have special meaning, that's all
right.

On the other hand while [] and *? can be expanded under some
circumstances so can {} () <> $ # @ ~ & | ; and whatnot but all can be
part of a filename.

Thanks

Michal
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2010-06-28 Thread Michael Richter
On 29 June 2010 02:18, Eric  wrote:

> [] are there for the same reason as * and ?:
>
> ~ $ ls -d p[lu]*
> play  public_html
>

Ah.  I was unaware of that expansion.  I always used something like p{l,u}*
in those situations.

-- 
"Perhaps people don't believe this, but throughout all of the discussions of
entering China our focus has really been what's best for the Chinese people.
It's not been about our revenue or profit or whatnot."
--Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2010-06-28 Thread Eric
On 28 June 2010 at 3:26 pm, "Michael Richter" wrote:
> On 28 June 2010 21:20, Kevin Greiner  wrote:
>
>> For a few files I see the following error:
>>fossil: filename contains illegal characters: prep_20100113[1] clean/
>> 135816_0001.ps
>>
>> I understand it's the square brackets that are causing this error but
>> not
>> why this is by design. Could someone explain the rationale?
>>
>> Also, is there a way to get these files into fossil without renaming
>> them?
>>
>
> Well, looking in file.c for the function file_is_simple_pathname (failure
> of
> which causes that error in add.c), I see:
>
> /*
> ** Return true if the filename given is a valid filename for
> ** a file in a repository.  Valid filenames follow all of the
> ** following rules:
> **
> ** *  Does not begin with "/"
> ** *  Does not contain any path element named "." or ".."
> ** *  Does not contain any of these characters in the path: "\*[]?"
> ** *  Does not end with "/".
> ** *  Does not contain two or more "/" characters in a row.
> ** *  Contains at least one character
> */
>
> So it's definitely being done by design.  Now the * and ? ban I can see.
>  Files with that in the name would cause confusion on almost every
> platform.
>  Some quick tests, though, see that files like "test[1].txt" are permitted
> under both Linux and Windows so I can't quite understand the justification
> there.  (Under Windows a file with [] in the name has to be put in quotes,
> mind, but so do files with spaces.)

[] are there for the same reason as * and ?:

~ $ ls -d p[lu]*
play  public_html


Quoting stops the shell from expanding that, but doesn't affect what
fossil does with it. I think there's an argument for allowing wildcard
characters in filenames, but it would inevitably cause other kinds of
confusion for users and scripts, so I don't really know which way it
should be.

Regards,

Eric

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] filename contains illegal characters

2010-06-28 Thread Michael Richter
On 28 June 2010 21:20, Kevin Greiner  wrote:

> For a few files I see the following error:
>fossil: filename contains illegal characters: prep_20100113[1] clean/
> 135816_0001.ps
>
> I understand it's the square brackets that are causing this error but not
> why this is by design. Could someone explain the rationale?
>
> Also, is there a way to get these files into fossil without renaming them?
>

Well, looking in file.c for the function file_is_simple_pathname (failure of
which causes that error in add.c), I see:

/*
** Return true if the filename given is a valid filename for
** a file in a repository.  Valid filenames follow all of the
** following rules:
**
** *  Does not begin with "/"
** *  Does not contain any path element named "." or ".."
** *  Does not contain any of these characters in the path: "\*[]?"
** *  Does not end with "/".
** *  Does not contain two or more "/" characters in a row.
** *  Contains at least one character
*/

So it's definitely being done by design.  Now the * and ? ban I can see.
 Files with that in the name would cause confusion on almost every platform.
 Some quick tests, though, see that files like "test[1].txt" are permitted
under both Linux and Windows so I can't quite understand the justification
there.  (Under Windows a file with [] in the name has to be put in quotes,
mind, but so do files with spaces.)

Tragically, fossil won't let me add "test[1].txt" even if I do put it in
quotes, although it will allow me to add "test 1.txt".  So it looks to me
like right now you just can't add those files until (if?) this gets changed.

-- 
"Perhaps people don't believe this, but throughout all of the discussions of
entering China our focus has really been what's best for the Chinese people.
It's not been about our revenue or profit or whatnot."
--Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users