Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-15 Thread Tomoaki AOKI 
On Tue, 12 Sep 2017 14:46:12 -0230
"Jonathan Anderson"  wrote:

> On 12 Sep 2017, at 14:38, Julian Elischer wrote:
> 
> > “〓是一个〓〓多字符文件名〓度的文件目的是命名一个文件用中文或者日文或者〓文字符并且要求字符〓度超〓八十五个字符然后拷〓〓文件到我〓的共享文件〓看看是否能〓拷〓〓文件到我.txt”
> > (I have no idea what that says but apparently it's a real filename 
> > from a windows machine that blew up when written via samba.)
> 
> Google Translate says, amusingly:
> "This is a test file for the length of the file name. The purpose is to 
> name a file in Chinese or Japanese or Korean characters and require the 
> character to be longer than 85 characters and then copy the file into 
> our shared folder to see if it can copy the file To me" (.txt, I guess)
> 
> No matter what number you choose for a path length, you're never going 
> to win against that specific user. :)

Most people who doesn't know the "internals" would not matter how
long "a character" is. So we'd better assuming the longest-possible
character.

At worst, file name string can contain shift-in / shift-out or
charset change sequences, and become much longer (character sets
specific). But if we decide the "filename standard" as UTF-8, it
wouldn't be needed and 4 (6?) bytes/character would be sufficient.

So if we decide max length on "UTF-8 characters" to be 256, 1536
"bytes" would be sufficient.

*Possibly 5 and 6 bytes character in UTF-8 could be already prohibited.
 If so, 1024 bytes is sufficient.

> 
> 
> Jon
> --
> Jonathan Anderson
> jonat...@freebsd.org
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> 
> 


-- 
Tomoaki AOKI
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Julian Elischer 

On 13/9/17 1:16 am, Jonathan Anderson wrote:

On 12 Sep 2017, at 14:38, Julian Elischer wrote:

“这是一个测试多字符文件名长度的文件目的是命名一个文件用中文或者日文或者韩文字符并且要求字符长度超过八十五个字符然后拷贝该文件到我们的共享文件夹看看是否能够拷贝该文件到我.txt” 

(I have no idea what that says but apparently it's a real filename 
from a windows machine that blew up when written via samba.)


Google Translate says, amusingly:
"This is a test file for the length of the file name. The purpose is 
to name a file in Chinese or Japanese or Korean characters and 
require the character to be longer than 85 characters and then copy 
the file into our shared folder to see if it can copy the file To 
me" (.txt, I guess)


No matter what number you choose for a path length, you're never 
going to win against that specific user. :)

ha!
it was give as an example to show us the limit.
Apparently however our company has difficulty selling in China due to 
this limit because Microsoft has a much larger limit and people use it.





Jon
--
Jonathan Anderson
jonat...@freebsd.org




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Conrad Meyer 
On Tue, Sep 12, 2017 at 2:23 PM, Rainer Duffner  wrote:
> And there’s also this bug:
>
>
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215067
>
> "g_dev_taste: make_dev_p() failed on importing pool with snapshots with long
> names“
>
>
>
> But maybe that has nothing to do with it.

Hi Rainer,

Yes, I think that one is mostly orthogonal to the filesystem path
lengths.  Still, it might also be nice to expand that one while we're
there.

ino64 was nice enough to bump MNAMELEN from 88 to 1024 and d_namlen to
16 bits, so we don't have to expand those.

Best,
Conrad
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Conrad Meyer 
On Tue, Sep 12, 2017 at 2:11 PM, Ben RUBSON  wrote:
> Hi Conrad,
>
> This patch makes me think about another related bug #184340 :
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184340
> It is about PATH_MAX which in some cases can be too small.
>
> Not sure if it's the case / and how to do it,
> but perhaps it is time to raise some other limits /
> think about a global solution regarding these length limits ?
>
> Many thanks !

Hi Ben,

Yes, I think that bug is about basically the same issue under discussion here.

Best,
Conrad
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Ben RUBSON 
On 12/9/17 2:17 pm, Conrad Meyer wrote:
> On Sat, Sep 9, 2017 at 9:09 AM, Julian Elischer  wrote:
>> maybe we could get it into -current.
>> It'd be silly to have to have people re-inventing hte wheel all the time.
>> How about you put those changes into the reviews.freebsd.org and we can get
>> some general consensus on them.
>> We'll have to do similar for the Asian customers and anyone who uses UTF-8.
>> So it
>> would be silly to have to develop it all again (but subtly different of
>> course).
>> 
>> The key issue is how many system calls and other APIs would be broken,
>> and how many would be broken in a non backwards compatible way?
>> 
>> We would need it in a stable/10 and 11 branch but if the patch is isolated
>> enough we could carry it forward until we get to 12.
>> 
>> One has to allow people to do whatever they are used to with Windows.
>> And in this case the issue is serving files over samba to windows machines.
> Hey Julian,
> 
> I've thrown the patch up at https://reviews.freebsd.org/D12330 .  I
> haven't actually tested it on FreeBSD, but it does compile.  We also
> have some patches against contrib/pjdfstest to fix those tests against
> long file names, but I think we can hold off on those changes until
> we've nailed down what the architectural change will be (if any).

Hi Conrad,

This patch makes me think about another related bug #184340 :
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184340
It is about PATH_MAX which in some cases can be too small.

Not sure if it's the case / and how to do it,
but perhaps it is time to raise some other limits /
think about a global solution regarding these length limits ?

Many thanks !

Ben

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Rainer Duffner 

> Am 12.09.2017 um 23:11 schrieb Ben RUBSON :
> 
> On 12/9/17 2:17 pm, Conrad Meyer wrote:
>> On Sat, Sep 9, 2017 at 9:09 AM, Julian Elischer  wrote:
>>> maybe we could get it into -current.
>>> It'd be silly to have to have people re-inventing hte wheel all the time.
>>> How about you put those changes into the reviews.freebsd.org and we can get
>>> some general consensus on them.
>>> We'll have to do similar for the Asian customers and anyone who uses UTF-8.
>>> So it
>>> would be silly to have to develop it all again (but subtly different of
>>> course).
>>> 
>>> The key issue is how many system calls and other APIs would be broken,
>>> and how many would be broken in a non backwards compatible way?
>>> 
>>> We would need it in a stable/10 and 11 branch but if the patch is isolated
>>> enough we could carry it forward until we get to 12.
>>> 
>>> One has to allow people to do whatever they are used to with Windows.
>>> And in this case the issue is serving files over samba to windows machines.
>> Hey Julian,
>> 
>> I've thrown the patch up at https://reviews.freebsd.org/D12330 .  I
>> haven't actually tested it on FreeBSD, but it does compile.  We also
>> have some patches against contrib/pjdfstest to fix those tests against
>> long file names, but I think we can hold off on those changes until
>> we've nailed down what the architectural change will be (if any).
> 
> Hi Conrad,
> 
> This patch makes me think about another related bug #184340 :
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184340 
> 
> It is about PATH_MAX which in some cases can be too small.
> 
> Not sure if it's the case / and how to do it,
> but perhaps it is time to raise some other limits /
> think about a global solution regarding these length limits ?
> 
> Many thanks !
> 
> Ben



And there’s also this bug:


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215067 


"g_dev_taste: make_dev_p() failed on importing pool with snapshots with long 
names“



But maybe that has nothing to do with it.






___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Jonathan Anderson 

On 12 Sep 2017, at 14:38, Julian Elischer wrote:


“这是一个测试多字符文件名长度的文件目的是命名一个文件用中文或者日文或者韩文字符并且要求字符长度超过八十五个字符然后拷贝该文件到我们的共享文件夹看看是否能够拷贝该文件到我.txt”
(I have no idea what that says but apparently it's a real filename 
from a windows machine that blew up when written via samba.)


Google Translate says, amusingly:
"This is a test file for the length of the file name. The purpose is to 
name a file in Chinese or Japanese or Korean characters and require the 
character to be longer than 85 characters and then copy the file into 
our shared folder to see if it can copy the file To me" (.txt, I guess)


No matter what number you choose for a path length, you're never going 
to win against that specific user. :)



Jon
--
Jonathan Anderson
jonat...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Julian Elischer 

On 12/9/17 2:17 pm, Conrad Meyer wrote:

On Sat, Sep 9, 2017 at 9:09 AM, Julian Elischer  wrote:

maybe we could get it into -current.
It'd be silly to have to have people re-inventing hte wheel all the time.
How about you put those changes into the reviews.freebsd.org and we can get
some general consensus on them.
We'll have to do similar for the Asian customers and anyone who uses UTF-8.
So it
would be silly to have to develop it all again (but subtly different of
course).

The key issue is how many system calls and other APIs would be broken,
and how many would be broken in a non backwards compatible way?

We would need it in a stable/10 and 11 branch but if the patch is isolated
enough we could carry it forward until we get to 12.

One has to allow people to do whatever they are used to with Windows.
And in this case the issue is serving files over samba to windows machines.

Hey Julian,

I've thrown the patch up at https://reviews.freebsd.org/D12330 .  I
haven't actually tested it on FreeBSD, but it does compile.  We also
have some patches against contrib/pjdfstest to fix those tests against
long file names, but I think we can hold off on those changes until
we've nailed down what the architectural change will be (if any).


thanks!

that looks a lot like a proof -of concept patch we derived a while 
back but never really tested.
The issue for us is that using UTF-8 the filenames become too short 
for common usage in China and Japan.

Apparently they routinely nae files with the contents of a small novella.

e.g.

“这是一个测试多字符文件名长度的文件目的是命名一个文件用中文或者日文或者韩文字符并且要求字符长度超过八十五个字符然后拷贝该文件到我们的共享文件夹看看是否能够拷贝该文件到我.txt”
(I have no idea what that says but apparently it's a real filename from a 
windows machine that blew up when written via samba.)

Does anyone else have any thoughts about whether FreeBSD 12 might grow longer 
path/filename support?
 (I'm told Linux uses 1K and 4096 for filenames and path length.)

Julian



It's quite possible this accidentally breaks even more APIs than
expected and we should do some fine tuning to reduce the damage.  Our
$WORK product mostly doesn't care about ABI, so we may not have
noticed some ABI breakage.

If anyone else is interested, please subscribe or add yourself as a
reviewer on the phabricator revision.

Best,
Conrad



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"