Re: problem with ls, not show a correct list
On 09.04.2017 19:31, Nilton José Rizzo wrote: >try this > > ls [a-b,k-m] > >in sh using the locale setting to pt_BR.UTF-8 > > it's not work For the set below I got on -current a A b k K l m with any non-C locale which is right. > # ls > a b d f j m p t x > A B D g k M q u y > aa c e h K n r v z > Aa C E i l o s w Z > It's not show the correct list, this commant MUST BE SHOW like a C > locale Must not, according to CLDR linguists. >Note, I know that Unicode have some differents, but when tha basic > list replacement > and other basic default behavior must be preserved. No, Unicode is the same here, all sorting done as described by CLDR rules. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: problem with ls, not show a correct list
On 07.04.2017 23:20, Nilton José Rizzo wrote: > Em 2017-04-07 05:51, Toomas Soome escreveu: >>> On 7. apr 2017, at 11:29, Andrey Chernov wrote: >>> >>>> Hi Allan, the ls show all files without case match >>>> >>>> ls [a-z]* >>>> >>>> show all files beginning with a and A like this [aA-zZ]* >>> >>> No, last "Z" is not included. > > >Look this, it's a great error I see no error. You do not include Z into [a-z] expression above, so why you expect it is unknown. > I lower- equal a upper-case why Z not show in this list? No, it is not case-equal sorting. As I already mention, it is dictionaries sorting, letters considered first, their case - next (and maybe many other factors for non-ASCII - next). > image when a admin use like thinks some rm -rf /*/[A-Z]* Admin should either run C or US-ASCII locale or do not use a-z ranges without knowing sorting on his locale. I see no errors in your sh examples. In general it was decided (not by me) to use CLDR collation since it is able to stable sort all Unicode chars. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: problem with ls, not show a correct list
On 07.04.2017 11:51, Toomas Soome wrote:
>
>> On 7. apr 2017, at 11:29, Andrey Chernov wrote:
>>
>>> Hi Allan, the ls show all files without case match
>>>
>>> ls [a-z]*
>>>
>>> show all files beginning with a and A like this [aA-zZ]*
>>
>> No, last "Z" is not included.
>>
>
> This is to define set of chars: { a, A-z, Z } ? A-z of course does not make
> any sense;) Of course note that in few locales z is sorted after s, meaning
> that list like that can be rather short;)
No, [A-z] makes perfect sense with CLDR collation we have, but maybe
unexpected effect.
Historically all latin letters have single case, so new times
upper/lower considered by CLDR as minor modification to the letter which
considered first. It is also the sorting used in dictionaries.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: problem with ls, not show a correct list
> Hi Allan, the ls show all files without case match > > ls [a-z]* > > show all files beginning with a and A like this [aA-zZ]* No, last "Z" is not included. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 30.03.2017 21:53, Bruce Evans wrote: > I think it was the sizing. The non-updated mode is 80x25, so the row > address can be out of bounds in the teken layer. I have text 80x30 mode set at rc stage, and _after_ that may have many kernel messages on console, all without causing reboot. How it is different from shutdown stage? Syscons mode is unchanged since rc stage. > - sysctl debug.kdb.break_to_debugger. This is documented in ddb(4), but > only as equivalent to the unbroken BREAK_TO_DEBUGGER. Thanx. Setting debug.kdb.break_to_debugger=1 makes both Ctrl-Alt-ESC and Ctrl-PrtScr works in sc only mode and "c" exit don't cause all chars beeps like in vt. I.e. it works. But I don't understand why debugging via serial involved in sc case while not involved in vt case and fear that some serial noise may provoke break. Is there a chance to untie serial and sc console debuggers? ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 30.03.2017 18:13, Bruce Evans wrote: > On Thu, 30 Mar 2017, Andrey Chernov wrote: > >> On 30.03.2017 12:34, Andrey Chernov wrote: >>> On 30.03.2017 12:23, Andrey Chernov wrote: >>>> Yes, only for reboot/shutdown. The system does not do anythings wrong >>>> even under high load. On reboot or hang those lines are never printed: >>>> >>>> kernel: Waiting (max 60 seconds) for system process `vnlru' to >>>> stop...done >>>> kernel: Waiting (max 60 seconds) for system process `bufdaemon' to >>>> stop...done >>>> kernel: Waiting (max 60 seconds) for system process `syncer' to stop... >>>> kernel: Syncing disks, vnodes remaining...5 3 0 1 0 0 done >>>> kernel: All buffers synced. >>>> (it is from 10-stable sample, old -current samples are lost) >>>> >>>> Moreover, GELI swap deactivation lines are never printed too (I already >>>> mention that I change swap to normal, but nothing is changed). >>> >>> I start to have raw guess that _any_ kernel printf in shutdown mode >>> cause not printf but premature reboot. >> >> Finally I have good news and bad news with today's -current: >> >> 1) It seems your latest commit r316136 fix premature reboot issue. > > Now I need to know how that helped. Do you used a non-default mode? Perhaps it isn't really helped, but just hide the problem, changing some another race time parameters. I use 80x30 text mode on all screens. >> 2) I still can't enter KDB using Ctrl-Alt-ESC, while booting, after >> booting, after login and while shutdown - nothing happens. >> boot -d enters KDB normally, but the keyboard sequence handler is >> broken, not boot -d. > > Try "~b". What? It just prints \n, new csh prompt and ~b > It is an old bug that Ctrl-Alt-ESC (and Ctrl-PrtScr) Ctrl-PrtScr does nothing too. > But I think the misconfiguration is the > same for vt. No, Ctrl-Alt-ESC works for vt at every phase of the system lifecycle. I use Russian keymap for syscons, but Ctrl, Alt, ESC of course are not remapped. I surely remember it works for syscons long time ago. Just to not forget it, I use PS/2 keyboard and have no vt* lines in the kernel config. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 30.03.2017 12:34, Andrey Chernov wrote: > On 30.03.2017 12:23, Andrey Chernov wrote: >> Yes, only for reboot/shutdown. The system does not do anythings wrong >> even under high load. On reboot or hang those lines are never printed: >> >> kernel: Waiting (max 60 seconds) for system process `vnlru' to stop...done >> kernel: Waiting (max 60 seconds) for system process `bufdaemon' to >> stop...done >> kernel: Waiting (max 60 seconds) for system process `syncer' to stop... >> kernel: Syncing disks, vnodes remaining...5 3 0 1 0 0 done >> kernel: All buffers synced. >> (it is from 10-stable sample, old -current samples are lost) >> >> Moreover, GELI swap deactivation lines are never printed too (I already >> mention that I change swap to normal, but nothing is changed). > > I start to have raw guess that _any_ kernel printf in shutdown mode > cause not printf but premature reboot. Finally I have good news and bad news with today's -current: 1) It seems your latest commit r316136 fix premature reboot issue. 2) I still can't enter KDB using Ctrl-Alt-ESC, while booting, after booting, after login and while shutdown - nothing happens. boot -d enters KDB normally, but the keyboard sequence handler is broken, not boot -d. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 30.03.2017 14:23, Andriy Gapon wrote: > On 30/03/2017 12:34, Andrey Chernov wrote: >> On 30.03.2017 12:23, Andrey Chernov wrote: >>> Yes, only for reboot/shutdown. The system does not do anythings wrong >>> even under high load. On reboot or hang those lines are never printed: >>> >>> kernel: Waiting (max 60 seconds) for system process `vnlru' to stop...done >>> kernel: Waiting (max 60 seconds) for system process `bufdaemon' to >>> stop...done >>> kernel: Waiting (max 60 seconds) for system process `syncer' to stop... >>> kernel: Syncing disks, vnodes remaining...5 3 0 1 0 0 done >>> kernel: All buffers synced. >>> (it is from 10-stable sample, old -current samples are lost) >>> >>> Moreover, GELI swap deactivation lines are never printed too (I already >>> mention that I change swap to normal, but nothing is changed). >> >> I start to have raw guess that _any_ kernel printf in shutdown mode >> cause not printf but premature reboot. > > This sounds somewhat familiar... > I vaguely recall an opposite issue that happened in the past. After one of my > changes the reboot started hanging for one user. Turned out that the actual > bug > was always there, but previously the system rebooted because of a printf that > caused a LOR (between spinlocks, AFAIR), witness tried to report it... using > printf, and that recursed and there was a triple fault in the end. > > Let me try to dig some details, maybe the current issue is related in some > ways. > > By chance, do you have WITNESS but not WITNESS_SKIPSPIN in your kernel config? No, I don't have WITNESS* I think removing all vt* lines from the kernel confing (and leaving sc) will be enough to reproduce it, but I am not sure. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 30.03.2017 12:23, Andrey Chernov wrote: > Yes, only for reboot/shutdown. The system does not do anythings wrong > even under high load. On reboot or hang those lines are never printed: > > kernel: Waiting (max 60 seconds) for system process `vnlru' to stop...done > kernel: Waiting (max 60 seconds) for system process `bufdaemon' to > stop...done > kernel: Waiting (max 60 seconds) for system process `syncer' to stop... > kernel: Syncing disks, vnodes remaining...5 3 0 1 0 0 done > kernel: All buffers synced. > (it is from 10-stable sample, old -current samples are lost) > > Moreover, GELI swap deactivation lines are never printed too (I already > mention that I change swap to normal, but nothing is changed). I start to have raw guess that _any_ kernel printf in shutdown mode cause not printf but premature reboot. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
> We don't understand the bug yet. It might not even be in sc. Do you only > see problems for shutdown? The shutdown environment is special for > locking. Yes, only for reboot/shutdown. The system does not do anythings wrong even under high load. On reboot or hang those lines are never printed: kernel: Waiting (max 60 seconds) for system process `vnlru' to stop...done kernel: Waiting (max 60 seconds) for system process `bufdaemon' to stop...done kernel: Waiting (max 60 seconds) for system process `syncer' to stop... kernel: Syncing disks, vnodes remaining...5 3 0 1 0 0 done kernel: All buffers synced. (it is from 10-stable sample, old -current samples are lost) Moreover, GELI swap deactivation lines are never printed too (I already mention that I change swap to normal, but nothing is changed). > A hang in sc means that deadlock occurred and sc's new deadlock detection > didn't work. Hangs are rare. Most common are premature reboots. > Check that ddb works before shutdown, or just put a lot of printfs in I can't check it ddb because I can't enter ddb in sc mode, as I already write, nothing happens. Only vt mode allows Ctrl-Alt-ESC, but the bug does not exist in vt mode, so it is pointless. >>> You might have entered ddb in a context which used to race or deadlock. >> >> No. I try about 20 times on machine which does nothing and can't enter >> KDB in sc only mode, but got one dead hang instead, when start to repeat >> it too fast. > > Even earlier than shutdown, and when booting? I mean in normal operation mode after booting, earlier than shutdown. Shutdown with premature reboot is too fast to press anything at the right time. I don't try to enter ddb when booting yet, but tell you results later. > I call this line-drawing characters for cp437, and use them occasionally, > but I don't know the termcap method for using them very well. See ac, as, ae. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 30.03.2017 9:51, Andrey Chernov wrote: > On 30.03.2017 8:53, Bruce Evans wrote: >>> Maybe two will be enough too, I don't check. I just don't need _any_ of >>> vt lines. What is matter it is that syscons only mode (without any vt) >>> was recently broken, causing shutdown problems and file system damage >>> each time. Syscons only mode works for years until you break it recently. >> >> Actually, I fixed it not so recently (over the last few months), partly >> with much older local fixes. > > Please commit your fix as soon as possible. vt is broken as designed in > many aspects (I even mention not all of them), but from other hand I > can't allow dirty filesystem (or hang) on each reboot using sc only mode > as always. It is dangerous, and fsck takes big time. Moreover, using sc > while keeping vt bloat compiled in the kernel just as the bug workaround > is the best demotivator for perfectionist. > >> The escape sequences in dmesg are very interesting. You should debug >> those. > > I'll send you them a bit later. Since I don't want vt at all, I don't > want to debug or fix it, let it die. Here it is: kernel: allscreens_kbd cursor^[[=0A^[[=7F^[[=0G^[[=0H^[[=7Ividcontrol: setting cursor type: Inappropriate ioctl for device It is caused by vidcontrol call which left from previous sc setup. >>> I'll try. thanx. But most dangerous new syscons bug is the first one, >>> damaging file system on each reboot. I try to go to KDB to debug it, but >>> seeing that I can't even enter KDB I understand that all that bugs, >>> including nasty one, are introduced by your syscons changes, it was a >>> hint to add completely unneeded and unused vt to my kernel config file. >> >> It's normal to have a slightly damaged file system after a panic. > > In sc only mode I have no kernel panic, i.e panic with trace on console > or entering KDB. I have silent reboot in the middle or end of shutdown > sequence or rare dead hang on reboot (which absolutely not acceptable > for remote machine). > >> You might have entered ddb in a context which used to race or deadlock. > > No. I try about 20 times on machine which does nothing and can't enter > KDB in sc only mode, but got one dead hang instead, when start to repeat > it too fast. In vt mode I can enter each time, but there are exit > problems I already mention. > I use text mode in sc. > >> Strings for function keys: >> - these are just broken in both sc and vt > > I have all function keys working in sc only mode with TERM=cons25 and > similar ones. > >> Pseudographics: >> - I don't use it enough to see problems in it. Even finding the unicode >> glyph for the block character took me some time. > > Even cp437 have it and dialog library use it for all windows frames, > f.e. all ports config windows use pseudographics if it is available and > working (replaced by +-| etc poor looking ASCII otherwise). ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 30.03.2017 8:53, Bruce Evans wrote: >> Maybe two will be enough too, I don't check. I just don't need _any_ of >> vt lines. What is matter it is that syscons only mode (without any vt) >> was recently broken, causing shutdown problems and file system damage >> each time. Syscons only mode works for years until you break it recently. > > Actually, I fixed it not so recently (over the last few months), partly > with much older local fixes. Please commit your fix as soon as possible. vt is broken as designed in many aspects (I even mention not all of them), but from other hand I can't allow dirty filesystem (or hang) on each reboot using sc only mode as always. It is dangerous, and fsck takes big time. Moreover, using sc while keeping vt bloat compiled in the kernel just as the bug workaround is the best demotivator for perfectionist. > The escape sequences in dmesg are very interesting. You should debug > those. I'll send you them a bit later. Since I don't want vt at all, I don't want to debug or fix it, let it die. >> I'll try. thanx. But most dangerous new syscons bug is the first one, >> damaging file system on each reboot. I try to go to KDB to debug it, but >> seeing that I can't even enter KDB I understand that all that bugs, >> including nasty one, are introduced by your syscons changes, it was a >> hint to add completely unneeded and unused vt to my kernel config file. > > It's normal to have a slightly damaged file system after a panic. In sc only mode I have no kernel panic, i.e panic with trace on console or entering KDB. I have silent reboot in the middle or end of shutdown sequence or rare dead hang on reboot (which absolutely not acceptable for remote machine). > You might have entered ddb in a context which used to race or deadlock. No. I try about 20 times on machine which does nothing and can't enter KDB in sc only mode, but got one dead hang instead, when start to repeat it too fast. In vt mode I can enter each time, but there are exit problems I already mention. I use text mode in sc. > Strings for function keys: > - these are just broken in both sc and vt I have all function keys working in sc only mode with TERM=cons25 and similar ones. > Pseudographics: > - I don't use it enough to see problems in it. Even finding the unicode > glyph for the block character took me some time. Even cp437 have it and dialog library use it for all windows frames, f.e. all ports config windows use pseudographics if it is available and working (replaced by +-| etc poor looking ASCII otherwise). ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 29.03.2017 6:29, Bruce Evans wrote: > Using rc_debug=yes I see that it is the kernel problem, not rc > problem. > Sometimes rc backward sequence executed even fully, sometimes only > partly, but in unpredictable moment inside rc sequence the kernel > decide > to reboot quickly (or even deadly hang in rare cases). Always without > any "Syncing buffers..." leaving FS dirty. No zfs etc. just normal > UFS, > no EFI, no GPT. > I change GELI swap to normal one, but it does not help. The same > untouched config works for years, I see this bug for the first time in > FreeBSD. I forget to mention that typescript and dmesg does not survive after this reboot (or rare hang). >>> >>> Good to note. >>> The simple explanation to the problem might be r307755, depending on >>> when you last synced/built ^/head. >>> >>> I have a few more questions (if reverting that doesn't pan out): >> >> I just found the cause, it is new syscons bug (bde@ cc'ed). I never >> compile vt driver into kernel, i.e. I don't have this lines in the >> kernel config: >> >> devicevt >> devicevt_vga >> devicevt_efifb >> >> When I add them, the bug described is gone. It seems syscons goes off to >> early, provoking reboot. > > Bah, I only have vt and vt_vga to check that I didn't break them. > > Unfortunately, syscons still works right when I remove these lines. Maybe two will be enough too, I don't check. I just don't need _any_ of vt lines. What is matter it is that syscons only mode (without any vt) was recently broken, causing shutdown problems and file system damage each time. Syscons only mode works for years until you break it recently. > Kernel messages in syscons are now supposed to be colorized by CPU. The It looks really crazy on 8-core CPU and should not be default. And I don't see colors in vt mode (which should be parallel at that point, at least), but what about invisible escapes on vidcontrol errors (f.e. invalid argument) in vt mode? >> Moreover, I can't enter KDB via Ctrl-Alt-ESC in the syscons only mode >> anymore - nothing happens. In the vt mode I can, but can't exit via "c" >> properly, all chars typed after "c" produce beep unless I switch to >> another screen and back. > > Try backing out r315984 only. This is supposed to fix parsing of output. I'll try. thanx. But most dangerous new syscons bug is the first one, damaging file system on each reboot. I try to go to KDB to debug it, but seeing that I can't even enter KDB I understand that all that bugs, including nasty one, are introduced by your syscons changes, it was a hint to add completely unneeded and unused vt to my kernel config file. vt is real downgrade. Its default console font is plain ugly, it is impossible to work with it. I can't find proper TERM for it to make function keys and pseudographics works in ncurses apps (not with xterm, a little better with xterm-sco), lynx can't display all things properly, etc. All we need is KMS integration alone and not vt. > But I suspect it is a usb keyboard problem. No, I have PS/2 keyboard. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
New syscons bugs: shutdown -r doesn't execute rc.d sequence and others
On 29.03.2017 0:46, Ngie Cooper (yaneurabeya) wrote: > >> On Mar 28, 2017, at 14:27, Andrey Chernov wrote: > > … > >>> Using rc_debug=yes I see that it is the kernel problem, not rc problem. >>> Sometimes rc backward sequence executed even fully, sometimes only >>> partly, but in unpredictable moment inside rc sequence the kernel decide >>> to reboot quickly (or even deadly hang in rare cases). Always without >>> any "Syncing buffers..." leaving FS dirty. No zfs etc. just normal UFS, >>> no EFI, no GPT. >>> I change GELI swap to normal one, but it does not help. The same >>> untouched config works for years, I see this bug for the first time in >>> FreeBSD. >>> >> >> I forget to mention that typescript and dmesg does not survive after >> this reboot (or rare hang). > > Good to note. > The simple explanation to the problem might be r307755, depending on when you > last synced/built ^/head. > > I have a few more questions (if reverting that doesn't pan out): I just found the cause, it is new syscons bug (bde@ cc'ed). I never compile vt driver into kernel, i.e. I don't have this lines in the kernel config: device vt device vt_vga device vt_efifb When I add them, the bug described is gone. It seems syscons goes off to early, provoking reboot. I also find some lines of the kernel messages strange colored instead of white in the syscons only mode. Even in vt mode vidcontrol errors have invisible escapes prepended (although visible through /var/log/messages). Moreover, I can't enter KDB via Ctrl-Alt-ESC in the syscons only mode anymore - nothing happens. In the vt mode I can, but can't exit via "c" properly, all chars typed after "c" produce beep unless I switch to another screen and back. All it means that syscons becomes very broken now by itself and even damages the kernel operations. signature.asc Description: OpenPGP digital signature
Re: shutdown -r doesn't execute rc.d sequence
On 29.03.2017 0:15, Andrey Chernov wrote: > On 28.03.2017 22:33, Ngie Cooper (yaneurabeya) wrote: >> >>> On Mar 28, 2017, at 12:30, Andrey Chernov wrote: >>> >>> With latest -current amd64, reboot happens almost immediately, leaving >>> FS dirty. No proper backward rc.d or /usr/local/etc/rc.d sequence >>> execution is shown. No deactivating GELI swap too. >> >> Hi Andrey, >> Do you have a typescript demonstrating this? Adding rc_debug=yes to >> /etc/rc.conf would be super helpful, along with `boot -v`, to see whether >> the issue is in userspace or rc(5). >> I’ll double check my amd64/i386 VMs too (if possible, redirect the >> output over serial to a typescript for analysis). >> Are you using vanilla FreeBSD, a fork, or a packaged variant (mfsbsd, >> nanobsd, etc)? >> Thanks! >> -Ngie >> > > Using rc_debug=yes I see that it is the kernel problem, not rc problem. > Sometimes rc backward sequence executed even fully, sometimes only > partly, but in unpredictable moment inside rc sequence the kernel decide > to reboot quickly (or even deadly hang in rare cases). Always without > any "Syncing buffers..." leaving FS dirty. No zfs etc. just normal UFS, > no EFI, no GPT. > I change GELI swap to normal one, but it does not help. The same > untouched config works for years, I see this bug for the first time in > FreeBSD. > I forget to mention that typescript and dmesg does not survive after this reboot (or rare hang). signature.asc Description: OpenPGP digital signature
Re: shutdown -r doesn't execute rc.d sequence
On 28.03.2017 22:33, Ngie Cooper (yaneurabeya) wrote: > >> On Mar 28, 2017, at 12:30, Andrey Chernov wrote: >> >> With latest -current amd64, reboot happens almost immediately, leaving >> FS dirty. No proper backward rc.d or /usr/local/etc/rc.d sequence >> execution is shown. No deactivating GELI swap too. > > Hi Andrey, > Do you have a typescript demonstrating this? Adding rc_debug=yes to > /etc/rc.conf would be super helpful, along with `boot -v`, to see whether the > issue is in userspace or rc(5). > I’ll double check my amd64/i386 VMs too (if possible, redirect the > output over serial to a typescript for analysis). > Are you using vanilla FreeBSD, a fork, or a packaged variant (mfsbsd, > nanobsd, etc)? > Thanks! > -Ngie > Using rc_debug=yes I see that it is the kernel problem, not rc problem. Sometimes rc backward sequence executed even fully, sometimes only partly, but in unpredictable moment inside rc sequence the kernel decide to reboot quickly (or even deadly hang in rare cases). Always without any "Syncing buffers..." leaving FS dirty. No zfs etc. just normal UFS, no EFI, no GPT. I change GELI swap to normal one, but it does not help. The same untouched config works for years, I see this bug for the first time in FreeBSD. signature.asc Description: OpenPGP digital signature
Re: shutdown -r doesn't execute rc.d sequence
On 28.03.2017 22:33, Ngie Cooper (yaneurabeya) wrote: > >> On Mar 28, 2017, at 12:30, Andrey Chernov wrote: >> >> With latest -current amd64, reboot happens almost immediately, leaving >> FS dirty. No proper backward rc.d or /usr/local/etc/rc.d sequence >> execution is shown. No deactivating GELI swap too. > > Hi Andrey, > Do you have a typescript demonstrating this? Adding rc_debug=yes to > /etc/rc.conf would be super helpful, along with `boot -v`, to see whether the > issue is in userspace or rc(5). > I’ll double check my amd64/i386 VMs too (if possible, redirect the > output over serial to a typescript for analysis). > Are you using vanilla FreeBSD, a fork, or a packaged variant (mfsbsd, > nanobsd, etc)? > Thanks! > -Ngie I don't have serial, so typescript may not work treating as dirty, but I'll try. As I say, it is today's -current, vanilla. It looks like regression because few weeks ago all things works. signature.asc Description: OpenPGP digital signature
shutdown -r doesn't execute rc.d sequence
With latest -current amd64, reboot happens almost immediately, leaving FS dirty. No proper backward rc.d or /usr/local/etc/rc.d sequence execution is shown. No deactivating GELI swap too. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Can't build -current incrementally due to libnv changes
Yes, 'make includes' (and 'make obj' and 'make depend') is done before 'make all' in /usr/src/lib ===> libnv (all) cc -O2 -pipe -march=sandybridge -I/usr/src/lib/libnv/../../sys -I/usr/src/lib/libnv -MD -MF.depend.cnvlist.o -MTcnvlist.o -std=gnu99 -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c /usr/src/lib/libnv/../../sys/contrib/libnv/cnvlist.c -o cnvlist.o /usr/src/lib/libnv/../../sys/contrib/libnv/cnvlist.c:49:10: fatal error: 'sys/cnv.h' file not found #include ^ 1 error generated. *** Error code 1 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: date(1) default format changed between 10.3 and 11.0-BETA3
On 05.08.2016 18:44, Mark Martinec wrote: > On 2016-08-05 17:23, Andrey Chernov wrote: >> On 05.08.2016 17:47, Mark Martinec wrote: >>> [Bug 211598] >>> date(1) default format in en_EN locale breaks compatibility with 10.3 >>> and violates POSIX >>> >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211598 >> >> It breaks compatibility but not violates POSIX. POSIX care of only its >> own POSIX (or C) locale. > > POSIX does say that the default format should be the same > as with "+%a %b %e %H:%M:%S %Z %Y". > It also says that %a and %b are locale's abbreviated names. It is true for _POSIX_ locale only, as I already say. en_US.* is not POSIX or C locale. > It may not say what an abbreviated name really looks like > in a particular locale, but it does distinguish between > full and abbreviated names. It says nothing about other locales. Current format comes from CLDR, ask bapt@ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: date(1) default format changed between 10.3 and 11.0-BETA3
On 05.08.2016 17:47, Mark Martinec wrote: > [Bug 211598] > date(1) default format in en_EN locale breaks compatibility with 10.3 > and violates POSIX > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211598 It breaks compatibility but not violates POSIX. POSIX care of only its own POSIX (or C) locale. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: svn commit: r302601 - in head/sys: arm/include arm64/include [clang 3.8.0: powerpc has odd mix of signed wchar_t and unsigned char]
On 13.07.2016 11:53, Mark Millard wrote: > [The below does note that TARGET=powerpc has a mix of signed wchar_t and > unsigned char types and most architectures have both being signed types.] POSIX says nothing about wchar_t and char should be the same (un)signed. It is arm ABI docs may say so only. They are different entities differently encoded and cross assigning between wchar_t and char is not recommended. > > On 2016-Jul-11, at 8:57 PM, Andrey Chernov wrote: > >> On 12.07.2016 5:44, Mark Millard wrote: >>> My understanding of the criteria for __WCHAR_MIN and __WCHAR_MAX: >>> >>> A) __WCHAR_MIN and __WCHAR_MAX: same type as the integer promotion of >>> ___wchar_t (if that is distinct). >>> B) __WCHAR_MIN is the low value for ___wchar_t as an integer type; not >>> necessarily a valid char value >>> C) __WCHAR_MAX is the high value for ___wchar_t as an integer type; not >>> necessarily a valid char value >> >> It seems you are right about "not a valid char value", I'll back this >> change out. >> >>> As far as I know arm FreeBSD uses unsigned character types (of whatever >>> width). >> >> Probably it should be unsigned for other architectures too, clang does >> not generate negative values with L'' literals and locale use only >> positive values too. > > Looking around: > > # grep -i wchar sys/*/include/_types.h > sys/arm/include/_types.h:typedef unsigned int___wchar_t; > sys/arm/include/_types.h:#define __WCHAR_MIN 0 /* min > value for a wchar_t */ > sys/arm/include/_types.h:#define __WCHAR_MAX __UINT_MAX /* max > value for a wchar_t */ > sys/arm64/include/_types.h:typedefunsigned int___wchar_t; > sys/arm64/include/_types.h:#define__WCHAR_MIN 0 /* min > value for a wchar_t */ > sys/arm64/include/_types.h:#define__WCHAR_MAX __UINT_MAX /* max > value for a wchar_t */ > sys/mips/include/_types.h:typedef int ___wchar_t; > sys/mips/include/_types.h:#define __WCHAR_MIN __INT_MIN /* min > value for a wchar_t */ > sys/mips/include/_types.h:#define __WCHAR_MAX __INT_MAX /* max > value for a wchar_t */ > sys/powerpc/include/_types.h:typedef int ___wchar_t; > sys/powerpc/include/_types.h:#define __WCHAR_MIN __INT_MIN /* min > value for a wchar_t */ > sys/powerpc/include/_types.h:#define __WCHAR_MAX __INT_MAX /* max > value for a wchar_t */ > sys/riscv/include/_types.h:typedefint ___wchar_t; > sys/riscv/include/_types.h:#define__WCHAR_MIN __INT_MIN /* min > value for a wchar_t */ > sys/riscv/include/_types.h:#define__WCHAR_MAX __INT_MAX /* max > value for a wchar_t */ > sys/sparc64/include/_types.h:typedef int ___wchar_t; > sys/sparc64/include/_types.h:#define __WCHAR_MIN __INT_MIN /* min > value for a wchar_t */ > sys/sparc64/include/_types.h:#define __WCHAR_MAX __INT_MAX /* max > value for a wchar_t */ > sys/x86/include/_types.h:typedef int ___wchar_t; > sys/x86/include/_types.h:#define __WCHAR_MIN __INT_MIN /* min > value for a wchar_t */ > sys/x86/include/_types.h:#define __WCHAR_MAX __INT_MAX /* max > value for a wchar_t */ > > So only arm and arm64 have unsigned wchar_t types. > > [NOTE: __CHAR16_TYPE__ and __CHAR32_TYPE__ are always unsigned: in C++11 > terms char16_t is like std::uint_least16_t and char32_t is like > std::uint_least32_t despite being distinct types. So __CHAR16_TYPE__ and > __CHAR32_TYPE__ are ignored below.] > > The clang 3.8.0 compiler output has an odd mix for TARGET_ARCH=powerpc and > TARGET_ARCH=powerpc64 . . . > > armv6 has unsigned types for both char and __WCHAR_TYPE__. > aarch64 has unsigned types for both char and __WCHAR_TYPE__. > powerpc has unsigned for char but signed for __WCHAR_TYPE__. > powerpc64 has unsigned for char but signed for __WCHAR_TYPE__. > amd64 has signed types for both char and __WCHAR_TYPE__. > i386 has signed types for both char and __WCHAR_TYPE__. > mips has signed types for both char and __WCHAR_TYPE__. > sparc64 has signed types for both char and __WCHAR_TYPE__. > (riscv is not covered by clang as I understand) > > The details via compiler #define's. . . > > # clang --target=armv6-freebsd11 -std=c99 -E -dM - < /dev/null | more > . . . > #define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__ > . . . > #define __CHAR_BIT__ 8 > #define __CHAR_UNSIGNED__ 1 > . . . > #define __WCHAR_MAX__ 4294967295U > #define __WCHAR_TYPE__ unsigned int &g
Re: GOST in OPENSSL_BASE
On 12.07.2016 12:59, Daniel Kalchev wrote: > The standard HTTPS implementation is already sufficiently broken, with the > door wide open by the concept of “multiple CAs”. The protocol design is > flawed, as any CA can issue certificate for any site. Applications are > required to trust that certificates, as long as they trust the CA that issued > them. > > It is trivial to play MTIM with this protocol and in fact, there are > commercially available “solutions” for “securing one’s corporate network” > that doe exactly that. Some believe this is with the knowledge and approval > of the corporation, but who is to say what the black box actually does and > whose interests it serves? > > There is of course an update to the protocol, DANE, that just shuts this door > off. But… it faces heavy resistance, as it’s acceptance would mean the end of > the lucrative CA business and the ability to intercept “secure” HTTPS > communication. Those relying on the HPPTS flaws will never let it become wide > spread. > > In summary — anyone can sniff HTTPS traffic. No need for any cipher backdoors > here. Nor any need for GOST to be involved. You forget to mention that CA must already be in the trusted root list to allow it happens. signature.asc Description: OpenPGP digital signature
Re: GOST in OPENSSL_BASE
On 12.07.2016 12:16, Andrey Chernov wrote:
> On 12.07.2016 8:48, Kevin Oberman wrote:
>> >> May be need file PR for dns/bind910?
>> >>
>> >> # grep -3 BROK /poudriere/ports/default/dns/bind910/Makefile
>> >> .include http://bsd.port.pre.mk>>
>> >>
>> >> .if ( ${PORT_OPTIONS:MGOST} || ${PORT_OPTIONS:MGOST_ASN1} ) &&
>> ${SSL_DEFAULT} == base
>> >> BROKEN= OpenSSL from the base system does not support GOST, add \
>> >> DEFAULT_VERSIONS+=ssl=openssl to your /etc/make.conf and
>> rebuild everything \
>> >> that needs SSL.
>> >> .endif
>> >>
>> >
>> > I dislike idea to use GOST in the bind, it is unneeded there, DNSSEC
>> > don't use GOST, so I vote for removing GOST option from there.
>> >
>>
>> I need to note that RFC exists, proposing GOST (old version) for DNSSEC:
>> https://tools.ietf.org/html/rfc5933
>> but nobody really use it.
>>
>> In case people are not aware of it, Russian law now requires ALL
>> encrypted traffic must either be accessible by the FSB or that the
>> private keys must be available to the FSB.
>
> It is not quite so. All traffic must be available for 6 months and they
> express intention to ask big companies for their private keys, but later
> is not required by the law (not yet...)
>
>> I have always assumed that
>> GOST has a hidden vulnerability/backdoor that the FSB is already using,
>
> I already answer this question elsewhere in this thread with the reference.
>
>> but this makes it mandatory. Putin gave the FSB 2 weeks to implement the
>> law, which is clearly impossible, but I suspect that there will be a
>> huge effort to pick all low-hanging fruit. As a result, I suspect no one
>> outside of Russia will touch GOST. (Not that they do now, either.) I'd
>> hate to see its support required for any protocol except in Russia as
>> someone will be silly enough to use it.
>
> I already explain required GOST usage pattern in this thread.
>
Ah, I see, freebsd-current list was excluded by someone, so I repeat
what I wrote:
Official documents workflow here require using GOST signatures for
authenticity and consistency verification, they are needed or, in some
cases, required for both people and companies. Since it is official in
any case, there is no harm to have FSB backdoor in the algo, unless some
hacker will find it. Just don't use GOST for something else to stay on
safe side.
BTW, latest GOST based on elliptic curves, so from math point of view
probability of having backdoor here is minimal. I don't examine its
implementation.
See
https://ru.wikipedia.org/wiki/%D0%93%D0%9E%D0%A1%D0%A2_%D0%A0_34.10-2012
You can consider GOST goals are the same as FIPS ones with the reason to
have things "domestically produced".
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 12.07.2016 8:48, Kevin Oberman wrote:
> >> May be need file PR for dns/bind910?
> >>
> >> # grep -3 BROK /poudriere/ports/default/dns/bind910/Makefile
> >> .include http://bsd.port.pre.mk>>
> >>
> >> .if ( ${PORT_OPTIONS:MGOST} || ${PORT_OPTIONS:MGOST_ASN1} ) &&
> ${SSL_DEFAULT} == base
> >> BROKEN= OpenSSL from the base system does not support GOST, add \
> >> DEFAULT_VERSIONS+=ssl=openssl to your /etc/make.conf and
> rebuild everything \
> >> that needs SSL.
> >> .endif
> >>
> >
> > I dislike idea to use GOST in the bind, it is unneeded there, DNSSEC
> > don't use GOST, so I vote for removing GOST option from there.
> >
>
> I need to note that RFC exists, proposing GOST (old version) for DNSSEC:
> https://tools.ietf.org/html/rfc5933
> but nobody really use it.
>
> In case people are not aware of it, Russian law now requires ALL
> encrypted traffic must either be accessible by the FSB or that the
> private keys must be available to the FSB.
It is not quite so. All traffic must be available for 6 months and they
express intention to ask big companies for their private keys, but later
is not required by the law (not yet...)
> I have always assumed that
> GOST has a hidden vulnerability/backdoor that the FSB is already using,
I already answer this question elsewhere in this thread with the reference.
> but this makes it mandatory. Putin gave the FSB 2 weeks to implement the
> law, which is clearly impossible, but I suspect that there will be a
> huge effort to pick all low-hanging fruit. As a result, I suspect no one
> outside of Russia will touch GOST. (Not that they do now, either.) I'd
> hate to see its support required for any protocol except in Russia as
> someone will be silly enough to use it.
I already explain required GOST usage pattern in this thread.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: svn commit: r302601 - in head/sys: arm/include arm64/include [__WCHAR_MAX definition mostly]
On 12.07.2016 5:44, Mark Millard wrote: > My understanding of the criteria for __WCHAR_MIN and __WCHAR_MAX: > > A) __WCHAR_MIN and __WCHAR_MAX: same type as the integer promotion of > ___wchar_t (if that is distinct). > B) __WCHAR_MIN is the low value for ___wchar_t as an integer type; not > necessarily a valid char value > C) __WCHAR_MAX is the high value for ___wchar_t as an integer type; not > necessarily a valid char value It seems you are right about "not a valid char value", I'll back this change out. > As far as I know arm FreeBSD uses unsigned character types (of whatever > width). Probably it should be unsigned for other architectures too, clang does not generate negative values with L'' literals and locale use only positive values too. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 12.07.2016 1:44, Andrey Chernov wrote:
> On 11.07.2016 21:41, Slawa Olhovchenkov wrote:
>> On Mon, Jul 11, 2016 at 02:28:45PM -0400, Jung-uk Kim wrote:
>>
>>> On 07/10/16 10:10 AM, Andrey Chernov wrote:
>>>> On 10.07.2016 16:30, Slawa Olhovchenkov wrote:
>>>>> I am surprised lack of support GOST in openssl-base.
>>>>> Can be this enabled before 11.0 released?
>>>>
>>>> AFAIK openssl maintainers says something like they can't support this
>>>> code and it will become rotten shortly with new changes, so they drop it.
>>>
>>> [OpenSSL-maintainer-for-the-base hat on]
>>>
>>> GOST is supported on FreeBSD 10.x and 11.x. We will not drop it on
>>> these branches unless secteam explicitly ask us to do so. However, we
>>> *may* drop it from 12.0 *iff* we import OpenSSL 1.1.0 branch.
>>>
>>> [OpenSSL-maintainer-for-the-base hat off]
>>>
>>> Jung-uk Kim
>>>
>>
>> Thanks!
>>
>> May be need file PR for dns/bind910?
>>
>> # grep -3 BROK /poudriere/ports/default/dns/bind910/Makefile
>> .include
>>
>> .if ( ${PORT_OPTIONS:MGOST} || ${PORT_OPTIONS:MGOST_ASN1} ) &&
>> ${SSL_DEFAULT} == base
>> BROKEN= OpenSSL from the base system does not support GOST, add \
>> DEFAULT_VERSIONS+=ssl=openssl to your /etc/make.conf and rebuild
>> everything \
>> that needs SSL.
>> .endif
>>
>
> I dislike idea to use GOST in the bind, it is unneeded there, DNSSEC
> don't use GOST, so I vote for removing GOST option from there.
>
I need to note that RFC exists, proposing GOST (old version) for DNSSEC:
https://tools.ietf.org/html/rfc5933
but nobody really use it.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 11.07.2016 21:41, Slawa Olhovchenkov wrote:
> On Mon, Jul 11, 2016 at 02:28:45PM -0400, Jung-uk Kim wrote:
>
>> On 07/10/16 10:10 AM, Andrey Chernov wrote:
>>> On 10.07.2016 16:30, Slawa Olhovchenkov wrote:
>>>> I am surprised lack of support GOST in openssl-base.
>>>> Can be this enabled before 11.0 released?
>>>
>>> AFAIK openssl maintainers says something like they can't support this
>>> code and it will become rotten shortly with new changes, so they drop it.
>>
>> [OpenSSL-maintainer-for-the-base hat on]
>>
>> GOST is supported on FreeBSD 10.x and 11.x. We will not drop it on
>> these branches unless secteam explicitly ask us to do so. However, we
>> *may* drop it from 12.0 *iff* we import OpenSSL 1.1.0 branch.
>>
>> [OpenSSL-maintainer-for-the-base hat off]
>>
>> Jung-uk Kim
>>
>
> Thanks!
>
> May be need file PR for dns/bind910?
>
> # grep -3 BROK /poudriere/ports/default/dns/bind910/Makefile
> .include
>
> .if ( ${PORT_OPTIONS:MGOST} || ${PORT_OPTIONS:MGOST_ASN1} ) && ${SSL_DEFAULT}
> == base
> BROKEN= OpenSSL from the base system does not support GOST, add \
> DEFAULT_VERSIONS+=ssl=openssl to your /etc/make.conf and rebuild
> everything \
> that needs SSL.
> .endif
>
I dislike idea to use GOST in the bind, it is unneeded there, DNSSEC
don't use GOST, so I vote for removing GOST option from there.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 11.07.2016 23:13, Slawa Olhovchenkov wrote: > On Mon, Jul 11, 2016 at 07:48:44PM +0300, Andrey Chernov wrote: > >> On 11.07.2016 19:29, Slawa Olhovchenkov wrote: >>> On Mon, Jul 11, 2016 at 11:04:33AM -0500, Mark Felder wrote: >>> >>>> >>>> >>>> On Mon, Jul 11, 2016, at 05:29, Slawa Olhovchenkov wrote: >>>>> >>>>> I.e. GOST will be available in openssl. >>>>> Under BSD-like license. >>>>> Can be this engine import in base system and enabled at time 1.1.0? >>>>> And can be GOST enabled now? >>>>> >>>> >>>> I think the wrong question is being asked here. Instead we need to focus >>>> on decoupling openssl from base so this can all be handled by ports. >>> >>> This is wrong direction with current policy. >>> ports: unsupported by FreeBSD core and securite team, no guaranted to >>> comaptible >>> between options and applications. >>> >>> base: supported by FreeBSD core and securite team, covered by CI, >>> checked for forward and backward API and ABI compatibility. >>> >> >> Ports are supported by secteam, and recently I notice "headsup" mail >> with intention to make base openssl private and switch all ports to >> security/openssl port. > > I mean `support` is commit reviewing, auditing and etc. > Secteam do it for ports? At least CVEs are tracked. You better ask about whole list of ports secteam duties secteam themselves. > >> Adding of GOST as 3rd party plugin is technically possible in both >> (base, ports) cases, the rest of decision is up to FreeBSD openssl >> maintainers and possible contributors efforts. >> >> I need to specially point to "patches" section of the 3rd party GOST >> plugin, from just viewing I don't understand, are those additional >> openssl patches should be applied to openssl for GOST, or they are just >> reflect existent changes in the openssl. >> >> ___ >> freebsd-secur...@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-security >> To unsubscribe, send any mail to "freebsd-security-unsubscr...@freebsd.org" > ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 11.07.2016 19:29, Slawa Olhovchenkov wrote: > On Mon, Jul 11, 2016 at 11:04:33AM -0500, Mark Felder wrote: > >> >> >> On Mon, Jul 11, 2016, at 05:29, Slawa Olhovchenkov wrote: >>> >>> I.e. GOST will be available in openssl. >>> Under BSD-like license. >>> Can be this engine import in base system and enabled at time 1.1.0? >>> And can be GOST enabled now? >>> >> >> I think the wrong question is being asked here. Instead we need to focus >> on decoupling openssl from base so this can all be handled by ports. > > This is wrong direction with current policy. > ports: unsupported by FreeBSD core and securite team, no guaranted to > comaptible > between options and applications. > > base: supported by FreeBSD core and securite team, covered by CI, > checked for forward and backward API and ABI compatibility. > Ports are supported by secteam, and recently I notice "headsup" mail with intention to make base openssl private and switch all ports to security/openssl port. Adding of GOST as 3rd party plugin is technically possible in both (base, ports) cases, the rest of decision is up to FreeBSD openssl maintainers and possible contributors efforts. I need to specially point to "patches" section of the 3rd party GOST plugin, from just viewing I don't understand, are those additional openssl patches should be applied to openssl for GOST, or they are just reflect existent changes in the openssl. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 10.07.2016 18:28, Andrey Chernov wrote: > On 10.07.2016 18:13, Andrey Chernov wrote: >> On 10.07.2016 18:12, Andrey Chernov wrote: >>> On 10.07.2016 18:01, Slawa Olhovchenkov wrote: >>>> On Sun, Jul 10, 2016 at 05:10:04PM +0300, Andrey Chernov wrote: >>>> >>>>> On 10.07.2016 16:30, Slawa Olhovchenkov wrote: >>>>>> I am surprised lack of support GOST in openssl-base. >>>>>> Can be this enabled before 11.0 released? >>>>> >>>>> AFAIK openssl maintainers says something like they can't support this >>>>> code and it will become rotten shortly with new changes, so they drop it. >>>>> >>>> >>>> Upstream or FreeBSD maintainers? >>>> >>> >>> Openssl maintainers. >>> >> I.e. upstream. >> > They mean built-in one, dropped from openssl 1.1.0 and above. It is > still available as 3rd party at: > https://github.com/gost-engine/engine > >From their Changelog: *) The GOST engine was out of date and therefore it has been removed. An up to date GOST engine is now being maintained in an external repository. See: https://wiki.openssl.org/index.php/Binaries. Libssl still retains support for GOST ciphersuites (these are only activated if a GOST engine is present). [Matt Caswell] ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 10.07.2016 18:13, Andrey Chernov wrote: > On 10.07.2016 18:12, Andrey Chernov wrote: >> On 10.07.2016 18:01, Slawa Olhovchenkov wrote: >>> On Sun, Jul 10, 2016 at 05:10:04PM +0300, Andrey Chernov wrote: >>> >>>> On 10.07.2016 16:30, Slawa Olhovchenkov wrote: >>>>> I am surprised lack of support GOST in openssl-base. >>>>> Can be this enabled before 11.0 released? >>>> >>>> AFAIK openssl maintainers says something like they can't support this >>>> code and it will become rotten shortly with new changes, so they drop it. >>>> >>> >>> Upstream or FreeBSD maintainers? >>> >> >> Openssl maintainers. >> > I.e. upstream. > They mean built-in one, dropped from openssl 1.1.0 and above. It is still available as 3rd party at: https://github.com/gost-engine/engine ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 10.07.2016 18:12, Andrey Chernov wrote: > On 10.07.2016 18:01, Slawa Olhovchenkov wrote: >> On Sun, Jul 10, 2016 at 05:10:04PM +0300, Andrey Chernov wrote: >> >>> On 10.07.2016 16:30, Slawa Olhovchenkov wrote: >>>> I am surprised lack of support GOST in openssl-base. >>>> Can be this enabled before 11.0 released? >>> >>> AFAIK openssl maintainers says something like they can't support this >>> code and it will become rotten shortly with new changes, so they drop it. >>> >> >> Upstream or FreeBSD maintainers? >> > > Openssl maintainers. > I.e. upstream. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 10.07.2016 18:01, Slawa Olhovchenkov wrote: > On Sun, Jul 10, 2016 at 05:10:04PM +0300, Andrey Chernov wrote: > >> On 10.07.2016 16:30, Slawa Olhovchenkov wrote: >>> I am surprised lack of support GOST in openssl-base. >>> Can be this enabled before 11.0 released? >> >> AFAIK openssl maintainers says something like they can't support this >> code and it will become rotten shortly with new changes, so they drop it. >> > > Upstream or FreeBSD maintainers? > Openssl maintainers. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GOST in OPENSSL_BASE
On 10.07.2016 16:30, Slawa Olhovchenkov wrote: > I am surprised lack of support GOST in openssl-base. > Can be this enabled before 11.0 released? AFAIK openssl maintainers says something like they can't support this code and it will become rotten shortly with new changes, so they drop it. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Recent -current stable panic at later boot stage
On 10.07.2016 4:30, Matthew Macy wrote: > > > > On Sat, 09 Jul 2016 18:19:57 -0700 Andrey Chernov > wrote > > BTW, it never happens with 11-ALPHA3 even with world build for 12. > > > > On 09.07.2016 22:06, Andrey Chernov wrote: > > > SCHED_ULE used. I have textdump only. Interesting parts: > > > <118>Starting syslogd. > > > kernel trap 9 with interrupts disabled > > > > > > Fatal trap 9: general protection fault while in kernel mode > > > cpuid = 7; apic id = 07 > > > instruction pointer= 0x20:0x806b2585 > > > stack pointer= 0x28:0xfe022ccce8d0 > > > frame pointer= 0x28:0xfe022ccce950 > > > code segment= base 0x0, limit 0xf, type 0x1b > > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > > processor eflags= resume, IOPL = 0 > > > current process= 10 (idle: cpu7) > > > > > > It happens at amd64/amd64/cpu_switch.S:ld_ldt:lldt%ax > > > > > > db> bt > > > > > > Tracing pid 10 tid 19 td 0xf80002292000 > > > ld_ldt() at ld_ldt/frame 0xfe022ccce8e0 > > > sched_switch() at sched_switch+0x51d/frame 0xfe022ccce950 > > > mi_switch() at mi_switch+0x156/frame 0xfe022ccce980 > > > sched_idletd() at sched_idletd+0xf9/frame 0xfe022cccea70 > > > fork_exit() at fork_exit+0x84/frame 0xfe022ccceab0 > > > fork_trampoline() at fork_trampoline+0xe/frame 0xfe022ccceab0 > > > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > > Have their been no other stability issues on this machine? At first glance it > looks like the ldt got corrupted. Is there no way to inspect that? It seems something wrong happens during incremental kernel update from 11-ALPLA3 to 12, i.e. some old kernel parts affects it. I clean all object and include files and rebuild the kernel from scratch and this panic is gone. Sorry for the noise, I should do it before reporting :( ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Recent -current stable panic at later boot stage
BTW, it never happens with 11-ALPHA3 even with world build for 12. On 09.07.2016 22:06, Andrey Chernov wrote: > SCHED_ULE used. I have textdump only. Interesting parts: > <118>Starting syslogd. > kernel trap 9 with interrupts disabled > > Fatal trap 9: general protection fault while in kernel mode > cpuid = 7; apic id = 07 > instruction pointer = 0x20:0x806b2585 > stack pointer = 0x28:0xfe022ccce8d0 > frame pointer = 0x28:0xfe022ccce950 > code segment = base 0x0, limit 0xf, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = resume, IOPL = 0 > current process = 10 (idle: cpu7) > > It happens at amd64/amd64/cpu_switch.S:ld_ldt:lldt%ax > > db> bt > > Tracing pid 10 tid 19 td 0xf80002292000 > ld_ldt() at ld_ldt/frame 0xfe022ccce8e0 > sched_switch() at sched_switch+0x51d/frame 0xfe022ccce950 > mi_switch() at mi_switch+0x156/frame 0xfe022ccce980 > sched_idletd() at sched_idletd+0xf9/frame 0xfe022cccea70 > fork_exit() at fork_exit+0x84/frame 0xfe022ccceab0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfe022ccceab0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Recent -current stable panic at later boot stage
SCHED_ULE used. I have textdump only. Interesting parts: <118>Starting syslogd. kernel trap 9 with interrupts disabled Fatal trap 9: general protection fault while in kernel mode cpuid = 7; apic id = 07 instruction pointer = 0x20:0x806b2585 stack pointer = 0x28:0xfe022ccce8d0 frame pointer = 0x28:0xfe022ccce950 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= resume, IOPL = 0 current process = 10 (idle: cpu7) It happens at amd64/amd64/cpu_switch.S:ld_ldt:lldt%ax db> bt Tracing pid 10 tid 19 td 0xf80002292000 ld_ldt() at ld_ldt/frame 0xfe022ccce8e0 sched_switch() at sched_switch+0x51d/frame 0xfe022ccce950 mi_switch() at mi_switch+0x156/frame 0xfe022ccce980 sched_idletd() at sched_idletd+0xf9/frame 0xfe022cccea70 fork_exit() at fork_exit+0x84/frame 0xfe022ccceab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe022ccceab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: difference in SIGCHLD behavior between Linux and FreeBSD breaks apt
On 07.07.2016 10:14, Don Lewis wrote: > On 6 Jul, Matthew Macy wrote: >> >> >> >> On Wed, 06 Jul 2016 23:48:53 -0700 Andrey Chernov >> wrote >> > On 07.07.2016 9:40, Matthew Macy wrote: >> > > >> > > >> > > >> > > On Wed, 06 Jul 2016 23:28:40 -0700 Andrey Chernov >> > > wrote >> > > > On 07.07.2016 7:52, K. Macy wrote: >> > > > > On Wednesday, July 6, 2016, Don Lewis >> > > > > wrote: >> > > > > >> > > > >> On 6 Jul, Matthew Macy wrote: >> > > > >>> As a first step towards managing linux user space in a >> > > > >>> chrooted >> > > > >>> /compat/linux, initially for i915 testing with intel gpu >> > > > >>> tools, later on to get widevine and steam to work I'm >> > > > >>> trying to get apt to work. I've fixed a number of issues >> > > > >>> to date in pseudofs/linprocfs but now I'm running in to >> > > > >>> a bug caused by differences in SIGCHLD handling between >> > > > >>> Linux and FreeBSD. The situation is that apt will spawn >> > > > >>> dpkg and wait on a pipe read. On Linux when dpkg exits >> > > > >>> the SIGCHLD to apt causes a short read on the pipe >> > > > >>> which lets apt then continue. On FreeBSD a SIGCHLD is >> > > > >>> silently ignored. I've even experimented with doing a >> > > > >>> kill -20 to no effect. >> > > > >>> >> > > > >>> It would be easy enough to check sysvec against linux in >> > > > >>> pipe_read and break out of the loop when it's awakened >> > > > >>> from msleep (assuming there aren't deeper issues with >> > > > >>> signal propagation for anything other than >> > > > >>> SIGINT/SIGKILL) and then do a short read. However, I'm >> > > > >>> assuming that anyone who has worked in this area >> > > > >>> probably has a cleaner solution. >> > > > >> >> > > > >> It shoulds like SA_RESTART is set in sa_flags for SIGCHLD >> > > > >> but shouldn't be in this case. >> > > > > >> > > > > >> > > > > >> > > > > Good point. >> > > > > >> > > > > Thinking more about it, this seems like a bug in FreeBSD. >> > > > > Not a valid behavioral difference. >> > > > >> > > > You better need consult with POSIX before fixing things toward >> > > > any Linuxisms blindly in our native code. I don't have a >> > > > time now to see, is it really a bug according to POSIX, but >> > > > please read or just find all SIGCHLD there: >> > > > http://pubs.opengroup.org/onlinepubs/9699919799/functions/wait.html >> > > > it explain SIGCHLD actions in deep details. >> > > > And that one too: >> > > > >> http://pubs.opengroup.org/onlinepubs/009695399/functions/sigaction.html >> > > >> > > >> > > >> > > I was pretty clear in my initial email that I'm only interested >> > > in changing behavior for Linux programs. >> > >> > Of course, but in case it is FreeBSD bug, it should be fixed in our >> > native code first before making any changes in Linuxator. >> > >> > > And I was asking for help with that, not a link to SUSv3 or POSIX. >> > >> > In case I was not helpful, sorry for that. Before you try to change >> > something in Linuxator you need to be sure that FreeBSD does it >> > right (or wrong, then fix FreeBSD native code first). I am just >> > insisting on proper steps of fixing it. >> > >> >> >> I'm sorry for snapping . I misunderstood your intent. Using a SIGCHLD >> to deliberately interrupt a pipe read is such a weird idiom. I'll test >> fork vs clone on Linux and see how OS X responds to a SIGCHLD during a >> pipe read. > > It really depends on how signal handling has been set up. From my > understanding of the FreeBSD man pages and the Open Group documents, the > default handling for SIGCHLD is to just ignore it, in which case it > shouldn't interrupt the pipe read. If the process has set up a SIGCHLD > signal handler, then what happens with the read should depend on whether > or not SA_RESTART was passed to sigaction(). I would expect that Linux > would be the same as FreeBSD and the Open Group specs. Linux as SysV derivative was always different regarding to SA_RESTART and other SA_* flags for signal(), see differences at the end of: http://linux.die.net/man/2/signal ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: difference in SIGCHLD behavior between Linux and FreeBSD breaks apt
On 07.07.2016 9:40, Matthew Macy wrote: > > > > On Wed, 06 Jul 2016 23:28:40 -0700 Andrey Chernov > wrote > > On 07.07.2016 7:52, K. Macy wrote: > > > On Wednesday, July 6, 2016, Don Lewis wrote: > > > > > >> On 6 Jul, Matthew Macy wrote: > > >>> As a first step towards managing linux user space in a chrooted > > >>> /compat/linux, initially for i915 testing with intel gpu tools, later > > >>> on to get widevine and steam to work I'm trying to get apt to work. > > >>> I've fixed a number of issues to date in pseudofs/linprocfs but now > > >>> I'm running in to a bug caused by differences in SIGCHLD handling > > >>> between Linux and FreeBSD. The situation is that apt will spawn dpkg > > >>> and wait on a pipe read. On Linux when dpkg exits the SIGCHLD to apt > > >>> causes a short read on the pipe which lets apt then continue. On > > >>> FreeBSD a SIGCHLD is silently ignored. I've even experimented with > > >>> doing a kill -20 to no effect. > > >>> > > >>> It would be easy enough to check sysvec against linux in pipe_read and > > >>> break out of the loop when it's awakened from msleep (assuming there > > >>> aren't deeper issues with signal propagation for anything other than > > >>> SIGINT/SIGKILL) and then do a short read. However, I'm assuming that > > >>> anyone who has worked in this area probably has a cleaner solution. > > >> > > >> It shoulds like SA_RESTART is set in sa_flags for SIGCHLD but shouldn't > > >> be in this case. > > > > > > > > > > > > Good point. > > > > > > Thinking more about it, this seems like a bug in FreeBSD. Not a valid > > > behavioral difference. > > > > You better need consult with POSIX before fixing things toward any > > Linuxisms blindly in our native code. I don't have a time now to see, is > > it really a bug according to POSIX, but please read or just find all > > SIGCHLD there: > > http://pubs.opengroup.org/onlinepubs/9699919799/functions/wait.html > > it explain SIGCHLD actions in deep details. > > And that one too: > > http://pubs.opengroup.org/onlinepubs/009695399/functions/sigaction.html > > > > I was pretty clear in my initial email that I'm only interested in changing > behavior for Linux programs. Of course, but in case it is FreeBSD bug, it should be fixed in our native code first before making any changes in Linuxator. > And I was asking for help with that, not a link to SUSv3 or POSIX. In case I was not helpful, sorry for that. Before you try to change something in Linuxator you need to be sure that FreeBSD does it right (or wrong, then fix FreeBSD native code first). I am just insisting on proper steps of fixing it. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: difference in SIGCHLD behavior between Linux and FreeBSD breaks apt
On 07.07.2016 7:52, K. Macy wrote: > On Wednesday, July 6, 2016, Don Lewis wrote: > >> On 6 Jul, Matthew Macy wrote: >>> As a first step towards managing linux user space in a chrooted >>> /compat/linux, initially for i915 testing with intel gpu tools, later >>> on to get widevine and steam to work I'm trying to get apt to work. >>> I've fixed a number of issues to date in pseudofs/linprocfs but now >>> I'm running in to a bug caused by differences in SIGCHLD handling >>> between Linux and FreeBSD. The situation is that apt will spawn dpkg >>> and wait on a pipe read. On Linux when dpkg exits the SIGCHLD to apt >>> causes a short read on the pipe which lets apt then continue. On >>> FreeBSD a SIGCHLD is silently ignored. I've even experimented with >>> doing a kill -20 to no effect. >>> >>> It would be easy enough to check sysvec against linux in pipe_read and >>> break out of the loop when it's awakened from msleep (assuming there >>> aren't deeper issues with signal propagation for anything other than >>> SIGINT/SIGKILL) and then do a short read. However, I'm assuming that >>> anyone who has worked in this area probably has a cleaner solution. >> >> It shoulds like SA_RESTART is set in sa_flags for SIGCHLD but shouldn't >> be in this case. > > > > Good point. > > Thinking more about it, this seems like a bug in FreeBSD. Not a valid > behavioral difference. You better need consult with POSIX before fixing things toward any Linuxisms blindly in our native code. I don't have a time now to see, is it really a bug according to POSIX, but please read or just find all SIGCHLD there: http://pubs.opengroup.org/onlinepubs/9699919799/functions/wait.html it explain SIGCHLD actions in deep details. And that one too: http://pubs.opengroup.org/onlinepubs/009695399/functions/sigaction.html ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 'make depend' or 'make' bug on recent --current
On 01.06.2016 23:04, Bryan Drewery wrote: > No. The build and how dependencies are generated and handled is still > fundamentally the same. Open the .depend.* files and see. It is only > simple dependencies for the target built. There's nothing new about its > content. If foo.c includes stdlib.h then it includes sys/_types.h which > includes sys/cdefs.h, etc. This graph is in the old (mkdep) and new > .depend* content. This is easily provable by just comparing mkdep > output to the new versions. > > Without specific evidence of a bug I cannot help. Thanx for pointing me to MD *div sources, I overlook them. About 'make depend', I don't notice anything suspicious during simple tests and don't have time or energy to look deeper in that area. Probably several clang libs rebuilds during the single day while none usually expected cause my overreaction. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 'make depend' or 'make' bug on recent --current
On 01.06.2016 22:36, Bryan Drewery wrote: >>> The graph in the original commit for WITH_FAST_DEPEND disagrees. >>> https://svnweb.freebsd.org/base?view=revision&revision=290433 >>> >>> We run the preprocessor once now, not twice. >> >> It sounds good, just implemented bad. You measure some spherical chicken >> in vacuum, not what really happens. In the old times I almost never have >> clang libs rebuild (few files from there max when FreeBSD_version is >> increased), but now I got them fully rebuilt with any header change. >> That is the biggest slowdown and not what you try to measure. >> Don't ever use 'make world'. Try to rebuild the system incrementally and >> you'll see. > > I cannot argue with a lack of solid evidence. > > The old 'make depend' ran 'mkdep' which ran 'cc -E -M' which produces > *the same output* as 'cc foo.c -o foo.o -MD -MF .depend.foo.o -MT > foo.o'. There's nothing different in the actual .depend file > implementation/content. Clang rebuilds often because it is changing > often! Just look at recent commit logs and you'll see r300984 which > will cause a rebuild of clang. Or r301011 which modified > sys/sys/param.h which will rebuild just about everything. These are > normal and how the old system worked as well. > > There certainly are some issues with the new system. > 1. Processing the split files can be slow over NFS > 2. make cleandepend - will cause the next build to make a lot of guesses > and not be very efficient. > 3. make cleandepend - There is no way to generate the .depend* files > again without rebuilding. > 4. It doesn't fix all of the missing dependencies that have been missing > forever such as crt, csu, libgcc, etc for static builds. > 5. the csu builds don't use it yet but have workarounds for it > 6. removing the 'make depend' tree-walk can hurt downstream builds which > relied on having a multi-phase build to generate a .mk file to include > (odd case I hit at work). No such case exists in the upstream FreeBSD tree. > There are two different things: pure recursive dependency (it is how make depends works now) and just nested include graph with dependencies marked directly by Makefile's rules (it is how old system works). Now it is enough to touch some low level include to rebuild everything, before only files which include it directly or have make rules for it are rebuilt. signature.asc Description: OpenPGP digital signature
Re: 'make depend' or 'make' bug on recent --current
On 01.06.2016 21:18, Bryan Drewery wrote: > On 6/1/2016 6:11 AM, Andrey Chernov wrote: >> Steps to reproduce: >> >> cd /usr/src/lib/libc/stdlib >> touch *div*.c >> cd .. >> make depend >> make >> >> And see how imaxdiv.o only is recompiled. >> No div.o ldiv.o lldiv.o are recompiled. > > My dev system is busy at the moment. I'll test it and get back to you. I need to add that I do 'make cleandepend/cleandir/cleanobj' + 'make obj' again and full rebuild with no old files, but the bug repeated again. >> P.S. new make depend is simple disgusting. It tends to recompile >> everything in the system if some minor header file is touched, but > > If the header is used by all source files then that is expected. > > However if you do not have a .depend.obj.o file then it is quite > aggressive with building. If you touch any header it will rebuild > everything. But you shouldn't get into that situation unless you rm -f > .depend* first. > >> completely forget to recompile source code changes. I suggest to back >> out all AI in that area. >> 'make depend' is not time-consuming task and good old way never made >> mistakes. > > The graph in the original commit for WITH_FAST_DEPEND disagrees. > https://svnweb.freebsd.org/base?view=revision&revision=290433 > > We run the preprocessor once now, not twice. It sounds good, just implemented bad. You measure some spherical chicken in vacuum, not what really happens. In the old times I almost never have clang libs rebuild (few files from there max when FreeBSD_version is increased), but now I got them fully rebuilt with any header change. That is the biggest slowdown and not what you try to measure. Don't ever use 'make world'. Try to rebuild the system incrementally and you'll see. signature.asc Description: OpenPGP digital signature
'make depend' or 'make' bug on recent --current
Steps to reproduce: cd /usr/src/lib/libc/stdlib touch *div*.c cd .. make depend make And see how imaxdiv.o only is recompiled. No div.o ldiv.o lldiv.o are recompiled. P.S. new make depend is simple disgusting. It tends to recompile everything in the system if some minor header file is touched, but completely forget to recompile source code changes. I suggest to back out all AI in that area. 'make depend' is not time-consuming task and good old way never made mistakes. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Recent -current cpucontrol panic: [cpuctl, 129]: cannot bind to target cpu 1
On 22.05.2016 12:32, Konstantin Belousov wrote: > On Sun, May 22, 2016 at 12:19:32PM +0300, Andrey Chernov wrote: >> On 22.05.2016 10:14, Konstantin Belousov wrote: >>> On Sun, May 22, 2016 at 09:47:08AM +0300, Andrey Chernov wrote: >>>> With microcode_update_enable="YES" in rc.conf: >>>> >>>> db:0:kdb.enter.panic> show pcpu >>>> cpuid= 1 >>>> dynamic pcpu = 0xfe02ac1fd300 >>>> curthread= 0xf8000ae75a00: pid 736 "cpucontrol" >>>> curpcb = 0xfe023754eb80 >>>> fpcurthread = none >>>> idlethread = 0xf800022b5000: tid 13 "idle: cpu1" >>>> curpmap = 0xf8000af02138 >>>> tssp = 0x80c1cf78 >>>> commontssp = 0x80c1cf78 >>>> rsp0 = 0xfe023754eb80 >>>> gs32p= 0x80c237d0 >>>> ldt = 0x80c23810 >>>> tss = 0x80c23800 >>>> db:0:kdb.enter.panic> bt >>>> Tracing pid 736 tid 100121 td 0xf8000ae75a00 >>>> kdb_enter() at kdb_enter+0x3b/frame 0xfe023754e620 >>>> vpanic() at vpanic+0x19f/frame 0xfe023754e6a0 >>>> kassert_panic() at kassert_panic+0x126/frame 0xfe023754e710 >>>> cpuctl_do_cpuid_count() at cpuctl_do_cpuid_count+0x85/frame >>>> 0xfe023754e750 >>>> cpuctl_ioctl() at cpuctl_ioctl+0x389/frame 0xfe023754e800 >>>> devfs_ioctl_f() at devfs_ioctl_f+0x156/frame 0xfe023754e860 >>>> kern_ioctl() at kern_ioctl+0x204/frame 0xfe023754e8c0 >>>> sys_ioctl() at sys_ioctl+0x171/frame 0xfe023754e9a0 >>>> amd64_syscall() at amd64_syscall+0x28e/frame 0xfe023754eab0 >>>> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe023754eab0 >>>> --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x8009778ea, rsp = >>>> 0x7fffe788, rbp = 0x7fffe7d0 --- >>> >>> Show verbose dmesg of your boot. >> >> Attached. > Thanks. > >> >>> What scheduler do you use ? >> >> ULE >> >>> In the fired KASSERT, add printout of td->td_oncpu and show the updated >>> panic message. >> >> I add, but can't reproduce this bug again so far. >> > Do you have core file for the panic ? If yes, it would be at least > interesting to see the output of 'p *td' in kgdb. > No, just textdump. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Recent -current cpucontrol panic: [cpuctl, 129]: cannot bind to target cpu 1
On 22.05.2016 10:14, Konstantin Belousov wrote: > On Sun, May 22, 2016 at 09:47:08AM +0300, Andrey Chernov wrote: >> With microcode_update_enable="YES" in rc.conf: >> >> db:0:kdb.enter.panic> show pcpu >> cpuid= 1 >> dynamic pcpu = 0xfe02ac1fd300 >> curthread= 0xf8000ae75a00: pid 736 "cpucontrol" >> curpcb = 0xfe023754eb80 >> fpcurthread = none >> idlethread = 0xf800022b5000: tid 13 "idle: cpu1" >> curpmap = 0xf8000af02138 >> tssp = 0x80c1cf78 >> commontssp = 0x80c1cf78 >> rsp0 = 0xfe023754eb80 >> gs32p= 0x80c237d0 >> ldt = 0x80c23810 >> tss = 0x80c23800 >> db:0:kdb.enter.panic> bt >> Tracing pid 736 tid 100121 td 0xf8000ae75a00 >> kdb_enter() at kdb_enter+0x3b/frame 0xfe023754e620 >> vpanic() at vpanic+0x19f/frame 0xfe023754e6a0 >> kassert_panic() at kassert_panic+0x126/frame 0xfe023754e710 >> cpuctl_do_cpuid_count() at cpuctl_do_cpuid_count+0x85/frame >> 0xfe023754e750 >> cpuctl_ioctl() at cpuctl_ioctl+0x389/frame 0xfe023754e800 >> devfs_ioctl_f() at devfs_ioctl_f+0x156/frame 0xfe023754e860 >> kern_ioctl() at kern_ioctl+0x204/frame 0xfe023754e8c0 >> sys_ioctl() at sys_ioctl+0x171/frame 0xfe023754e9a0 >> amd64_syscall() at amd64_syscall+0x28e/frame 0xfe023754eab0 >> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe023754eab0 >> --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x8009778ea, rsp = >> 0x7fffe788, rbp = 0x7fffe7d0 --- > > Show verbose dmesg of your boot. Attached. > What scheduler do you use ? ULE > In the fired KASSERT, add printout of td->td_oncpu and show the updated > panic message. I add, but can't reproduce this bug again so far. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Recent -current cpucontrol panic: [cpuctl, 129]: cannot bind to target cpu 1
With microcode_update_enable="YES" in rc.conf: db:0:kdb.enter.panic> show pcpu cpuid= 1 dynamic pcpu = 0xfe02ac1fd300 curthread= 0xf8000ae75a00: pid 736 "cpucontrol" curpcb = 0xfe023754eb80 fpcurthread = none idlethread = 0xf800022b5000: tid 13 "idle: cpu1" curpmap = 0xf8000af02138 tssp = 0x80c1cf78 commontssp = 0x80c1cf78 rsp0 = 0xfe023754eb80 gs32p= 0x80c237d0 ldt = 0x80c23810 tss = 0x80c23800 db:0:kdb.enter.panic> bt Tracing pid 736 tid 100121 td 0xf8000ae75a00 kdb_enter() at kdb_enter+0x3b/frame 0xfe023754e620 vpanic() at vpanic+0x19f/frame 0xfe023754e6a0 kassert_panic() at kassert_panic+0x126/frame 0xfe023754e710 cpuctl_do_cpuid_count() at cpuctl_do_cpuid_count+0x85/frame 0xfe023754e750 cpuctl_ioctl() at cpuctl_ioctl+0x389/frame 0xfe023754e800 devfs_ioctl_f() at devfs_ioctl_f+0x156/frame 0xfe023754e860 kern_ioctl() at kern_ioctl+0x204/frame 0xfe023754e8c0 sys_ioctl() at sys_ioctl+0x171/frame 0xfe023754e9a0 amd64_syscall() at amd64_syscall+0x28e/frame 0xfe023754eab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe023754eab0 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x8009778ea, rsp = 0x7fffe788, rbp = 0x7fffe7d0 --- ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Dropping some locales/encodings?
On 01.03.2016 12:54, Baptiste Daroussin wrote: >> What benefit does it bring to remove an already existing locale? > > The benefit is the fact we have no source for those and collation for one are > wrong for those. (We have added proper collation support in head). Unicode mapping for CP1251 (0x98 is undefined) http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1251.TXT Collation is the same as in CLDR -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: Dropping some locales/encodings?
On 01.03.2016 2:23, Baptiste Daroussin wrote: > Hi all, > > I have updated few month ago the locales to cldr v27.0.1. I would like to > simplify the generation of those locales from cldr to POSIX locales that we do > provides. > > I can properly generate almost any of the said locales/encodings but a few > that > I would like to remove (provided that unicode version are available) > > Here is the list of locales/encodings: > > be_BY.CP1251 CP1251 is Windows native (single characters mode) and widely used to represent Cyrillic: Russian, Bulgarian, Serbian, Ukrainian, Belarusian (i.e. be_BY), Macedonian. IMHO it will be better to not remove it to make easy handling of native encoded texts comes from Windows. Don't know about other mentioned. -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: /bin/ls formatting broken for non-C(?) locales
On 25.11.2015 18:12, Andrey Chernov wrote: > On 25.11.2015 17:35, Baptiste Daroussin wrote: >>> BTW, array size looks suspicious: >>> static wchar_t wab_months[12][MAX_ABMON_WIDTH * 2 * MB_LEN_MAX]; >>> what MB_LEN_MAX doing here? This constant is for multiple-bytes encoded, >>> not for wide chars. >> Bad copy/paste sorry it should be "MAX_ABMON_WIDTH * 2" > > I don't check deep enough, it seems first array > MAX_ABMON_WIDTH * MB_LEN_MAX + 1 > and second one > MAX_ABMON_WIDTH * 2 + 1 > No. We can't assume anything here and should integrate limits from the locale for months fields instead. F.e. in abstract general case in wide array can be 100 zero-width characters + 5 of normal characters, so width-oriented sizes not prevents overflowing. First array size should be from locale internals, second one == first * sizeof(wchar_t) it will be safe for not overflowing. -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: /bin/ls formatting broken for non-C(?) locales
On 25.11.2015 17:35, Baptiste Daroussin wrote: >> BTW, array size looks suspicious: >> static wchar_t wab_months[12][MAX_ABMON_WIDTH * 2 * MB_LEN_MAX]; >> what MB_LEN_MAX doing here? This constant is for multiple-bytes encoded, >> not for wide chars. > Bad copy/paste sorry it should be "MAX_ABMON_WIDTH * 2" I don't check deep enough, it seems first array MAX_ABMON_WIDTH * MB_LEN_MAX + 1 and second one MAX_ABMON_WIDTH * 2 + 1 -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: /bin/ls formatting broken for non-C(?) locales
On 25.11.2015 16:51, Baptiste Daroussin wrote:
> wcslcat works like strlcat:
> to quote the manpage:
> "It will append at most dstsize - strlen(dst) - 1 characters."
> So here with your version it will be n - wcslen(wab_months[i]) -1
> which won't fit what we want to do.
>
> btw that makes me figure out that what I want is wcsncat because we do care to
> make sure we have appended the right number of spaces at the end of the
> abbreviated month based on character width, not the on the len of the wide
> string
>
> so I changed it
>
> if ((n = max_month_width - wab_months_width[i]) > 0)
> wcsncat(wab_months[i], L" "/* MAX_ABMON_WIDTH */, n);
Sorry, I initially mean wcsncat() functionality here, wcslcat() is
sneaked in due to wrong memorizing.
About width counting and fallback...
Without attempt to nicely truncate data damaged by unknown way the code
will be simpler. Instead all that loop adding width one by one wchar, just:
width = wcswidth(wab_months[i], MAX_ABMON_WIDTH);
if (width == -1) {
max_month_width = -1;
return;
}
wab_months_width[i] = width;
About
/* NULL terminate to simplify padding later */
wab_months[i][j] = L'\0';
You don't need it, mbstowcs() null-terminates result, if there is a room.
BTW, array size looks suspicious:
static wchar_t wab_months[12][MAX_ABMON_WIDTH * 2 * MB_LEN_MAX];
what MB_LEN_MAX doing here? This constant is for multiple-bytes encoded,
not for wide chars.
--
http://ache.vniz.net/
signature.asc
Description: OpenPGP digital signature
Re: /bin/ls formatting broken for non-C(?) locales
On 25.11.2015 15:53, Baptiste Daroussin wrote:
> What I did for now is set max_month_width to -1 and in ls_strftime I fallback
> on
> the plain strftime meaning you keep localized information but the alignement
> is
> broken as of now.
It will be enough.
>> 3) wcwidth/wcswidth may return -1 too, it needs to be checked too.
> done and truncate the name of the month to the latest valid character
I think there is no point for sofisticated truncating, if locale is not
valid. F.e. you may truncate to just 1 char. The thing above will be
enough for all such cases.
> Review updated (if you prefer a diff by mail just tell me, given you do not
> have
> a phabricator account.)
I don't have phabricator.
This one
if ((n = max_month_width - wab_months_width[i]) > 0) {
wcslcat(wab_months[i], L" "/* MAX_ABMON_WIDTH */,
max_month_width + 1);
}
should be
if ((n = max_month_width - wab_months_width[i]) > 0)
wcslcat(wab_months[i], L" "/* MAX_ABMON_WIDTH */, n);
I.e. you append n spaces and n is the difference between max and current.
--
http://ache.vniz.net/
signature.asc
Description: OpenPGP digital signature
Re: /bin/ls formatting broken for non-C(?) locales
On 25.11.2015 4:31, Andrey Chernov wrote:
> 4) The whole processing looks overcomplicated and not effective. What
> about this instead?
> for (i = 0; i < 12; i++) {
> count wcswidth() of each month and store it in wab_months_width[].
> count max_width_month.
> }
> for (i = 0; i < 12; i++) {
> if ((n = max_width_month - wab_months_width[i]) > 0)
> call wcscat(wab_months[i], L" ") n times.
> }
Last line can be optimized further:
wcslcat(wab_months[i], L" "/* MAX_ABMON_WIDTH */, n);
--
http://ache.vniz.net/
signature.asc
Description: OpenPGP digital signature
Re: /bin/ls formatting broken for non-C(?) locales
On 25.11.2015 3:15, Baptiste Daroussin wrote:
> On Sat, Nov 21, 2015 at 11:57:46PM +0300, Andrey Chernov wrote:
>> On 21.11.2015 15:18, Ed Schouten wrote:
>>> Hi Baptiste,
>>>
>>> I suppose you should use the wcswidth() function somewhere to compute
>>> the visible width of the month name. Some characters may be
>>> double-width, others may have no effective width at all.
>>>
>>
>> I agree. Checking error return of wide chars functions with some
>> fallback will be good too.
>
> I have updated the code https://reviews.freebsd.org/D4239
>
> Tested by modifying some locales to add double width and zero width unicode in
> the locales
>
> Also added the error checking for the return of wide chars functions. For now
> I
> haven't added fallback, suggestions welcome if needed.
1) For just 1 char in wcswidth(&wab_months[i][j], 1); it is better to
use another function wcwidth(wab_months[i][j]);
2) By fallback I mean something which not stops ls working with
incorrect for some reason locale, like setting max_width_month to
MAX_ABMON_WIDTH on error return (from
mbstowcs/wcwidth/wcswidth/wcswidth) and exit from
populate_abbreviated_month().
3) wcwidth/wcswidth may return -1 too, it needs to be checked too.
4) The whole processing looks overcomplicated and not effective. What
about this instead?
for (i = 0; i < 12; i++) {
count wcswidth() of each month and store it in wab_months_width[].
count max_width_month.
}
for (i = 0; i < 12; i++) {
if ((n = max_width_month - wab_months_width[i]) > 0)
call wcscat(wab_months[i], L" ") n times.
}
5) If there is no %b is strftime() format, there is no sense to spend
CPU cycles on from populate_abbreviated_month(), so it should be called
only once inside ls_strftime() on first %b instead of calling it in
printtime() for all cases.
--
http://ache.vniz.net/
signature.asc
Description: OpenPGP digital signature
Re: /bin/ls formatting broken for non-C(?) locales
On 21.11.2015 15:18, Ed Schouten wrote: > Hi Baptiste, > > I suppose you should use the wcswidth() function somewhere to compute > the visible width of the month name. Some characters may be > double-width, others may have no effective width at all. > I agree. Checking error return of wide chars functions with some fallback will be good too. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: /bin/ls formatting broken for non-C(?) locales
On 21.11.2015 3:35, Baptiste Daroussin wrote: > Here is what I do propose (sorry for the ugly pad_to_col name, if one has > better > please share :D > > https://reviews.freebsd.org/D4239 The whole function is ugly, not only its name. Please no hardcoded constants assuming some internal encoding knowledge, they are wrong for non-UTF-8 encodings in any case, use wide chars instead. BTW, the same 3 chars restriction is in tar, cpio, pax, lots of ftp clients, i.e. where 'ls' emulated. -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: make installworld failing with locales due to broken symlinks
On 16.11.2015 5:37, Andrey Chernov wrote: > On 16.11.2015 4:57, NGie Cooper wrote: >> make: stopped in /usr/src/git >> $ ls -l /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE >> lrwxr-xr-x 1 root wheel 27 Nov 1 16:24 >> /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE >> $ readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE >> /usr/share/locale/la_LN.ISO8859-1 >> $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE` >> ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory > > As I remember, we already hit this type of problem before with old > locales. "ln -sf" and "rm -rf" should be used everywhere on install > target to avoid that. > Oops, auto-typo. "rm -f" instead, since all directories are created by mtree. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: make installworld failing with locales due to broken symlinks
On 16.11.2015 4:57, NGie Cooper wrote: > make: stopped in /usr/src/git > $ ls -l /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE > lrwxr-xr-x 1 root wheel 27 Nov 1 16:24 > /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE > $ readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE > /usr/share/locale/la_LN.ISO8859-1 > $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE` > ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory As I remember, we already hit this type of problem before with old locales. "ln -sf" and "rm -rf" should be used everywhere on install target to avoid that. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: libXO-ification - Why - and is it a symptom of deeper issues?
On 15.11.2015 20:37, Adrian Chadd wrote: > On 15 November 2015 at 09:10, Dan Partelly wrote: >> Meaning, is that simple to push things in head , if somone does the work, >> even with with no proper review of the problem at hand , and the proposed >> solutions ? > > Nope and yup. The juniper folk had a solution to a problem multiple > people had requested work on, and their proposal was by far the > furthest along code and use wise. > > It's all fine and good making technical decisions based on drawings > and handwaving and philosophizing, but at some point someone has to do > the code. Juniper's libxo was the furthest along in implementation and > production. It seems it is the only and final argument for libXO existence. I remember 2 or 3 discussions against libXO spontaneously happens in the FreeBSD lists, all ended with that, approximately: "we already have the code and you have just speculations". Alternative and more architecture clean ideas, like making standalone template-oriented parser probably based on liXO, are never seriously considered, because nobody will code it, not for other reasons. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Need help fixing failing locale tests
On 15.11.2015 18:38, Andrey Chernov wrote: >> For the traditional checks, It's ironic, DragonFly uses short locales >> like "fr_FR" which link to the approprioprate ISO8859 or UTF-8 locale. >> Bapt removed them to avoid a bike shed and if he had not done that, this >> gettext configure would not be an issue as "fr_FR" is the very first check. > > But I dislike short locales due to their uncertainty And gettext-0.19.6/gettext-tools/configure is good example of that uncertainty. If DragonFly short locale fr_FR links to 8859-1, configure test does what it is intended, but if it links to UTF-8 it does not. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Need help fixing failing locale tests
On 15.11.2015 18:17, John Marino wrote: See gettext-0.19.6/gettext-tools/configure, part started with # Test for the usual locale name. I don't have time to dig more such code now, but I remember I saw more for sure. > Of course it's the same topic. > If the configure of a port is affected by the locale, that's a big > problem. How is this not the same topic? This is environment problem, not presence or absence of some locales in the system (f.e. 8859-1 ones) we talked about. And can be easily fixed in the environment, unlike configure's silent script fallbacks, which can be fixed by recompiling only. About environment problem, we already have LANG=C somewhere in the /usr/ports/Mk/* (probably not in every place) and many ports set LANG=C or LC_ALL by themselves. > But I did look at gettext, which is configuring package based on what's > on the system (not environment). For the unicode checks, there is > obviously no issue. > > For the traditional checks, It's ironic, DragonFly uses short locales > like "fr_FR" which link to the approprioprate ISO8859 or UTF-8 locale. > Bapt removed them to avoid a bike shed and if he had not done that, this > gettext configure would not be an issue as "fr_FR" is the very first check. All parts are optional excepting language itself, so fr_FR is not short enough, but fr is. From POSIX: language[_territory][.codeset][@modifier] But I dislike short locales due to their uncertainty and remember some programs and shell scripts that attempts to parse locale name directly by itself for reasons I don't remember now. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Need help fixing failing locale tests
On 15.11.2015 18:01, Baptiste Daroussin wrote: > I have restored the 8859-1 Thanx! BTW, speaking with John I find at least one configure script from ports which depends on 8859-1 presence. See gettext-0.19.6/gettext-tools/configure, part started with # Test for the usual locale name. -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: Need help fixing failing locale tests
On 15.11.2015 17:47, John Marino wrote: >>> Please provide an example of such a program (in ports). >> >> See gettext-0.19.6/gettext-tools/configure, part started with >> # Test for the usual locale name. >> I don't have time to dig more such code now, but I remember I saw more >> for sure. > If the ports framework is not overriding locale to "C" during the build > then that's probably something that should be introduced. Ports should > not be producing different results based on the configuration of the > building machine at the time. This is not about resetting locale to "C" during the build. Please really look at "configure" specific part mentioned above, it is you who ask at least one specific example and then it is you who ignore the answer takes my time to find it. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Need help fixing failing locale tests
On 15.11.2015 16:54, John Marino wrote: > We are talking about people that install FreeBSD 11 as a release. If > it's an upgrade, it's documented in UPDATING (it will be) and anybody on > -CURRENT is taking responsibility for knowing what they are doing. *IF* > this is an obstacle, it's either trivial or that user shouldn't be using > -CURRENT to begin with. As I already say, I don't want to insist on any particular point of view in such area as human behavior. I just say that it is POLA violation (even while it is upgrade) and we can let users decide by themselves, what it best for them (without me at least). >> I tell about is not different text document encoding, but program >> failure due to inability to set his 8859-1 locale. > > Please provide an example of such a program (in ports). See gettext-0.19.6/gettext-tools/configure, part started with # Test for the usual locale name. I don't have time to dig more such code now, but I remember I saw more for sure. >> In any case it is related to the user behavior an various views on it. >> They can be different and I see no point to insist on my particular view >> at all. But... Programs configure soft-fails shows real danger here. > > and the impact of this is ... ? Various. It depends on for what reason program sense locale. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Need help fixing failing locale tests
On 15.11.2015 16:14, John Marino wrote: >> It is user confusion and his responsibility. It not leads to wrong >> program build f.e. Moreover, you can't protect users who set 8859-1 that >> way, they do not convert to 8859-15 as you assume but start to complain >> everywhere that FreeBSD is not working instead. > > Invalid. "locale -a" shows what locales are available. > The confusion is not with one user. It's with one user that produces > document in one encoding and a second user that choses the wrong one > (usually -1). -15 was designed to replace -1. No end-user use 'locale -a' to check locales. An end-user keep some 8859-1 version is set many years ago, and "FreeBSD not working" problem I tell about is not different text document encoding, but program failure due to inability to set his 8859-1 locale. In any case it is related to the user behavior an various views on it. They can be different and I see no point to insist on my particular view at all. But... Programs configure soft-fails shows real danger here. > OpenBSD removed ISO8859* completely. OpenBSD was never good in locales in any case for all that years and contributes nothing in that area (f.e. Citrus was made by NetBSD). Now they try to simplify their life of supporting them, but since for us now all locales are autogenerated from CLDR I see no room for simplification in that way. Our "cleaned" locales (against to POLA) can be restored by autogeneration with minimal efforts, and they even took very little disk space. > Also invalid. Locales are not standardized with regard to encoding, so > maintaining a museum of locales is specific to FreeBSD. Linux calls > them differently. Most of our (old) locales have the same names as linux ones. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Need help fixing failing locale tests
> Well, there is "harm". The -1/-15 confusion happens a lot. It is user confusion and his responsibility. It not leads to wrong program build f.e. Moreover, you can't protect users who set 8859-1 that way, they do not convert to 8859-15 as you assume but start to complain everywhere that FreeBSD is not working instead. > Is the plan to keep every locale ever created for ever and ever? Never > do any kind of kind of cleanup or reorg? It will be nice to do it that way. FreeBSD have a little part of world locales, which indirectly assumes that they are really used. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Need help fixing failing locale tests
On 15.11.2015 16:00, Baptiste Daroussin wrote: > On Sun, Nov 15, 2015 at 03:56:26PM +0300, Andrey Chernov wrote: >> On 15.11.2015 15:46, Baptiste Daroussin wrote: >>> On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote: >>>> On 15.11.2015 10:09, John Marino wrote: >>>> ISO8859-1 locales are legacy even if obsoleted in modern world (I agree >>>> with that). Lots of ports (even at configure stage!) have checks for >>>> them. Since we generate locales from CLDR now, it will be no cost to >>>> bring all 8859-1 back to not violate POLA and not fix every failing port. >>>> >>> Exp-run have been made and no ports were failing with the removed locales. >> >> There is soft-fail, configure just marks that locales are not supported >> and use "C". Sorry I don't remember port names where I saw it right now >> and don't have a time to search for them right now too. Soft-fails (like >> in tcl with nl_langinfo) are almost impossible to detect excepting >> specific situation happens or source code inspection. Do we ever need >> them when there is no harm to keep 8859-1 locales? > > Is it ok if I readd those locales as aliases on 8859-15? It is hacking solution leads to wrong collating order and character classes. It is better to generate true 8859-1 just in the same way you already do for 8859-15. BTW, I can't check right now, but in case 8859-5 is removed too, it is better to restore it, it was used in Suns as their standard Russian encoding. -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: Need help fixing failing locale tests
On 15.11.2015 15:46, Baptiste Daroussin wrote: > On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote: >> On 15.11.2015 10:09, John Marino wrote: >> ISO8859-1 locales are legacy even if obsoleted in modern world (I agree >> with that). Lots of ports (even at configure stage!) have checks for >> them. Since we generate locales from CLDR now, it will be no cost to >> bring all 8859-1 back to not violate POLA and not fix every failing port. >> > Exp-run have been made and no ports were failing with the removed locales. There is soft-fail, configure just marks that locales are not supported and use "C". Sorry I don't remember port names where I saw it right now and don't have a time to search for them right now too. Soft-fails (like in tcl with nl_langinfo) are almost impossible to detect excepting specific situation happens or source code inspection. Do we ever need them when there is no harm to keep 8859-1 locales? -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: Need help fixing failing locale tests
On 15.11.2015 15:24, Andrey Chernov wrote: > On 15.11.2015 10:09, John Marino wrote: >> We (DragonFly) didn't just update locales. We took the opportunity to >> do spring cleaning. We didn't want to be as drastic as OpenBSD which >> removed all encodings except for C/POSIX and UTF, but we did remove >> several locales intentionally. >> >> In the case of ISO8859-1: >> All ISO8859-* is basically obsolete. >> In western Europe, if somebody wants ISO-8859, they want ISO8859-15, not >> ISO8859-1. They are similar, but the former is tailored for western >> europe with "Euro" currency and 9 other symbols. It comes at the >> expense of removing 10 characters from ISO8859-1. Forget to reply on that part: >> There's also a common >> problem that users view -15 documents with -1 accidently. So there was >> a conscience decision to have either ISO8859-1 or ISO8859-15 but not >> both. For western Europe this means the ISO8859-1 versions were dropped. It is pure user problem choosing its own locale (self footshooting), it not leads to program build failures (like configure checks) or tests failures etc. BTW, linux keeps 8859-1 locales. > ISO8859-1 locales are legacy even if obsoleted in modern world (I agree > with that). Lots of ports (even at configure stage!) have checks for > them. Since we generate locales from CLDR now, it will be no cost to > bring all 8859-1 back to not violate POLA and not fix every failing port. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Need help fixing failing locale tests
On 15.11.2015 10:09, John Marino wrote: > We (DragonFly) didn't just update locales. We took the opportunity to > do spring cleaning. We didn't want to be as drastic as OpenBSD which > removed all encodings except for C/POSIX and UTF, but we did remove > several locales intentionally. > > In the case of ISO8859-1: > All ISO8859-* is basically obsolete. > In western Europe, if somebody wants ISO-8859, they want ISO8859-15, not > ISO8859-1. They are similar, but the former is tailored for western > europe with "Euro" currency and 9 other symbols. It comes at the > expense of removing 10 characters from ISO8859-1. There's also a common > problem that users view -15 documents with -1 accidently. So there was > a conscience decision to have either ISO8859-1 or ISO8859-15 but not > both. For western Europe this means the ISO8859-1 versions were dropped. ISO8859-1 locales are legacy even if obsoleted in modern world (I agree with that). Lots of ports (even at configure stage!) have checks for them. Since we generate locales from CLDR now, it will be no cost to bring all 8859-1 back to not violate POLA and not fix every failing port. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD_HEAD-tests - Build #1675 - Unstable
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 09.11.2015 0:46, Baptiste Daroussin wrote: >> BTW, see other tests fails here in jenkins too, they looks >> related to locale changes. > > I will look into them. About that one, it looks like collate support is broken in regex: FAILED: bin.sh.builtins.functional_test.case7 All our collate files was in order A-Za-z but CLDR collate organized as aA<...>-zZ Since RE ranges should use collate (per POSIX), new a-z includes now every letter, small and big, excluding Z (for de_DE locale in the test). It means, case7 two tests (below) should succeed even with new collate, but both fails (second one includes various oO variants, with umlaut too). Our regex code have collate support for single byte locales, but it seems it is not in the working state now. unset LC_ALL LC_CTYPE=de_DE.ISO8859-1 export LC_CTYPE LC_COLLATE=de_DE.ISO8859-1 export LC_COLLATE c1=e # o umlaut c2=$(printf '\366') case $c1$c2 in [a-z][a-z]) ;; *) echo wrong at $LINENO ;; esac case $c1$c2 in [a-f][n-p]) ;; *) echo wrong at $LINENO ;; esac - -- http://ache.vniz.net/ -BEGIN PGP SIGNATURE- iQEcBAEBCAAGBQJWQAw2AAoJEKUckv0MjfbKRA0H/3c2n0slDo78mnxUv+wi7LKU lppgVWkBitskMNq02OBaV4kOogbrwe5RAfhDqFhOIB4oISh/1qMZi2lCiSyx6wAY ZRqaJXefMxLm+oFA9RECiSUrwPaAAS71spCVDBzVDZoRbHobJCkmt9/DdgDBE3P4 lzTucr2XAhXTBBmHIQKDzKBfnm8Pi/r6H74K1UkthCRe0TP01aW8QnVuJYrzIYE1 hPv7kbdO62w7NxARRaQQkMediDh5odl3pcIbH16EexnaCOG6ouTnY4dcCMwDOcz+ 9IwBwh+Nn0ERICcHciYS9C5j+bSoP1BBAwwhYi2+kCitRGFOQ/U8PjYkBPbT0Xw= =y4/D -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD_HEAD-tests - Build #1675 - Unstable
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 09.11.2015 0:46, Baptiste Daroussin wrote:
>> Error Message:
> printf("%'.4f", 12345678.0625) ==> [1,23,45,678.0625],
> expected [123,456,78.0625]<>
Looks like numericdef "grouping" handling problem...
>>>
>>> Seems like the issue is only with hi_IN.ISCII-DEV, I'll dig
>>> into it
>>
>> Old FreeBSD hi_IN.ISCII-DEV locale uses 2;3 grouping. New
>> hi_IN.UTF-8 locale use 3;2 grouping instead. I don't know what is
>> right in India.
>
> About the locales we take the rules from http://cldr.unicode.org/
> if there are issues they should be reported there! (note that those
> rules are also used by the ICU project for examples.
According to https://en.wikipedia.org/wiki/Indian_numbering_system
cldr is right, old FreeBSD and the test is wrong here.
- --
http://ache.vniz.net/
-BEGIN PGP SIGNATURE-
iQEcBAEBCAAGBQJWP8gCAAoJEKUckv0MjfbKnXsIAIXvlY48Op9uTRd+k7KUe6s+
Npmtn+b0AN+3tA47DvhPPAyF6hIyBXaEN93U60We5jv99U+nO4Tq6pXVFfp8nh67
+H43shwt/wckcf4cOGvn+koPYNqlU9CYvbxfV8hlWGZ56YZehKlxWJRxD7iITRva
3qqYvVdbf6aSGVzBzuoXgzpMYD4z1/eT4WzYyE1/aCy5KB1UGKHi0/XeLwHCSx7G
p8k+lhC10OJt+ueDIVK6X53wvQK1F1aJMhPEP1gW5CSN+3c2eTqz2zIKQVAGLQmm
xo0aHaHpjyv1aKg1XHxglrpc7N/z0waSpDm9LVbwWm6Xq2g6288nmlV0oQkJjvg=
=H4nd
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD_HEAD-tests - Build #1675 - Unstable
On 09.11.2015 0:08, Baptiste Daroussin wrote:
> On Sun, Nov 08, 2015 at 11:10:58PM +0300, Andrey Chernov wrote:
>> On 08.11.2015 22:08, jenkins-ad...@freebsd.org wrote:
>>> FreeBSD_HEAD-tests - Build #1675 - Unstable:
>>>
>>> Build information:
>>> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1675/ Full
>>> change log:
>>> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1675/changes
>>>
>>>
Full build log:
https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1675/console
>> ...
>>> FAILED:
>>> lib.libc.stdio.printfloat_test.thousands_separator_and_other_locale_tests
>>>
>>>
>>>
Error Message:
>>> printf("%'.4f", 12345678.0625) ==> [1,23,45,678.0625], expected
>>> [123,456,78.0625]<>
>>
>> Looks like numericdef "grouping" handling problem...
>
> Seems like the issue is only with hi_IN.ISCII-DEV, I'll dig into
> it
Old FreeBSD hi_IN.ISCII-DEV locale uses 2;3 grouping. New hi_IN.UTF-8
locale use 3;2 grouping instead. I don't know what is right in India.
BTW, see other tests fails here in jenkins too, they looks related to
locale changes.
--
http://ache.vniz.net/
signature.asc
Description: OpenPGP digital signature
Re: FreeBSD_HEAD-tests - Build #1675 - Unstable
On 08.11.2015 22:08, jenkins-ad...@freebsd.org wrote:
> FreeBSD_HEAD-tests - Build #1675 - Unstable:
>
> Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1675/
> Full change log:
> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1675/changes
> Full build log:
> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1675/console
...
> FAILED:
> lib.libc.stdio.printfloat_test.thousands_separator_and_other_locale_tests
>
> Error Message:
> printf("%'.4f", 12345678.0625) ==> [1,23,45,678.0625], expected
> [123,456,78.0625]<>
Looks like numericdef "grouping" handling problem...
--
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD_HEAD-tests - Build #1636 - Still Unstable
On 02.11.2015 11:36, jenkins-ad...@freebsd.org wrote: > FreeBSD_HEAD-tests - Build #1636 - Still Unstable: > > Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1636/ > Full change log: > https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1636/changes > Full build log: > https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1636/console > > Change summaries: > > No changes > > > The failed test cases: > > 4 tests failed. > FAILED: lib.libc.stdio.fmemopen2_test.test_data_length > > Error Message: > /builds/FreeBSD_HEAD/lib/libc/tests/stdio/fmemopen2_test.c:203: pos == 0 not > met > > FAILED: lib.libc.stdio.fmemopen_test.test09 > > Error Message: > 3956 checks failed; see output for more details > > FAILED: lib.libc.stdio.fmemopen_test.test11 > > Error Message: > 6020 checks failed; see output for more details > > FAILED: lib.libc.stdio.fmemopen_test.test14 > > Error Message: > 1806 checks failed; see output for more details It is already fixed in -current a day ago, i.e. runtime is outdated. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ls is broken for non-C locales due to libxo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 17.06.2015 16:58, Marcel Moolenaar wrote: > >> On Jun 16, 2015, at 9:53 PM, Andrey Chernov >> wrote: > >> Should be no any %FF, but single char in pre libxo ls or nothing >> in post libxo one. >> >> Use LANG=ru_RU.KOI8-R before touch command. It looks like you >> create file with name "%FF" instead. > > No difference: > > fbsdvm64% env LANG=ru_RU.KOI8-R touch `env LANG=ru_RU.KOI8-R printf > "\377"` fbsdvm64% ls -al total 8 -rw-r--r-- 1 marcel staff0 > Jun 17 06:56 %FF drwxr-xr-x 3 marcel staff 102 Jun 17 06:56 . > drwxr-xr-x 12 marcel staff 408 Jun 17 06:55 .. The original bug was fixed in r284494 by kan@ In any case, what you demonstrates is very strange and can be display (console,xterm,etc.) bug or probably libxio bug, because ls alone _never_ use %xx encoding, it should print '?' for invalid character or just character itself for valid ones. - -- http://ache.vniz.net/ -BEGIN PGP SIGNATURE- iQEcBAEBCAAGBQJVgaHdAAoJEKUckv0MjfbKtUkIAMFtFY+o3CQ607jxr5sRkoFE wh27gsUYA/F9FfLmzzcTI+iMPJI0Q72gMY/lgVlaekGSSehZ6EMOvgsRZsSHhLuC b8bDL2ij2u+3eqlbhurpww6ZiKHLYWkBcO4ZKaoyZ0umXyij8sp0dC5WKXOdBqtR 4OGfr9SEuodnKqKEjBAakPBzKaefwHEVIpVMYV2K7ajFswRV3vfRk7n0CTz0K5lZ qVXSrICOPJetGPtknZw9J/XQbbgnIQ9sKHE6LX0bBBVajSjrnJFtk7lSqGozYC8i 6GekMc3huR4IkV1JtxNR5OEH2GsoPiJwg/4XeO5ZeAHawrMtaznVBt5oVSAxaZc= =mmEI -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ls is broken for non-C locales due to libxo
Marcel Moolenaar пишет: > > >On Jun 16, 2015, at 8:46 PM, Andrey Chernov wrote: > >Signed PGP part >On 17.06.2015 6:23, Marcel Moolenaar wrote: > >Date/time is fixed. I don?t know how to reproduce the filename >problem, so make sure sources are up-to-date and if still a >problem, provide me with a way to reproduce. > > >touch `printf "\377"` >env LANG=ru_RU.KOI8-R ls -l | od -bc | grep 377 > > >fbsdvm64% touch `printf "\377”` >fbsdvm64% ls -l | head -2 >total 96 >-rw-r--r-- 1 marcel staff 0 Jun 16 21:25 %FF > >fbsdvm64% env LANG=ru_RU.KOI8-R ls -l | head -2 >total 96 >-rw-r--r-- 1 marcel staff 0 16 ??? 21:25 %FF > >So, what’s wrong? > >-- >Marcel Moolenaar >mar...@xcllnt.net Should be no any %FF, but single char in pre libxo ls or nothing in post libxo one. Use LANG=ru_RU.KOI8-R before touch command. It looks like you create file with name "%FF" instead. signature.asc Description: PGP signature
Re: ls is broken for non-C locales due to libxo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 17.06.2015 6:23, Marcel Moolenaar wrote: > Date/time is fixed. I don?t know how to reproduce the filename > problem, so make sure sources are up-to-date and if still a > problem, provide me with a way to reproduce. touch `printf "\377"` env LANG=ru_RU.KOI8-R ls -l | od -bc | grep 377 - -- http://ache.vniz.net/ -BEGIN PGP SIGNATURE- iQEcBAEBCAAGBQJVgO2wAAoJEKUckv0MjfbKm94H/ic8hVCgYX1hFA6YAT8hksHF +RMrrus4iElF0Dz7sxx4q0CPBhIP2NPVXnNQZUeZwFHe4x/oFC7KYfZH6s74xoU3 AU1orszVWm2OhZ0RiB2CXnbxMhHyDm1gIRLm6sFtvE95SgDW0k1hg4Ym3tBfQOB1 XPsFF8dU187n4DXYSzWZ9FuCYjrCfU4BkceHvqJNwuC9wp7fKMM0A7TLUC5iOD7S Eq+HGFSWmtoesPyySS9NKxgWGf9aS0JwB3V7Cd0fyGSpNyAL1DQJvWlbtm+Bfdx9 7mpDeCzAnZXNZhE70SNWoHWqAXCpwJy76dU3sIlWlC6MmbbLyy0V64CO0kW6ubQ= =xWMi -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ls is broken for non-C locales due to libxo
On 17.06.2015 5:18, Andrey Chernov wrote: > Recent -current, LANG=ru_RU.KOI8-R locale. ls -t eats whole date column, just > ls eats whole filenames with printable national characters inside. Long live > libxo. > I mean ls -l. To be precise, for 8bit non-C locales at least: ls -lt: skip date column and eat filename with even single 8bit character inside. ls -l: just eat filename as above. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
ls is broken for non-C locales due to libxo
Recent -current, LANG=ru_RU.KOI8-R locale. ls -t eats whole date column, just ls eats whole filenames with printable national characters inside. Long live libxo. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: seekdir/readdir patch .. Call for Review.
On 03.05.2015 16:01, Julian Elischer wrote: >> Before making single-purpose changes to the libc readdir and seekdir >> code, or to the kernel code, it would be useful to state exact behaviour >> of the dirent machinery we want to see. No, 'make samba works in my >> situation' does not sound good enough. > Well samba is a MAJOR application for FreeBSD. there are many people > who combine the two, (e.g. FreeNAS, which suffers from this problem) > so taking it from "The Samba community does not recommend > running on FreeBSD due to problems with seekdir" to "Samba works > correctly on FreeBSD" is important.. Samba's behaviour here is governed > by Windows expectiations. Please look at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198819 too. Perhaps it is related or may be not (sorry currently I am not able to inspect the code). -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Massive libxo-zation that breaks everything
On 04.03.2015 19:21, John Baldwin wrote: > On Tuesday, March 03, 2015 09:09:43 AM David Chisnall wrote: >> Hopefully there's a lesson here that we can learn from: human-readable >> formats do not make good intermediate representations when communicating >> between tools. > > I think this is actually an argument against libxo-ification in the one case > where I've cringed a bit at the diffs: pciconf. The current pciconf code is > tailored to outputting something human readable. For non-human output I would > probably generate different output (not just put tags on the human output) > because I would want the non-human output to be both more verbose and more > raw. I think some other cases like 'netstat -s' are far more straightforward > as the current output maps fairly well to the backing structure, but in > general I would want machine-readable output that is closer to the structures > than to the human-readable formatting of them. > > For example, for something like 'mfiutil show drives', I would want the human > readable format to stay as it is (it only highlights certain fields in the PD > structures) but I would want the machine-readable format to basically output > tagged versions of the backing structures from sys/dev/mfi/mfireg.h. That > way > the machine-readable format has all of the data instead of only the subset > that is presented in the human-readable output. > > So while I am in general a big fan of having machine-readable output from > tools (and I think it belongs in the base system, and I don't think you want > a > post-processing tool), I think there is a bit of a flawed assumption that > says > that I want the same data in the human-readable format that I want in the > machine-readable format. I, for one, don't. I want the human-readable form > more condensed. > >> If your argument is about maintainability of these changes, then please >> point to concrete instances where the changes are complex and difficult to >> maintain. > > When I've looked at the xo diffs for pciconf, my reaction has been "ugh, I > guess I'm not going to work on pciconf again in the future because that's > super ugly". I don't object to the idea, I think I would just rather have a > very different schema for machine-readable output. I would probably want > pciconf -l in that case to dump the entire PCI header (right now the human- > readable pciconf -l only dumps a subset), and I would want it to dump fields > in capabilities that we don't currently bother printing (and that I don't > think the human-readable output should print due to it being too obscure, > etc.) > I agree that adding machine-readable output + separate option on a per-tool basis, when it is really needed, is more clean approach which don't break Unix way of things. In such case we don't intermix structured representation with human readable, so the chances that a bug in one representation code flow will affect other one are minimal unlike in unified libxo approach, which already demonstrate this weakness. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Massive libxo-zation that breaks everything
On 03.03.2015 4:45, Alfred Perlstein wrote: > > >> On Mar 2, 2015, at 5:37 PM, Andrey Chernov wrote: >> >>> On 03.03.2015 4:30, Alfred Perlstein wrote: >>> >>> >>>>> On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: >>>>> >>>>>> On 02.03.2015 22:55, Julian Elischer wrote: >>>>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>>>> >>>>>> >>>>>>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>>>>> Thanks! >>>>>>>> >>>>>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>>>>> putting into base, over a port or package, since processing XML in base >>>>>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>>>>> utilities anyway. >>>>>>>> >>>>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it. >>>>>>>> I'm >>>>>>>> trying to understand the use case for this.) >>>>>>> >>>>>>> To me it would almost seem more useful to have a programmable filter >>>>>>> for which you could produce >>>>>>> parse grammars to parse the output of various programs.. >>>>>>> thus >>>>>>> >>>>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>>>>> with a set of grammars in /usr/share/xmlize/ >>>>>>> then we could use it for out-of-tree programs as well if we wrote >>>>>>> grammars for them.. >>>>>>> >>>>>>> The sentiment of machine-readable output is nice, but I think it's >>>>>>> slightly off target. >>>>>>> we shouldn't have to change all out utilities, and it isn't going to >>>>>>> help at all with 3rd party apps, >>>>>>> e.g. samba stuff. A generally easy to program output grammar parser >>>>>>> would be truely useful. >>>>>>> and not just for FreeBSD. >>>>>>> >>>>>>> I've been watching with an uncomfortable feeling, but it's taken me a >>>>>>> while to put my >>>>>>> finger on what it was.. >>>>>> Are you sure it's not the hairs on the back of your neck standing up >>>>>> due to NIH? >>>>>> >>>>>> Juniper has been doing this for years and it's very useful for them. >>>>> I'm not saying the ability to generate machine readable output is wrong, >>>>> but that the 'unix way' would be to make a filter for it. It seems that >>>>> the noisy people don't >>>>> agree with me so I will not stand in the way of progress.. >>>> >>>> I agree. Even if someone starts with json and xml only, it will need >>>> some 3rd format soon, and adding any new format have real possibility to >>>> break all already existent (like adding json+xml breaks plain text in >>>> pipes). Moreover, it violates Unix principle 'one tool == one general >>>> function' and lots of other rules like Eric Raymond ones, making each >>>> program looks like systemd. It makes harder to merge changes from other >>>> BSDs too. >>>> Proper way to do this thing is to back out all changes and write >>>> completely separate templates-based parser - xml/json writer. >>> >>> >>> Read the library. It doesn't care what output format it needs. It is up to >>> the translation layer to do it. You could even do a csv format or most any >>> other structured output format without changing the userland utils. >> >> I am happy the library can do it. So please stop to change userland >> utils and back out all libxo changes from them. My concern is userland >> utils, feel free to implement anything you need/want without changing >> them in this ugly way. > > > The responsibility is on you to provide something better, both the > architecture AND code. So if you want it backed out, then write something > better. Otherwise step back and let progress happen. > As it seems you know a lot about my responsibility and duty, I am very surprised by your broad interests. If you let me to speak for myself a bit, I can tell what I feel. In this particular case my responsibility is just to give good advice at the road fork with one way leads to obvious hell, nothing more. I already express my opinion from the technical point of view and don't want to participate in the flame war, so continue this thread without me, please. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Massive libxo-zation that breaks everything
On 03.03.2015 3:48, Allan Jude wrote:
> On 2015-03-02 19:22, Andrey Chernov wrote:
>> On 02.03.2015 22:55, Julian Elischer wrote:
>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote:
>>>>
>>>>
>>>> On 3/2/15 4:14 AM, Julian Elischer wrote:
>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote:
>>>>>> Thanks!
>>>>>>
>>>>>> That does seem useful, but I'm not sure I see the reasoning behind
>>>>>> putting into base, over a port or package, since processing XML in base
>>>>>> is a pain, and it can't serve up JSON or HTML without additional
>>>>>> utilities anyway.
>>>>>>
>>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it.
>>>>>> I'm
>>>>>> trying to understand the use case for this.)
>>>>>
>>>>> To me it would almost seem more useful to have a programmable filter
>>>>> for which you could produce
>>>>> parse grammars to parse the output of various programs..
>>>>> thus
>>>>>
>>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
>>>>> with a set of grammars in /usr/share/xmlize/
>>>>> then we could use it for out-of-tree programs as well if we wrote
>>>>> grammars for them..
>>>>>
>>>>> The sentiment of machine-readable output is nice, but I think it's
>>>>> slightly off target.
>>>>> we shouldn't have to change all out utilities, and it isn't going to
>>>>> help at all with 3rd party apps,
>>>>> e.g. samba stuff. A generally easy to program output grammar parser
>>>>> would be truely useful.
>>>>> and not just for FreeBSD.
>>>>>
>>>>> I've been watching with an uncomfortable feeling, but it's taken me a
>>>>> while to put my
>>>>> finger on what it was..
>>>>>
>>>>>
>>>> Are you sure it's not the hairs on the back of your neck standing up
>>>> due to NIH?
>>>>
>>>> Juniper has been doing this for years and it's very useful for them.
>>> I'm not saying the ability to generate machine readable output is wrong,
>>> but that the 'unix way' would be to make a filter for it. It seems that
>>> the noisy people don't
>>> agree with me so I will not stand in the way of progress..
>>
>> I agree. Even if someone starts with json and xml only, it will need
>> some 3rd format soon, and adding any new format have real possibility to
>> break all already existent (like adding json+xml breaks plain text in
>> pipes). Moreover, it violates Unix principle 'one tool == one general
>> function' and lots of other rules like Eric Raymond ones, making each
>> program looks like systemd. It makes harder to merge changes from other
>> BSDs too.
>> Proper way to do this thing is to back out all changes and write
>> completely separate templates-based parser - xml/json writer.
>>
>
> Have you actually looked at libxo? It isn't really specific to xmj/json,
> and could handle adding another format without the need to modify wc
>
> xo_open_container("wc");
> xo_open_list("file");
> xo_open_instance("file");
> if (cnt(*argv) != 0)
> ++errors;
> xo_close_instance("file");
> xo_close_list("file");
> xo_close_container("wc");
> xo_finish();
>
>
> with cnt() being the existing functions in wc, but with the printf
> replaced with an xo wrapper. There is nothing specific to json or xml,
> that is all handled in libxo.
So, why you ever need to modify wc? Just load wc inside your
json/xml/etc writer, replacing its printf at the ld-elf.so level.
If you can't get json/xml/whatever from wc directly, it is not wc
problem, so don't touch it. The problem is on your side, feel free to
implement anything you need without affecting source code of others.
> Obviously, more care needs to be taken when converting the utilities to
> ensure that the existing output is not changed, but it is not quite as
> invasive as some people seem to think.
I think yes, it is very invasive and leads to dead end as architectural
design.
--
http://ache.vniz.net/
signature.asc
Description: OpenPGP digital signature
Re: Massive libxo-zation that breaks everything
On 03.03.2015 4:30, Alfred Perlstein wrote: > > >> On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: >> >>> On 02.03.2015 22:55, Julian Elischer wrote: >>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>> >>>> >>>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>>> Thanks! >>>>>> >>>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>>> putting into base, over a port or package, since processing XML in base >>>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>>> utilities anyway. >>>>>> >>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it. >>>>>> I'm >>>>>> trying to understand the use case for this.) >>>>> >>>>> To me it would almost seem more useful to have a programmable filter >>>>> for which you could produce >>>>> parse grammars to parse the output of various programs.. >>>>> thus >>>>> >>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>>> with a set of grammars in /usr/share/xmlize/ >>>>> then we could use it for out-of-tree programs as well if we wrote >>>>> grammars for them.. >>>>> >>>>> The sentiment of machine-readable output is nice, but I think it's >>>>> slightly off target. >>>>> we shouldn't have to change all out utilities, and it isn't going to >>>>> help at all with 3rd party apps, >>>>> e.g. samba stuff. A generally easy to program output grammar parser >>>>> would be truely useful. >>>>> and not just for FreeBSD. >>>>> >>>>> I've been watching with an uncomfortable feeling, but it's taken me a >>>>> while to put my >>>>> finger on what it was.. >>>> Are you sure it's not the hairs on the back of your neck standing up >>>> due to NIH? >>>> >>>> Juniper has been doing this for years and it's very useful for them. >>> I'm not saying the ability to generate machine readable output is wrong, >>> but that the 'unix way' would be to make a filter for it. It seems that >>> the noisy people don't >>> agree with me so I will not stand in the way of progress.. >> >> I agree. Even if someone starts with json and xml only, it will need >> some 3rd format soon, and adding any new format have real possibility to >> break all already existent (like adding json+xml breaks plain text in >> pipes). Moreover, it violates Unix principle 'one tool == one general >> function' and lots of other rules like Eric Raymond ones, making each >> program looks like systemd. It makes harder to merge changes from other >> BSDs too. >> Proper way to do this thing is to back out all changes and write >> completely separate templates-based parser - xml/json writer. > > > Read the library. It doesn't care what output format it needs. It is up to > the translation layer to do it. You could even do a csv format or most any > other structured output format without changing the userland utils. I am happy the library can do it. So please stop to change userland utils and back out all libxo changes from them. My concern is userland utils, feel free to implement anything you need/want without changing them in this ugly way. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Massive libxo-zation that breaks everything
On 02.03.2015 22:55, Julian Elischer wrote: > On 3/2/15 5:27 AM, Alfred Perlstein wrote: >> >> >> On 3/2/15 4:14 AM, Julian Elischer wrote: >>> On 3/1/15 10:49 AM, Harrison Grundy wrote: Thanks! That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package, since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm trying to understand the use case for this.) >>> >>> To me it would almost seem more useful to have a programmable filter >>> for which you could produce >>> parse grammars to parse the output of various programs.. >>> thus >>> >>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>> with a set of grammars in /usr/share/xmlize/ >>> then we could use it for out-of-tree programs as well if we wrote >>> grammars for them.. >>> >>> The sentiment of machine-readable output is nice, but I think it's >>> slightly off target. >>> we shouldn't have to change all out utilities, and it isn't going to >>> help at all with 3rd party apps, >>> e.g. samba stuff. A generally easy to program output grammar parser >>> would be truely useful. >>> and not just for FreeBSD. >>> >>> I've been watching with an uncomfortable feeling, but it's taken me a >>> while to put my >>> finger on what it was.. >>> >>> >> Are you sure it's not the hairs on the back of your neck standing up >> due to NIH? >> >> Juniper has been doing this for years and it's very useful for them. > I'm not saying the ability to generate machine readable output is wrong, > but that the 'unix way' would be to make a filter for it. It seems that > the noisy people don't > agree with me so I will not stand in the way of progress.. I agree. Even if someone starts with json and xml only, it will need some 3rd format soon, and adding any new format have real possibility to break all already existent (like adding json+xml breaks plain text in pipes). Moreover, it violates Unix principle 'one tool == one general function' and lots of other rules like Eric Raymond ones, making each program looks like systemd. It makes harder to merge changes from other BSDs too. Proper way to do this thing is to back out all changes and write completely separate templates-based parser - xml/json writer. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld broken on current with WITHOUT_CAPSICUM
It is broken exact at the same place even without WITHOUT_CAPSICUM. On 09.01.2015 5:23, Manfred Antar wrote: > On amd64 current build world is broken if defined WITHOUT_CAPSICUM svn > revision 276867 > here is the error: > /usr/src/usr.sbin/tcpdump/tcpdump/../../../contrib/tcpdump/tcpdump.c:80:10: > fatal error: 'libcapsicum.h' file not found > #include > ^ > 1 error generated. > make[5]: stopped in /usr/src/usr.sbin/tcpdump/tcpdump -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: CURRENT: WARNING! r273914 leaves filesystems in inconsistent/corrupted condition!
On 01.11.2014 21:26, Trond Endrestøl wrote: > What good does the file /entropy do if boot up is delayed everytime > during "Writing entropy file:"? Probably some entropy is needed before saved file is loaded. As raw guessing of alternative solution it is possible to make some part of previously saved entropy as kld module always loaded with the kernel. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient
On 13.09.2014 8:29, Peter Wemm wrote: > On Thursday, September 11, 2014 12:38:02 PM Patrick Kelsey wrote: >> On Wed, Sep 10, 2014 at 3:00 AM, Andrey Chernov wrote: >>> On 09.09.2014 21:53, Patrick Kelsey wrote: >>>> I don't think it is worth the trouble, as given the larger pattern of >>>> libc routines requiring multiple capsicum rights, it seems one will in >>>> general have to have libc implementation knowledge when using it in >>>> concert with capsicum. For example, consider the limitfd() routine in >>>> kdump.c, which provides rights for the TIOCGETA ioctl to be used on >>>> stdout so the eventual call to isatty() via printf() will work as >>> >>> intended. >>> >>>> I think the above kdump example is a good one for the subtle issues that >>>> can arise when using capsicum with libc. That call to isatty() is via a >>>> widely-used internal libc routine __smakebuf(). __smakebuf() also calls >>>> __swhatbuf(), which in turn calls _fstat(), all to make sure that output >>>> to a tty is line buffered by default. It would appear that programs >>>> that restrict rights on stdout without allowing CAP_IOCTL and CAP_FSTAT >>>> could be disabling the normally default line buffering when stdout is a >>>> tty. kdump goes the distance, but dhclient does not (restricting stdout >>>> to CAP_WRITE only). >>>> >>>> In any event, the patch attached to my first message is seeming like the >>>> way to go. >>> >>> Well, then commit it (if capsicum team agrees). >> >> Will do - thanks for the feedback. >> >> -Patrick > > Is there any possibility that this is related to the problem we've recently > hit in the freebsd.org cluster with this month's refresh? > > After running for a while: > Sep 10 02:39:44 ns2 unbound: [65258:0] notice: init module 0: validator > Sep 10 02:39:44 ns2 unbound: [65258:0] notice: init module 1: iterator > Sep 10 11:44:29 ns2 unbound: [65258:3] fatal error: event_dispatch returned > error -1, errno is Capabilities insufficient unbound itself does not use capsicum, just grep cap_, ldns too, so the problem can be somewhere else. -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient
On 09.09.2014 21:53, Patrick Kelsey wrote: > I don't think it is worth the trouble, as given the larger pattern of > libc routines requiring multiple capsicum rights, it seems one will in > general have to have libc implementation knowledge when using it in > concert with capsicum. For example, consider the limitfd() routine in > kdump.c, which provides rights for the TIOCGETA ioctl to be used on > stdout so the eventual call to isatty() via printf() will work as intended. > > I think the above kdump example is a good one for the subtle issues that > can arise when using capsicum with libc. That call to isatty() is via a > widely-used internal libc routine __smakebuf(). __smakebuf() also calls > __swhatbuf(), which in turn calls _fstat(), all to make sure that output > to a tty is line buffered by default. It would appear that programs > that restrict rights on stdout without allowing CAP_IOCTL and CAP_FSTAT > could be disabling the normally default line buffering when stdout is a > tty. kdump goes the distance, but dhclient does not (restricting stdout > to CAP_WRITE only). > > In any event, the patch attached to my first message is seeming like the > way to go. Well, then commit it (if capsicum team agrees). -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient
On 09.09.2014 1:13, Patrick Kelsey wrote: > You make a godo point about the wider use of fcntl() in libc - aside > from the rpc code, by my count there are 14 other entry points in libc > that use fcntl in their implementation. To experience breakage, > programs that use those entry points would also have to be supplying > them fds with restricted rights that do not include CAP_FCNTL. By my > count, there are currently only 12 programs in -current that call > cap_rights_limit(). I don't think these counts inform us very well as > to the presence and extent of any capsicum+libc issues similar to the > one that I've raised. Those 12 programs mentioned above would have to > be audited to determine if any of the 15 libc entry points (including > fcntl) that use fcntl are being used on those restricted fds without > being granted CAP_FCNTL rights, and whether there are overt or potential > failures occurring as a result. Consider that the failure mode in > tcpdump that I found requires that you be using multiple capture files > with size-based rotation, otherwise all works fine. Also consider that > the failure mode in dhclient only occurs when a rewritten client lease > file is smaller than its predecessor. Just to note by quick glance: tcpdump use fdopen(), so in some cases probably already broken without F_GETFL rights. openssh use fdopen(), so suspicious about F_GETFL too, but I don't traverse the order in which fdopen() and cap_rights_* there are applied. > I don't think that this read-only fcntl(F_GETFL) which doesn not modify > anything deserves any special rights at all (i.e. can be just enabled by > default in contrast to F_SETFL), but I am not capsicum expert. > > I don't think I am in a position to comment on the implications of > permanent F_GETFL rights either. I do think that the point about wider > use of fcntl(F_GETFL) in libc does argue against making a CAP_FSEEK > right in sys/capability.h, as it would appear users of capsicum and libc > are more in need of a map of capsicum rights required by libc entry > points than they are of convenience #defines. Theoretically it will be possible to get rid of fcntl(F_GETFL) in fseek(), but O_APPEND flag need to be stored somewhere in that case, and stdio _flags already have all bit occupied for 16bit short. So the price will be changing size of the main stdio structure __sFILE to add new space for flags, which is undesirable I think. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient
On 09.09.2014 0:28, Patrick Kelsey wrote: > In r268997, _ftello() was modified to use _fcntl(F_GETFL) in the > non-append, write-only path. Consequently, programs that use _ftello() > (via ftell, fgetpos, fsetpos, fseek, rewind...) on non-append, > write-only files and that use capsicum to restrict capabilities on the > associated fds to [CAP_SEEK, CAP_WRITE] broke as all ftell() (and > friends) calls on those files fail with ENOTCAPABLE due to lack of > CAP_FCNTL rights. There appear to be only two affected programs in the > tree - tcpdump and dhclient. This affects both CURRENT and 10-STABLE > (including 10.1-PRERELEASE) > > tcpdump, when configured to write to capture files rotated by size, > fails to rotate and captures indefinitely to the first file in the > series. This can be reproduced by a command such as: tcpdump -i > -C 1 -W 2 -w packets -v > > By inspection, dhclient will fail to trim old data from its client > leases file when rewriting that file with a lesser amount of data than > it currently contains. See the ftruncate() call in > dhclient.c:rewrite_client_leases(). > > The attached patch adds CAP_FCNTL to the limited rights established for > non-append, write-only files used by tcpdump and dhclient. It also > restricts the fcntl rights to CAP_FCNTL_GETFL. > > The current need to have CAP_FCNTL rights in order to get or set the > file position on non-append, write-only files is subtle. Perhaps part > of the answer is to define a CAP_FSEEK right in sys/capability.h that > resolves to CAP_SEEK|CAP_FCNTL, or to modify the CAP_SEEK description in > rights(4) to note the need for CAP_FCNTL when using ftell() and friends. > > -Patrick Stdio code use fcntl(F_GETFL) already in many places, f.e. fdopen(), freopen(). libc code in general use it in rpc code. According to your note, all that places are currently broken in anyway. I don't think that this read-only fcntl(F_GETFL) which doesn not modify anything deserves any special rights at all (i.e. can be just enabled by default in contrast to F_SETFL), but I am not capsicum expert. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Latest -current panic in uaudio_detach() / bus_dmamem_free()
On 23.06.2014 6:46, Alexander Kabaev wrote:
> Please provide us with the information on the actual audio hardware
> you are using, preferably in form of a dmesg output.
uaudio0: on
usbus3
uaudio0: No playback.
uaudio0: Record: 16000 Hz, 1 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: No MIDI sequencer.
pcm7: on uaudio0
uaudio0: No HID volume keys found.
Thanx, after backing out the patch below this panic is gone. Probably even
additional b->dmatag NULL check is needed for bus_dmamap_unload() too.
> This revision is
> your culpit:
> http://svnweb.freebsd.org/changeset/base/267581 and I have strong
> suspicion that restoring the NULL check on dmatag in the chunk below
> will cure your crash.
>
> -- Modified: head/sys/dev/sound/pcm/buffer.c
> ==
> --- head/sys/dev/sound/pcm/buffer.c Tue Jun 17 14:47:49
> 2014 (r267580) +++ head/sys/dev/sound/pcm/buffer.c Tue
> Jun 17 16:07:57 2014 (r267581) @@ -139,10 +139,9 @@
> sndbuf_free(struct snd_dbuf *b)
> if (b->buf) {
> if (b->flags & SNDBUF_F_MANAGED) {
> - if (b->dmamap)
> + if (b->buf_addr)
> bus_dmamap_unload(b->dmatag,
> b->dmamap);
> - if (b->dmatag)
> - bus_dmamem_free(b->dmatag, b->buf,
> b->dmamap);
> + bus_dmamem_free(b->dmatag, b->buf, b->dmamap);
> } else
> free(b->buf, M_DEVBUF);
> }
>
--
http://ache.vniz.net/
signature.asc
Description: OpenPGP digital signature
Latest -current panic in uaudio_detach() / bus_dmamem_free()
Always happens at shutdown after all buffers are synced, see screenshot: http://i.imgur.com/8WXTMPj.png -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: login.conf --> UTF-8
Few explanations to clarify maybe non-obvious moments:
On 05.04.2014 7:35, Andrey Chernov wrote:
>>> big hack and slowing sorting down up to 10 times.
Because our search for chains is linear because common single byte table
have no more than 2-3 chains. I don't think it worth efforts to optimize
search here, because better way to spend them is to implement
UCA:
>>> http://unicode.org/reports/tr10/
> "No code" situation doesn't mean wrong code can be committed.
Since we plan to change defaults from KOI8-R to UTF-8 ("russian" login
class), breaking sort order for non-alphabetic chars will violate POLA.
Sort order will be broken because only CP1251 is used to construct Alex
"chains" collation without merging with KOI8-R table.
Merging KOI8-R collation is absolute minimum, but proper hack will be
merging CP866 and ISO8859-5 too, as I already mention.
--
http://ache.vniz.net/
signature.asc
Description: OpenPGP digital signature
Re: login.conf --> UTF-8
On 05.04.2014 5:35, Andrey Chernov wrote: > Even his "version 2" have my objections. I already reply Alex about this > in 2008. In short: > 1) It is error there: almost all single chars above ASCII should be > "chains", i.t. two bytes minimum ... I check my whole correspondence with Alexey and withdraw objection #1 which was related to the \x80;...;\xff line in his table. While they are illegal sequences in UTF-8, our colldef(1) wants all single byte characters mapped. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: login.conf --> UTF-8
On 05.04.2014 6:39, Sean Bruno wrote: > On Sat, 2014-04-05 at 05:35 +0400, Andrey Chernov wrote: >> On 04.04.2014 16:46, Gleb Smirnoff wrote: >>> On Thu, Apr 03, 2014 at 01:34:33AM +0400, Andrey Chernov wrote: >>> A> On 02.04.2014 21:15, Gleb Smirnoff wrote: >>> A> > S> + :lang=en_US.UTF-8:\ >>> A> > S> + :charset=UTF-8: >>> A> > >>> A> > And I'd like to do same change for the 'russian' login class >>> A> > in /etc/login.conf. >>> A> >>> A> Please everybody remember that we don't have UTF-8 collation >>> A> implemented, just fallback to bytecode comparison. >>> >>> Any objections on checking in FreeBSD-compatible[1] UTF-8 collation >>> implementation from Alex Tutubalin? >>> >>> http://blog.lexa.ru/2008/03/03/freebsd_utf8_russian_collate_vtoraja_popitka.html >>> >> >> Even his "version 2" have my objections. I already reply Alex about this >> in 2008. In short: >> 1) It is error there: almost all single chars above ASCII should be >> "chains", i.t. two bytes minimum, since there almost no intersections >> with ISO8859-1 as UTF-8 subset. >> 2) The table itself is very incomplete, f.e. not covering either whole >> KOI8-R, nor ISO8859-5, nor CP866. It is made from CP1251 with all its >> restrictions. So, switching from f.e. KOI8-R to UTF-8 will cause sorting >> regression. Russian UTF-8 collation should be able to sort all major >> Russian charsets mentioned, i.e. we need combined table. >> 3) "charmap map.ISO8859-1" declaration is missing (needed mainly for >> using pure ASCII chars mnemonic names). >> >> Even in case above mentioned errors will be removed and the code will be >> committed afterwards, we should understand that this way (implementing >> multibyte collation via single byte one) even while being possible is a >> big hack and slowing sorting down up to 10 times. >> >> Proper "Unicode collation algorithm" is already implemented by ICU and >> other projects. See >> http://unicode.org/reports/tr10/ >> It will be better if someone adopt it instead of hacks. >> > > > If you have a different patch, I'd appreciate seeing it. I don't have a different patch. In case you have enough time to fix above mentioned obstacles, I can review yours (or somebody else's) one. "No code" situation doesn't mean wrong code can be committed. Do it properly even when it is a hack. -- http://ache.vniz.net/ signature.asc Description: OpenPGP digital signature
Re: login.conf --> UTF-8
On 04.04.2014 16:46, Gleb Smirnoff wrote: > On Thu, Apr 03, 2014 at 01:34:33AM +0400, Andrey Chernov wrote: > A> On 02.04.2014 21:15, Gleb Smirnoff wrote: > A> > S> + :lang=en_US.UTF-8:\ > A> > S> + :charset=UTF-8: > A> > > A> > And I'd like to do same change for the 'russian' login class > A> > in /etc/login.conf. > A> > A> Please everybody remember that we don't have UTF-8 collation > A> implemented, just fallback to bytecode comparison. > > Any objections on checking in FreeBSD-compatible[1] UTF-8 collation > implementation from Alex Tutubalin? > > http://blog.lexa.ru/2008/03/03/freebsd_utf8_russian_collate_vtoraja_popitka.html > Even his "version 2" have my objections. I already reply Alex about this in 2008. In short: 1) It is error there: almost all single chars above ASCII should be "chains", i.t. two bytes minimum, since there almost no intersections with ISO8859-1 as UTF-8 subset. 2) The table itself is very incomplete, f.e. not covering either whole KOI8-R, nor ISO8859-5, nor CP866. It is made from CP1251 with all its restrictions. So, switching from f.e. KOI8-R to UTF-8 will cause sorting regression. Russian UTF-8 collation should be able to sort all major Russian charsets mentioned, i.e. we need combined table. 3) "charmap map.ISO8859-1" declaration is missing (needed mainly for using pure ASCII chars mnemonic names). Even in case above mentioned errors will be removed and the code will be committed afterwards, we should understand that this way (implementing multibyte collation via single byte one) even while being possible is a big hack and slowing sorting down up to 10 times. Proper "Unicode collation algorithm" is already implemented by ICU and other projects. See http://unicode.org/reports/tr10/ It will be better if someone adopt it instead of hacks. -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
