RE: [HEADSUP] Upcoming GNU sort removal
Currently the sort structures in the sort utility use 32-bits counters. With Tb files, this may be the limitation point. I can take a look into fixing that. Oleg From: Dennis Glatting [mailto:free...@penx.com] Sent: Thursday, October 04, 2012 8:09 PM To: Gabor Kovesdan Cc: curr...@freebsd.org; Oleg Moskalenko Subject: Re: [HEADSUP] Upcoming GNU sort removal On Thu, 2012-10-04 at 18:16 +0200, Gabor Kovesdan wrote: Em 04-10-2012 16:50, Dennis Glatting escreveu: On Thu, 2012-10-04 at 12:53 +0200, Gabor Kovesdan wrote: Hi, it has been more than 3 months ago that BSD sort became default in HEAD and no serious complaints have been raised against it since then so I plan to permanently remove GNU sort from head in the next days. If you have any objection, please raise it now. Initially I had problems with multi TB files (--unique, five to ten files) but I haven't had to do that in two(?) months. I will be getting back to that project in a month or so. It challanges a system's resources. :) And did it go much better with base GNU sort? It's quite an extreme case... :) Multi GB is also rare not speaking about multi TB... Yes. However my problem now is ZFS stability -- typically locking up, case example today: last pid: 67998; load averages: 0.00, 0.00, 0.00up 1+19:50:51 19:02:10 80 processes: 1 running, 79 sleeping CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 146M Active, 2765M Inact, 35G Wired, 371M Buf, 86G Free ARC: 32G Total, 4141M MRU, 27G MFU, 55M Anon, 485M Header, 614M Other Swap: 233G Total, 233G Free PID USERNAMETHR PRI NICE SIZERES STATE C TIME WCPU COMMAND 17517 root 17 424 217M 128M tx-tx 21 25.3H 0.00% pbzip2 17568 root 17 524 201M 116M tx-tx 24 25.2H 0.00% pbzip2 17508 root 17 464 201M 116M tx-tx 33 24.6H 0.00% pbzip2 17544 root 17 524 205M 120M tx-tx 37 24.6H 0.00% pbzip2 17532 root 17 524 209M 123M tx-tx 35 24.5H 0.00% pbzip2 etc. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: openssl upgrade, libcrypto, libssl confusion
Anton, I use mostly static openssl libraries, but I suppose it applies to the dynamic ones, too: with any OpenSSL version (including 1.0.1c), you need libcrypto and libssl. With the new OpenSSL version, you need the new library versions, and you must recompile your binary code to use the new dynamic libraries (because there probably some headers incompatibilities between the versions). You just misunderstood the release notes. Do not delete the library, just re-install the new version. Regards, Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Anton Shterenlikht Sent: Wednesday, July 25, 2012 4:30 AM To: freebsd-current@freebsd.org Subject: openssl upgrade, libcrypto, libssl confusion In /usr/src/UPDATING I see 20120712: The OpenSSL has been upgraded to 1.0.1c. Any binaries requiring libcrypto.so.6 or libssl.so.6 must be recompiled. Also, there are configuration changes. Make sure to merge /etc/ssl/openssl.cnf. Looking at this: # make -C /usr/src check-old-libs Checking for old libraries /lib/libcrypto.so.6 /usr/lib/libssl.so.6 # Am I correct that these 2 libraries can be safely deleted? However, I can't see any version 7 of these libs. Have these libs been replaced by some other lib? Finally, I've rebuilt mail/fetchmail and mail/mutt already several times, but the binaries still use these 2 libs: TZAV ldd /usr/local/bin/mutt /usr/local/bin/mutt: libncursesw.so.8 = /lib/libncursesw.so.8 (0x1202f6000) libgssapi.so.10 = /usr/lib/libgssapi.so.10 (0x1203aa000) libheimntlm.so.11 = /usr/lib/libheimntlm.so.11 (0x1203ca000) libkrb5.so.11 = /usr/lib/libkrb5.so.11 (0x1203e4000) libhx509.so.11 = /usr/lib/libhx509.so.11 (0x1204c6000) libcom_err.so.5 = /usr/lib/libcom_err.so.5 (0x12055) libcrypto.so.6 = /lib/libcrypto.so.6 (0x120562000) ^^^ libasn1.so.11 = /usr/lib/libasn1.so.11 (0x120812000) libwind.so.11 = /usr/lib/libwind.so.11 (0x120918000) libheimbase.so.11 = /usr/lib/libheimbase.so.11 (0x120952000) libroken.so.11 = /usr/lib/libroken.so.11 (0x120968000) libcrypt.so.5 = /lib/libcrypt.so.5 (0x120996000) libssl.so.6 = /usr/lib/libssl.so.6 (0x1209d) ^^^ libz.so.6 = /lib/libz.so.6 (0x120a72000) libintl.so.9 = /usr/local/lib/libintl.so.9 (0x120aaa000) libthr.so.3 = /lib/libthr.so.3 (0x120acc000) libc.so.7 = /lib/libc.so.7 (0x120b1a000) libiconv.so.3 = /usr/local/lib/libiconv.so.3 (0x120dca000) TZAV TZAV ldd /usr/local/bin/fetchmail /usr/local/bin/fetchmail: libopie.so.7 = /usr/lib/libopie.so.7 (0x12010) libcrypt.so.5 = /lib/libcrypt.so.5 (0x12011e000) libkvm.so.5 = /lib/libkvm.so.5 (0x120158000) libcom_err.so.5 = /usr/lib/libcom_err.so.5 (0x120178000) libssl.so.6 = /usr/lib/libssl.so.6 (0x12018a000) ^^^ libcrypto.so.6 = /lib/libcrypto.so.6 (0x12022c000) ^^^ libc.so.7 = /lib/libc.so.7 (0x1204dc000) libmd.so.6 = /lib/libmd.so.6 (0x12078c000) TZAV Or will the new library (what is it?) will not be used unless I delete the old ones? Thanks -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 331 5944 Fax: +44 (0)117 929 4423 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
I am going to post the ministat results of my tests: 1) Text sort of 167-Mb file (1,000,000 random lines, each contains several fields, each field is either a floating point number or a binary string with random symbols between 0 and 255). Sorted on second field (-k 2,2 option): x OGNU + NGNU * NBSD (multi-threaded) +--+ | x* + | | x x * +++ | |x x * ** * +++* * **| | |_A_| |___M_|AA___| | +--+ N Min MaxMedian AvgStddev x 10 5.758 5.937 5.8485.8508 0.057276716 + 10 6.29 6.366 6.3326.33020.02123833 Difference at 95.0% confidence 0.4794 +/- 0.0405862 8.19375% +/- 0.693687% (Student's t, pooled s = 0.0431954) * 10 6.067 7.228 6.2256.34490.35979422 Difference at 95.0% confidence 0.4941 +/- 0.242055 8.445% +/- 4.13713% (Student's t, pooled s = 0.257616) == 2) Same file, numeric sort on the same field (-k 2,2 -n option): x OGNU.n + NGNU.n * NBSD.n (multi-threaded) +--+ | * x x x +++| | * xx x x | | |MA_| |__A_| |A | +--+ N Min MaxMedian AvgStddev x 10 7.142 7.338 7.2167.2179 0.066727389 + 10 8.231 8.307 8.2718.2677 0.022701199 Difference at 95.0% confidence 1.0498 +/- 0.0468287 14.5444% +/- 0.648785% (Student's t, pooled s = 0.0498392) * 10 6.91 7.094 6.986.9864 0.061449528 Difference at 95.0% confidence -0.2315 +/- 0.0602683 -3.2073% +/- 0.834983% (Student's t, pooled s = 0.0641428) = On these two tests, all three program produced the same output. So, on text sort, NBSD is slightly slower than GNU; on simple numeric sort, NBSD is slightly faster. I did not use ministat for complex numeric sort (-g) because the performance difference is huge (in favor of NBSD) and it would make no sense. Thanks Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Oleg Moskalenko Sent: Wednesday, June 27, 2012 6:45 PM To: FreeBSD Current Subject: RE: [HEADS-UP] BSD sort is the default sort in -CURRENT Hi As promised, I am supplying an example of comparison between several sort programs. The test file is a randomly generated 1,000,000 lines, each line contain a single floating point number. We are going to sort it three ways - as text, as -n numeric sort, and as -g numeric sort, with 4 programs: 1) Old BSD/GNU sort 5.3.0 2) New GNU sort 8.15 3) New BSD sort, single threaded 4) New BSD sort, multi-threaded The system is a 3-CPUs system, 1.5Gb of RAM, FreeBSD version 8.2. All times are in seconds. Locale C. == TEXT SORT sys user real Old BSD/GNU sort:0.0 1.692 2.008 New GNU sort:0.0 2.279 1.605 New BSD sort, st:0.0 1.964 2.300 New BSD sort, mt:0.0 2.385 1.897 == NUMERIC SORT -n sys user real Old BSD/GNU sort:0.0 4.357 4.674 New GNU sort:0.0 8.839 5.134 New BSD sort, st:0.0 5.308 5.592 New BSD sort, mt:0.0 4.581 2.489 == NUMERIC SORT -g sys user real Old BSD/GNU sort:0.0 45.37845.630 New GNU sort: ~450~121 ~300 New BSD sort, st:0.334.334 5.992 New BSD sort, mt:11.140 4.624 8.983 === Thanks Oleg ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
Now I am going to provide the second informational part about the new BSD sort: the differences between the new BSD sort and the GNU sort programs. Below: NBSD - new BSD sort; OGNU - old BSD/GNU sort, version 5.3.0; NGNU - new GNU sort, version 8.15. There are several areas: 1) -k option (sort fields/keys), with single byte locales. The -k option functionality is described in POSIX standard with deceitfully simple wording. Unfortunately, the real meaning is rather complex, and sometimes it defies the common sense. For example, a sort key can spread across boundaries with subsequent sort keys. A common-sense-based implementation would fail the POSIX standard. The OGNU implementation fails to follow the standard. In our tests, there are thousands various use cases when it does not work properly. The OGNU exact algorithm and the failure pattern is a mystery for us. The NGNU implements the POSIX algorithm properly, and the NBSD is compatible with NGNU when using -k in single-byte locales. 2) -k option (sort fields/keys), with multi-byte locales. Here the situation is reverse. During the -k calculation, NGNU basically ignores the symbol sizes (but it does not ignore the symbol sizes when it compares the fields). The result is that the sort keys are not calculated correctly by the NGNU. OGNU, on the other hand, seems to use the correct symbol sizes, but it is still using incorrect algorithm. So, the overall picture is: - OGNU uses wrong algorithm, but it uses correct symbol sizes; - NGNU uses correct algorithm, but it uses wrong symbol sizes; - NBSD uses NGNU-compatible algorithm with correct symbol sizes. So, for big majority of users who are using only single-byte locales, the NBSD sort key calculation is compatible with NGNU. For the multi-byte locales (like zh_HK.Big5HKSCS), we believe that only NBSD provides the correct results with -k option. 3) NGNU does not allow options -d and/or -i together with numeric sorting. OGNU and NBSD allows that. 4) NBSD adds NGNU-compatible options, which are absent in OGNU: -C (silent version of -c) -h (human numeric sort) -R (random sort) -V (version sort) --batch-size --compress-program --random-source --debug --files0-from 5) NBSD adds several of its own new proprietary options: --mergesort --qsort --heapsort --radixsort --nthreads=... (multi-threaded build only) 6) NBSD does not support option --parallel of NGNU. It has roughly the same meaning as --nthreads in NBSD. This difference will be fixed soon. Thanks Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Oleg Moskalenko Sent: Wednesday, June 27, 2012 6:45 PM To: FreeBSD Current Subject: RE: [HEADS-UP] BSD sort is the default sort in -CURRENT Hi As promised, I am supplying an example of comparison between several sort programs. The test file is a randomly generated 1,000,000 lines, each line contain a single floating point number. We are going to sort it three ways - as text, as -n numeric sort, and as -g numeric sort, with 4 programs: 1) Old BSD/GNU sort 5.3.0 2) New GNU sort 8.15 3) New BSD sort, single threaded 4) New BSD sort, multi-threaded The system is a 3-CPUs system, 1.5Gb of RAM, FreeBSD version 8.2. All times are in seconds. Locale C. == TEXT SORT sys user real Old BSD/GNU sort:0.0 1.692 2.008 New GNU sort:0.0 2.279 1.605 New BSD sort, st:0.0 1.964 2.300 New BSD sort, mt:0.0 2.385 1.897 == NUMERIC SORT -n sys user real Old BSD/GNU sort:0.0 4.357 4.674 New GNU sort:0.0 8.839 5.134 New BSD sort, st:0.0 5.308 5.592 New BSD sort, mt:0.0 4.581 2.489 == NUMERIC SORT -g sys user real Old BSD/GNU sort:0.0 45.37845.630 New GNU sort: ~450~121 ~300 New BSD sort, st:0.334.334 5.992 New BSD sort, mt:11.140 4.624 8.983 === Thanks Oleg ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
Doug, --nthreads option corresponds to --parallel option of NGNU and it will be renamed. The other four proprietary options will be marked as non-portable in the man page. After nthreads==parallel renaming, NBSD will support all NGNU options. Thank you for the suggestion. Oleg -Original Message- From: Doug Barton [mailto:do...@freebsd.org] Sent: Friday, June 29, 2012 2:02 PM To: Oleg Moskalenko Cc: FreeBSD Current Subject: Re: [HEADS-UP] BSD sort is the default sort in -CURRENT On 06/29/2012 01:50 PM, Oleg Moskalenko wrote: 5) NBSD adds several of its own new proprietary options: --mergesort --qsort --heapsort --radixsort --nthreads=... (multi-threaded build only) Oleg, First, thank you very much for providing both the performance numbers, and the breakdown in the differences in command line options. Everything looks great, my only concern is the above. Are there similar/identical options in NGNU that correspond to the options above? If so, I would be hesitant to add new names for them because it hurts portability between platforms. If these are totally new features then my assumption is that you have clearly marked them as non-portable in the man page? Once again, I really appreciate you addressing my concerns, and your hard work on this project. Doug ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
-Original Message- From: Doug Barton [mailto:do...@freebsd.org] Sent: Tuesday, June 26, 2012 11:18 PM To: Gabor Kovesdan Cc: FreeBSD Current; Oleg Moskalenko Subject: Re: [HEADS-UP] BSD sort is the default sort in -CURRENT On 06/26/2012 11:04 PM, Gabor Kovesdan wrote: Hi Folks, as I announced before, the default sort in -CURRENT has been changed to BSD sort. Has this been performance tested vs. the old one? If so, where are the results? Of course it was performance-tested. As this is a totally different program with different algorithms, it has totally different performance profile on different tests, comparing to the old sort. In the default compilation mode (single thread sort) the performance is on the same level as the old sort (sometimes faster, sometimes slower). The new sort is often significantly faster in numeric sort tests. In experimental multi-threading mode, the new sort is much faster than the old sort on multi-CPU systems. The sort speed comparison is not actually fair because the old sort cuts some corners and has a number of bugs. The concrete figures do not have much sense because you change the sort file and you get a totally different performance ratio. Since the import, the reported minor bugs have been fixed and BSD sort has passed the portbuild test. If you encounter any problems or incompatibility with the old GNU version, please report. Has this been thoroughly regression-tested against the old version, option by option? Of course we have the regression tests. We have an overnight test that runs through probably 17 millions various sort option combinations. But we actually had to compare the new sort against a fresh GNU sort implementation (version 8.15), because the old BSD GNU sort is very buggy and testing against the old GNU sort has no sense. Oleg ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
Doug, I'll post some performance figures, probably tomorrow. But I do not agree with you that we have to reproduce the old sort bugs. It makes no sense and I am not going to do that. Absolutely not. If some old scripts are relying on buggy behavior (and I hope they are not) then the old scripts must be fixed. Period. The system cannot grow replicating the old bugs. All system scripts that I've seen are using pretty basic sort features. In the basic area, the old sort and the new sort are 100% compatible. The incompatibilities are in more complex areas (numeric sorts and unusual key-based sorts). I am actually tested the new sort against the old GNU sort. There are some incompatibilities. All of them are due to the bugs of the old GNU sort. The new BSD sort program is compatible with the new GNU sort, a much cleaner program than the old GNU sort. Try to install the new GNU coreutils. If the scripts can work with the new GNU sort (version 8.15 and later) than they will work with the new BSD sort. There is a POSIX standard, and the program must be compatible with the POSIX standard. Take care, Oleg -Original Message- From: Doug Barton [mailto:do...@freebsd.org] Sent: Wednesday, June 27, 2012 1:35 AM To: Oleg Moskalenko Cc: Gabor Kovesdan; FreeBSD Current Subject: Re: [HEADS-UP] BSD sort is the default sort in -CURRENT On 06/26/2012 11:48 PM, Oleg Moskalenko wrote: -Original Message- From: Doug Barton [mailto:do...@freebsd.org] Sent: Tuesday, June 26, 2012 11:18 PM To: Gabor Kovesdan Cc: FreeBSD Current; Oleg Moskalenko Subject: Re: [HEADS-UP] BSD sort is the default sort in -CURRENT On 06/26/2012 11:04 PM, Gabor Kovesdan wrote: Hi Folks, as I announced before, the default sort in -CURRENT has been changed to BSD sort. Has this been performance tested vs. the old one? If so, where are the results? Of course it was performance-tested. Great, can you post the results somewhere? I understand what you're saying below that there are situations where worse performance may need explanation, but it would be helpful if we had the data to look at. As this is a totally different program with different algorithms, it has totally different performance profile on different tests, comparing to the old sort. In the default compilation mode (single thread sort) the performance is on the same level as the old sort (sometimes faster, sometimes slower). The new sort is often significantly faster in numeric sort tests. In experimental multi-threading mode, the new sort is much faster than the old sort on multi-CPU systems. This sounds encouraging. Is there a knob to enable the threaded build? The sort speed comparison is not actually fair because the old sort cuts some corners and has a number of bugs. Understood, but the existing sort is what we're changing away from, so that's what we have to test against. What we don't want is a situation where we are switching to the new sort by default without understanding what the tradeoffs are. (IOW, we don't want a repeat of the situation with grep.) The concrete figures do not have much sense because you change the sort file and you get a totally different performance ratio. I'm assuming that you'd run the performance tests on various different input files, and report the different results. Has this been thoroughly regression-tested against the old version, option by option? Of course we have the regression tests. We have an overnight test that runs through probably 17 millions various sort option combinations. This sounds great, but ... But we actually had to compare the new sort against a fresh GNU sort implementation (version 8.15), because the old BSD GNU sort is very buggy and testing against the old GNU sort has no sense. ... this not so much. The existing sort is what people have now, and what they rely on, particularly for scripts. Comparing apples to oranges doesn't help us understand how things are going to be different with the new version. Doug ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
-Original Message- But I do not agree with you that we have to reproduce the old sort bugs. It makes no sense and I am not going to do that. Absolutely not. That isn't what I said. What I asked is for you to *test* the existing sort vs. the new one, and to report where the behavior is different. That's a very basic part of any sort of replace a core utility project such as this one. The problem is that the sort program has huge number of possible options combination. I can list some, but I cannot promise to catch all of them. It would be enormous work. If some old scripts are relying on buggy behavior (and I hope they are not) then the old scripts must be fixed. Period. With respect, that's not your decision (or mine for that matter). We first need the data, then as a project we decide how many old bugs we want to be compatible with, if any. This is an incorrect approach. You never want old bugs we want to be compatible with in a clean POSIX-compliant system. The system cannot grow replicating the old bugs. And the project cannot grow if we lose users due to gratuitous differences in core utilities. There are users that we are loosing because the utilities do not work as expected. For example, a common complain is about a situation like that: try run a trivial command like $ ls -l /usr/bin | env LANG=en_US.UTF-8 sort -n -k 5 and see what it yields for the old BSD/GNU sort. I suspect that when you are talking about the old sort compatibility you are really do not know what you are talking about. Once you start digging, you prospective may change. All system scripts that I've seen are using pretty basic sort features. The system scripts are only a tiny fraction of how FreeBSD users use sort. This is even stronger emphasizes the need in a standard-compliant implementation. In the basic area, the old sort and the new sort are 100% compatible. The incompatibilities are in more complex areas (numeric sorts and unusual key-based sorts). So here's one to add to your regression test. I use the following to sort IPv4 addresses in a list: sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 When used with GNU sort that will sort a list of IPv4 addresses into a humanly-recognizable numeric order. Please ensure that this works the same way with the new sort. First, this is a pretty trivial use case. Don't expect anything different in the trivial cases. I think that 99% of users will never see the difference between the old sort and the new sort - for a usual non-expert usage the two are almost always compatible. Second, do you really think that I need lecturing which use cases to test ? I am actually tested the new sort against the old GNU sort. There are some incompatibilities. All of them are due to the bugs of the old GNU sort. Please list all of those explicitly. see above. The new BSD sort program is compatible with the new GNU sort, a much cleaner program than the old GNU sort. That's good, but not really relevant to the users of what we have in the base now. I bet many of them are installing the new GNU coreutils exactly for the reasons of better performance and compatibility. I realize that these questions may seem discouraging, but they need to be answered. It would have been nice if Gabor had posted a we think we're ready to make the new sort the default, any last concerns? message, but deal with where we are at and move forward. He actually did. You probably missed the messages. Thanks, Oleg ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
Marcus, I'll provide some incompatibilities description, as many as I can do. Thanks Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Marcus von Appen Sent: Wednesday, June 27, 2012 3:40 AM To: freebsd-current@freebsd.org Subject: Re: [HEADS-UP] BSD sort is the default sort in -CURRENT Daniel Gerzo dan...@freebsd.org: On 27.06.2012 10:43, Doug Barton wrote: On 06/27/2012 02:09 AM, Oleg Moskalenko wrote: Doug, I'll post some performance figures, probably tomorrow. That's great, thanks. But I do not agree with you that we have to reproduce the old sort bugs. It makes no sense and I am not going to do that. Absolutely not. That isn't what I said. What I asked is for you to *test* the existing sort vs. the new one, and to report where the behavior is different. That's a very basic part of any sort of replace a core utility project such as this one. [ snip ] Doug, are you implying that if we were about to import a new version of GNU sort, you would be asking for the same data? I believe we do not make this kind of work with any vendor code that is being updated in the base; I do not really understand why should Oleg or anyone else do this work when the bsdsort is compatible with a recent version of GNU sort. Seconded for -CURRENT. I think, we should at least provide some brief document, whatsoever on incompatibilities with the sort implementation that is currently active in RELENG_9, no matter how buggy it is. This allows adopters and people, who have to migrate their production systems to identify and quantify the areas to change and perform some risk management. This also allows them to move more quickly to the new release, since they can start with the necessary changes earlier and plan ahead. We provide such changes usually in the release notes for various tools, we updated and I think that giving out such a document earlier will be extremely benefitial for companies, which have to deal with more than one or two servers running FreeBSD, especially if we know that the currently shipped implementation is buggy and people most likely will have their own workarounds for that. Cheers Marcus ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
Olli, Thanks for the feedback. We are working on some minor improvements and fixes, when we are done we will commit them to the ports and to the base system. If you see any problems and inconsistencies, please let us know ASAP so we can fix that. Regards, Oleg -Original Message- From: olli hauer [mailto:oha...@gmx.de] Sent: Wednesday, June 27, 2012 10:56 AM To: FreeBSD Current Cc: Gabor Kovesdan; Oleg Moskalenko Subject: Re: [HEADS-UP] BSD sort is the default sort in -CURRENT On 2012-06-27 08:04, Gabor Kovesdan wrote: Hi Folks, as I announced before, the default sort in -CURRENT has been changed to BSD sort. Since the import, the reported minor bugs have been fixed and BSD sort has passed the portbuild test. If you encounter any problems or incompatibility with the old GNU version, please report. Gabor Thanks, I'm running textproc/bsdsort now a view weeks as BASE replacement on 8.3 and haven't seen any issues even on files with a view GB. Are you planing to update the port with your latest version? -- Regards, olli ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort is the default sort in -CURRENT
Hi As promised, I am supplying an example of comparison between several sort programs. The test file is a randomly generated 1,000,000 lines, each line contain a single floating point number. We are going to sort it three ways - as text, as -n numeric sort, and as -g numeric sort, with 4 programs: 1) Old BSD/GNU sort 5.3.0 2) New GNU sort 8.15 3) New BSD sort, single threaded 4) New BSD sort, multi-threaded The system is a 3-CPUs system, 1.5Gb of RAM, FreeBSD version 8.2. All times are in seconds. Locale C. == TEXT SORT sys user real Old BSD/GNU sort:0.0 1.692 2.008 New GNU sort:0.0 2.279 1.605 New BSD sort, st:0.0 1.964 2.300 New BSD sort, mt:0.0 2.385 1.897 == NUMERIC SORT -n sys user real Old BSD/GNU sort:0.0 4.357 4.674 New GNU sort:0.0 8.839 5.134 New BSD sort, st:0.0 5.308 5.592 New BSD sort, mt:0.0 4.581 2.489 == NUMERIC SORT -g sys user real Old BSD/GNU sort:0.0 45.37845.630 New GNU sort: ~450~121 ~300 New BSD sort, st:0.334.334 5.992 New BSD sort, mt:11.140 4.624 8.983 === Thanks Oleg ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: CURRENT: buildworld fails
So, the newer sort works fine, and the older sort does not work ? Did I get your correctly ? Thanks Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Sent: Monday, May 28, 2012 8:45 PM To: freebsd-current@freebsd.org Subject: CURRENT: buildworld fails I had same problem, but I resolved it. Maybe, your sort (/usr/bin/sort) is broken. cd /usr/src make update cd usr.bin/sort make obj depend all install Then, sort is changed. make cleandir cleandir make buildworld ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: FreeBSD 10 prognostication...
Modern large-scale virtualization technologies are based upon bare-metal versions of VMWare and XenServer. They are not Linux and they are not FreeBSD - the Hypervisors are a specialized breed of OSes (albeit, the hypervisor manager is usually a UNIX-like OS). Any conventional OS (Linux, FreeBSD) can only do their best to be a civilized guest OS. Linux or FreeBSD cannot be a server platform for real enterprise virtualization - the hypervisors have won that place. This is not a contest point between Linux and FreeBSD. The jail system is not helpful for enterprise virtualization, either. Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Dag-Erling Smørgrav Sent: Monday, May 21, 2012 1:58 AM To: Jamie Cc: Vincent Hoffman; Vance Siemens; Rick Macklem; freebsd- c...@freebsd.org; freebsd-current@freebsd.org Subject: Re: FreeBSD 10 prognostication... Jamie ja...@geniegate.com writes: Jails are usually more suited to cloud work than KVM or the latest OpenVZ/Containers/??? of the linux world ever will be... No, they're not. VMWare, RHEV (KVM-based) etc. provide features such as seamless migration of virtual machines from one physical machine to another, automatic restart on a different physical server if one fails etc. that simply aren't possible with jails; and there are certain things you still can't run reliably / safely in jails - anything that relies on SysV IPC, for instance, such as PostgreSQL. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: FreeBSD 10 prognostication...
-Original Message- From: Dag-Erling Smørgrav [mailto:d...@des.no] Sent: Monday, May 21, 2012 9:48 AM To: Oleg Moskalenko Cc: Jamie; Vincent Hoffman; Vance Siemens; Rick Macklem; freebsd- c...@freebsd.org; freebsd-current@freebsd.org Subject: Re: FreeBSD 10 prognostication... Oleg Moskalenko oleg.moskale...@citrix.com writes: Modern large-scale virtualization technologies are based upon bare-metal versions of VMWare and XenServer. They are not Linux and they are not FreeBSD AFAIK, RHEV is KVM on top of RHEL This is true. But KVMs share is still very small, around 2%, so its too early to say about the technology. KVM approach is to blend the hyperviser and the Linux kernel in one single thing. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: FreeBSD 10 prognostication...
Its not that bad. PCBSD (with FreeBSD 9.0 inside) has a relatively decent GUI setup, with several viable GUI choices. Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd-curr...@freebsd.org] On Behalf Of Hamell, Rick (SPARQ) Sent: Thursday, May 17, 2012 8:23 PM To: Vance Siemens Cc: Dag-Erling Smørgrav; freebsd-current@freebsd.org; freebsd-c...@freebsd.org; Vincent Hoffman Subject: Re: FreeBSD 10 prognostication... What, 8bit color ANSI isn't GUI enough? But seriously, it feels like it works even worse then it did a decade ago. Rick Hamell Sent from my iPhone On May 17, 2012, at 4:20 PM, Vance Siemens vance.siem...@gmail.com wrote: Eh, sorry. I got excited at the prospect of downloading FreeBSD from the App Store and having the installer just work in a modern GUI. You have to admit, FreeBSD is lacking in this area. It would be a boon. On Wed, May 16, 2012 at 7:18 AM, Dag-Erling Smørgrav d...@des.no wrote: Vance Siemens vance.siem...@gmail.com writes: Can you share a brief overview of what's wrong with it? Umm, it's about as factual as The Onion, except not as funny. FreeBSD never had to jettison two thirds of its code base and start from scratch. Apple is not involved in FreeBSD development. No Mac OS X or Darwin version includes FreeBSD. FreeBSD and Mac OS X will never merge. FreeBSD was never acquired by WinDriver Systems or by anyone else, although a company named WindRiver Systems (makers of the embedded operating system VxWorks, not of Windows video drivers) did at one point acquire BSDI, which had previously acquired Walnut Creek CD-ROM, which was heavily involved in the early history of both FreeBSD and Slackware Linux. The remains of Walnut Creek CD-ROM and BSDI are now known as FreeBSD Mall and iXsystems (of PC-BSD and FreeNAS fame). DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-c...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-chat To unsubscribe, send any mail to freebsd-chat-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [clang] r234928 amd64 buildworld error
We already have a fix for this problem with clang, and we are going to submit it soon. gcc behaves differently on the same sources, they can be compiled just fine with gcc. Thanks Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Anton Shterenlikht Sent: Sunday, May 13, 2012 11:54 PM To: freebsd-current@freebsd.org Subject: [clang] r234928 amd64 buildworld error clang -O2 -pipe -DSORT_THREADS -std=gnu99 -Qunused-arguments -fstack- protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict -prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast- qual -Wwri te-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar- subscripts - Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno- pointer-s ign -Wno-empty-body -Wno-string-plus-int -c /usr/src/usr.bin/sort/file.c /usr/src/usr.bin/sort/file.c:601:12: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(7)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n)catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:942:11: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(8)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n)catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:1279:10: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(9)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n)catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:1295:12: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] errx(2, getstr(10)); ^~ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n)catgets(catalog, 1, n, nlsstr[n]) ^ 4 errors generated. *** [file.o] Error code 1 Stop in /usr/src/usr.bin/sort. *** [all] Error code 1 -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 331 5944 Fax: +44 (0)117 929 4423 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: FYI FreeBSD clang build fails on new import of sort
I found another problem with sort compilation with clang, I'll send the fix soon. Thanks Oleg -Original Message- From: Gábor Kövesdán [mailto:ga...@t-hosting.hu] Sent: Monday, May 14, 2012 4:24 AM To: Garrett Cooper Cc: Outback Dingo; freebsd-current; Oleg Moskalenko Subject: Re: FYI FreeBSD clang build fails on new import of sort On 2012.05.14. 2:49, Garrett Cooper wrote: Yeah... errx(2, getstr(9)) should be errx(2, %s, getstr(9))... -Garrett Thanks, should be fixed now. Gabor ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: FYI FreeBSD clang build fails on new import of sort
Thank you for the error report, we are going to fix it ASAP. Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Garrett Cooper Sent: Sunday, May 13, 2012 5:49 PM To: Outback Dingo Cc: freebsd-current Subject: Re: FYI FreeBSD clang build fails on new import of sort On Sun, May 13, 2012 at 5:02 PM, Outback Dingo outbackdi...@gmail.com wrote: trying to rerun a clang build of FreeBSD CURRENT fails on new import of sort, cat /etc/src.conf WITH_CLANG_IS_CC=1 make world -SNIP--- clang -O2 -pipe -DSORT_THREADS -std=gnu99 -Qunused-arguments -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -c /usr/src/usr.bin/sort/coll.c clang -O2 -pipe -DSORT_THREADS -std=gnu99 -Qunused-arguments -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -c /usr/src/usr.bin/sort/file.c /usr/src/usr.bin/sort/file.c:601:12: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(7)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n) catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:942:11: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(8)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n) catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:1279:10: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(9)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n) catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:1295:12: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] errx(2, getstr(10)); ^~ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n) catgets(catalog, 1, n, nlsstr[n]) ^ 4 errors generated. *** [file.o] Error code 1 Stop in /usr/src/usr.bin/sort. Yeah... errx(2, getstr(9)) should be errx(2, %s, getstr(9))... -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: FYI FreeBSD clang build fails on new import of sort
Obviously, the option -Wall implies -Wformat-security in clang. The compiler that we used for the development does not turns on -Wformat-security with -Wall. It is an easy fix, we will submit it soon. Thanks Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Garrett Cooper Sent: Sunday, May 13, 2012 5:49 PM To: Outback Dingo Cc: freebsd-current Subject: Re: FYI FreeBSD clang build fails on new import of sort On Sun, May 13, 2012 at 5:02 PM, Outback Dingo outbackdi...@gmail.com wrote: trying to rerun a clang build of FreeBSD CURRENT fails on new import of sort, cat /etc/src.conf WITH_CLANG_IS_CC=1 make world -SNIP--- clang -O2 -pipe -DSORT_THREADS -std=gnu99 -Qunused-arguments -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -c /usr/src/usr.bin/sort/coll.c clang -O2 -pipe -DSORT_THREADS -std=gnu99 -Qunused-arguments -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -c /usr/src/usr.bin/sort/file.c /usr/src/usr.bin/sort/file.c:601:12: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(7)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n) catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:942:11: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(8)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n) catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:1279:10: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] err(2, getstr(9)); ^ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n) catgets(catalog, 1, n, nlsstr[n]) ^ /usr/src/usr.bin/sort/file.c:1295:12: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] errx(2, getstr(10)); ^~ /usr/src/usr.bin/sort/sort.h:52:20: note: expanded from macro 'getstr' #define getstr(n) catgets(catalog, 1, n, nlsstr[n]) ^ 4 errors generated. *** [file.o] Error code 1 Stop in /usr/src/usr.bin/sort. Yeah... errx(2, getstr(9)) should be errx(2, %s, getstr(9))... -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current- unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort coming to -CURRENT
Thank you, Michael. We will check this. Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd-curr...@freebsd.org] On Behalf Of Michael Scholz Sent: Tuesday, May 08, 2012 3:08 PM To: Gabor Kovesdan Cc: FreeBSD Current; Oleg Moskalenko Subject: Re: [HEADS-UP] BSD sort coming to -CURRENT Gabor Kovesdan wrote: Oleg Moskalenko has been working very hard on BSD sort and by now we think it is compatible with the base version (and has even more features, the ideas mostly taken from the latest GNU sort) and it is efficient. I just updated the textproc/bsdsort port to the latest version so that people can test it and I plan to commit it to HEAD in some days, for now installed as bsdsort, leaving GNU sort the default version. If someone has any objection, please raise it now. Future plans are to switch the logic and make BSD sort default once it has undergone enough testing and finally drop GNU sort permanently if no problems appear. BSD sort doesn't work with ports/textproc/ispell, it stops with sort: invalid option -- 1 % uname -a FreeBSD pumpkin.fth-devel.net 9.0-STABLE FreeBSD 9.0-STABLE #0: Fri May 4 15:34:58 CEST 2012 r...@pumpkin.fth-devel.net:/usr/obj/usr/src/sys/PUMPKIN i386 % ls -lF /usr/bin/*sort* lrwxr-xr-x 1 root wheel 13 May 8 22:15 /usr/bin/bsdsort@ - /usr/bin/sort -r-xr-xr-x 1 root wheel 65416 May 8 22:15 /usr/bin/gnusort* -r-xr-xr-x 1 root wheel 49280 May 8 22:15 /usr/bin/sort* -r-xr-xr-x 1 root wheel 7276 May 4 15:46 /usr/bin/tsort* # cd /usr/ports/textproc/ispell make [...] ./ispell -v | sed -e 1q @(#) International Ispell Version 3.3.02 12 Jun 2005 . ./config.sh; set +vx; sed -e s@!!BAKEXT!!@$BAKEXT@g -e s@!!COUNTSUFFIX!!@$COUNTSUFFIX@g -e s@!!DEFHASH!!@$DEFHASH@ -e s@!!DEFLANG!!@$DEFLANG@ -e s@!!HASHSUFFIX!!@$HASHSUFFIX@g -e s@!!LIBDIR!!@$LIBDIR@ -e s@!!DEFDICT!!@$DEFDICT@ -e s@!!LOOK_XREF!!@$LOOK_XREF@g -e s@!!MAN45SECT!!@$MAN45SECT@g -e s@!!POUNDBANG!!@$POUNDBANG@g -e s@!!SPELL_XREF!!@$SPELL_XREF@g -e s@!!STATSUFFIX!!@$STATSUFFIX@g -e s@!!TIB_XREF!!@$TIB_XREF@g -e s@!!WORDS!!@$WORDS@g $SORTTMP ispell.1X ispell.1 . ./config.sh; set +vx; sed -e s@!!BAKEXT!!@$BAKEXT@g -e s@!!COUNTSUFFIX!!@$COUNTSUFFIX@g -e s@!!DEFHASH!!@$DEFHASH@ -e s@!!DEFLANG!!@$DEFLANG@ -e s@!!HASHSUFFIX!!@$HASHSUFFIX@g -e s@!!LIBDIR!!@$LIBDIR@ -e s@!!DEFDICT!!@$DEFDICT@ -e s@!!LOOK_XREF!!@$LOOK_XREF@g -e s@!!MAN45SECT!!@$MAN45SECT@g -e s@!!POUNDBANG!!@$POUNDBANG@g -e s@!!SPELL_XREF!!@$SPELL_XREF@g -e s@!!STATSUFFIX!!@$STATSUFFIX@g -e s@!!TIB_XREF!!@$TIB_XREF@g -e s@!!WORDS!!@$WORDS@g $SORTTMP ispell.5X ispell.5 make LANGUAGE_TARGET=all SHELLDEBUG=+vx language-subdirs + cd languages/american + eval make BUILD=build DBUILD=build CBUILD=build SHELLDEBUG=+vx ' '\''MASTERDICTS=american.med+'\'' '\''HASHFILES=americanmed+.hash'\'' '\''EXTRADICT=/usr/share/dict/words'\' all + make BUILD=build DBUILD=build CBUILD=build SHELLDEBUG=+vx MASTERDICTS=american.med+ HASHFILES=americanmed+.hash EXTRADICT=/usr/share/dict/words all rm -f english.[0-3] american.[0-2] altamer.[012] british.[012] make -f ../english/Makefile 'DBUILD=' VARIANTS=american 'EXTRADICT=/usr/share/dict/words' 'SHELLDEBUG=+vx' 'AFFIXES=../english/english.aff' english.med+ + PATH=../..:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin + export PATH + ../../munchlist -v -l ../english/english.aff /usr/share/dict/words english.0 english.1 american.0 american.1 Collecting input. sort: invalid option -- 1 Usage: sort [-bcCdfigMmnrsuz] [-kPOS1[,POS2] ... ] [+POS1 [-POS2]] [-S memsize] [-T tmpdir] [-t separator] [-o outfile] [--batch-size size] [--files0-from file] [--heapsort] [--mergesort] [--radixsort] [--qsort] [--nthreads thread_no] [--human-numeric-sort] [--version-sort] [--random-sort [--random-source file]] [--compress-program program] [file ...] [...] Mike ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort coming to -CURRENT
Hi Michael We found the problem. In the ispell port, an unusual form of deprecated sort syntax for the key field(s) is used. There are three forms of key field(s) specification: POSIX standard: $ sort -kPOS1[,POS2] Documented deprecated form: $ sort +POS1 [-POS2] Undocumented deprecated form: $ sort -POS2 +POS1 The new sort supports two first forms. The ispell uses the third form (sort -1 +0). We can add support for the third form. We are going to do it today or tomorrow and we will test the ispell build. Thanks Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd-curr...@freebsd.org] On Behalf Of Michael Scholz Sent: Tuesday, May 08, 2012 3:08 PM To: Gabor Kovesdan Cc: FreeBSD Current; Oleg Moskalenko Subject: Re: [HEADS-UP] BSD sort coming to -CURRENT Gabor Kovesdan wrote: Oleg Moskalenko has been working very hard on BSD sort and by now we think it is compatible with the base version (and has even more features, the ideas mostly taken from the latest GNU sort) and it is efficient. I just updated the textproc/bsdsort port to the latest version so that people can test it and I plan to commit it to HEAD in some days, for now installed as bsdsort, leaving GNU sort the default version. If someone has any objection, please raise it now. Future plans are to switch the logic and make BSD sort default once it has undergone enough testing and finally drop GNU sort permanently if no problems appear. BSD sort doesn't work with ports/textproc/ispell, it stops with sort: invalid option -- 1 % uname -a FreeBSD pumpkin.fth-devel.net 9.0-STABLE FreeBSD 9.0-STABLE #0: Fri May 4 15:34:58 CEST 2012 r...@pumpkin.fth-devel.net:/usr/obj/usr/src/sys/PUMPKIN i386 % ls -lF /usr/bin/*sort* lrwxr-xr-x 1 root wheel 13 May 8 22:15 /usr/bin/bsdsort@ - /usr/bin/sort -r-xr-xr-x 1 root wheel 65416 May 8 22:15 /usr/bin/gnusort* -r-xr-xr-x 1 root wheel 49280 May 8 22:15 /usr/bin/sort* -r-xr-xr-x 1 root wheel 7276 May 4 15:46 /usr/bin/tsort* # cd /usr/ports/textproc/ispell make [...] ./ispell -v | sed -e 1q @(#) International Ispell Version 3.3.02 12 Jun 2005 . ./config.sh; set +vx; sed -e s@!!BAKEXT!!@$BAKEXT@g -e s@!!COUNTSUFFIX!!@$COUNTSUFFIX@g -e s@!!DEFHASH!!@$DEFHASH@ -e s@!!DEFLANG!!@$DEFLANG@ -e s@!!HASHSUFFIX!!@$HASHSUFFIX@g -e s@!!LIBDIR!!@$LIBDIR@ -e s@!!DEFDICT!!@$DEFDICT@ -e s@!!LOOK_XREF!!@$LOOK_XREF@g -e s@!!MAN45SECT!!@$MAN45SECT@g -e s@!!POUNDBANG!!@$POUNDBANG@g -e s@!!SPELL_XREF!!@$SPELL_XREF@g -e s@!!STATSUFFIX!!@$STATSUFFIX@g -e s@!!TIB_XREF!!@$TIB_XREF@g -e s@!!WORDS!!@$WORDS@g $SORTTMP ispell.1X ispell.1 . ./config.sh; set +vx; sed -e s@!!BAKEXT!!@$BAKEXT@g -e s@!!COUNTSUFFIX!!@$COUNTSUFFIX@g -e s@!!DEFHASH!!@$DEFHASH@ -e s@!!DEFLANG!!@$DEFLANG@ -e s@!!HASHSUFFIX!!@$HASHSUFFIX@g -e s@!!LIBDIR!!@$LIBDIR@ -e s@!!DEFDICT!!@$DEFDICT@ -e s@!!LOOK_XREF!!@$LOOK_XREF@g -e s@!!MAN45SECT!!@$MAN45SECT@g -e s@!!POUNDBANG!!@$POUNDBANG@g -e s@!!SPELL_XREF!!@$SPELL_XREF@g -e s@!!STATSUFFIX!!@$STATSUFFIX@g -e s@!!TIB_XREF!!@$TIB_XREF@g -e s@!!WORDS!!@$WORDS@g $SORTTMP ispell.5X ispell.5 make LANGUAGE_TARGET=all SHELLDEBUG=+vx language-subdirs + cd languages/american + eval make BUILD=build DBUILD=build CBUILD=build SHELLDEBUG=+vx ' '\''MASTERDICTS=american.med+'\'' '\''HASHFILES=americanmed+.hash'\'' '\''EXTRADICT=/usr/share/dict/words'\' all + make BUILD=build DBUILD=build CBUILD=build SHELLDEBUG=+vx MASTERDICTS=american.med+ HASHFILES=americanmed+.hash EXTRADICT=/usr/share/dict/words all rm -f english.[0-3] american.[0-2] altamer.[012] british.[012] make -f ../english/Makefile 'DBUILD=' VARIANTS=american 'EXTRADICT=/usr/share/dict/words' 'SHELLDEBUG=+vx' 'AFFIXES=../english/english.aff' english.med+ + PATH=../..:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin + export PATH + ../../munchlist -v -l ../english/english.aff /usr/share/dict/words english.0 english.1 american.0 american.1 Collecting input. sort: invalid option -- 1 Usage: sort [-bcCdfigMmnrsuz] [-kPOS1[,POS2] ... ] [+POS1 [-POS2]] [-S memsize] [-T tmpdir] [-t separator] [-o outfile] [--batch-size size] [--files0-from file] [--heapsort] [--mergesort] [--radixsort] [--qsort] [--nthreads thread_no] [--human-numeric-sort] [--version-sort] [--random-sort [--random-source file]] [--compress-program program] [file ...] [...] Mike ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: [HEADS-UP] BSD sort coming to -CURRENT
Michael, The situation is actually more complex than I described in my email, but the good news is that we already have a fix that supports all older syntax forms. I tested it with ispell ports and it builds just fine. We will update the bsdsort port soon. Thanks ! Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd-curr...@freebsd.org] On Behalf Of Oleg Moskalenko Sent: Tuesday, May 08, 2012 3:13 PM To: 'Michael Scholz'; Gabor Kovesdan Cc: FreeBSD Current Subject: RE: [HEADS-UP] BSD sort coming to -CURRENT Thank you, Michael. We will check this. Oleg -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd-curr...@freebsd.org] On Behalf Of Michael Scholz Sent: Tuesday, May 08, 2012 3:08 PM To: Gabor Kovesdan Cc: FreeBSD Current; Oleg Moskalenko Subject: Re: [HEADS-UP] BSD sort coming to -CURRENT Gabor Kovesdan wrote: Oleg Moskalenko has been working very hard on BSD sort and by now we think it is compatible with the base version (and has even more features, the ideas mostly taken from the latest GNU sort) and it is efficient. I just updated the textproc/bsdsort port to the latest version so that people can test it and I plan to commit it to HEAD in some days, for now installed as bsdsort, leaving GNU sort the default version. If someone has any objection, please raise it now. Future plans are to switch the logic and make BSD sort default once it has undergone enough testing and finally drop GNU sort permanently if no problems appear. BSD sort doesn't work with ports/textproc/ispell, it stops with sort: invalid option -- 1 % uname -a FreeBSD pumpkin.fth-devel.net 9.0-STABLE FreeBSD 9.0-STABLE #0: Fri May 4 15:34:58 CEST 2012 r...@pumpkin.fth-devel.net:/usr/obj/usr/src/sys/PUMPKIN i386 % ls -lF /usr/bin/*sort* lrwxr-xr-x 1 root wheel 13 May 8 22:15 /usr/bin/bsdsort@ - /usr/bin/sort -r-xr-xr-x 1 root wheel 65416 May 8 22:15 /usr/bin/gnusort* -r-xr-xr-x 1 root wheel 49280 May 8 22:15 /usr/bin/sort* -r-xr-xr-x 1 root wheel 7276 May 4 15:46 /usr/bin/tsort* # cd /usr/ports/textproc/ispell make [...] ./ispell -v | sed -e 1q @(#) International Ispell Version 3.3.02 12 Jun 2005 . ./config.sh; set +vx; sed -e s@!!BAKEXT!!@$BAKEXT@g -e s@!!COUNTSUFFIX!!@$COUNTSUFFIX@g -e s@!!DEFHASH!!@$DEFHASH@ -e s@!!DEFLANG!!@$DEFLANG@ -e s@!!HASHSUFFIX!!@$HASHSUFFIX@g -e s@!!LIBDIR!!@$LIBDIR@ -e s@!!DEFDICT!!@$DEFDICT@ -e s@!!LOOK_XREF!!@$LOOK_XREF@g -e s@!!MAN45SECT!!@$MAN45SECT@g -e s@!!POUNDBANG!!@$POUNDBANG@g -e s@!!SPELL_XREF!!@$SPELL_XREF@g -e s@!!STATSUFFIX!!@$STATSUFFIX@g -e s@!!TIB_XREF!!@$TIB_XREF@g -e s@!!WORDS!!@$WORDS@g $SORTTMP ispell.1X ispell.1 . ./config.sh; set +vx; sed -e s@!!BAKEXT!!@$BAKEXT@g -e s@!!COUNTSUFFIX!!@$COUNTSUFFIX@g -e s@!!DEFHASH!!@$DEFHASH@ -e s@!!DEFLANG!!@$DEFLANG@ -e s@!!HASHSUFFIX!!@$HASHSUFFIX@g -e s@!!LIBDIR!!@$LIBDIR@ -e s@!!DEFDICT!!@$DEFDICT@ -e s@!!LOOK_XREF!!@$LOOK_XREF@g -e s@!!MAN45SECT!!@$MAN45SECT@g -e s@!!POUNDBANG!!@$POUNDBANG@g -e s@!!SPELL_XREF!!@$SPELL_XREF@g -e s@!!STATSUFFIX!!@$STATSUFFIX@g -e s@!!TIB_XREF!!@$TIB_XREF@g -e s@!!WORDS!!@$WORDS@g $SORTTMP ispell.5X ispell.5 make LANGUAGE_TARGET=all SHELLDEBUG=+vx language-subdirs + cd languages/american + eval make BUILD=build DBUILD=build CBUILD=build SHELLDEBUG=+vx ' '\''MASTERDICTS=american.med+'\'' '\''HASHFILES=americanmed+.hash'\'' '\''EXTRADICT=/usr/share/dict/words'\' all + make BUILD=build DBUILD=build CBUILD=build SHELLDEBUG=+vx MASTERDICTS=american.med+ HASHFILES=americanmed+.hash EXTRADICT=/usr/share/dict/words all rm -f english.[0-3] american.[0-2] altamer.[012] british.[012] make -f ../english/Makefile 'DBUILD=' VARIANTS=american 'EXTRADICT=/usr/share/dict/words' 'SHELLDEBUG=+vx' 'AFFIXES=../english/english.aff' english.med+ + PATH=../..:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin + export PATH + ../../munchlist -v -l ../english/english.aff /usr/share/dict/words english.0 english.1 american.0 american.1 Collecting input. sort: invalid option -- 1 Usage: sort [-bcCdfigMmnrsuz] [-kPOS1[,POS2] ... ] [+POS1 [-POS2]] [-S memsize] [-T tmpdir] [-t separator] [-o outfile] [--batch-size size] [--files0-from file] [--heapsort] [--mergesort] [--radixsort] [--qsort] [--nthreads thread_no] [--human-numeric-sort] [--version-sort] [--random-sort [--random-source file]] [--compress-program program] [file ...] [...] Mike ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: CFT: new BSD-licensed sort available
This is true, debians do the symlinks trick. In Ubuntu : /usr/bin/java - /etc/alternatives/java /etc/alternatives/java - /usr/lib/jvm/java-6-openjdk/jre/bin/java Oleg From: Jonathan Anderson [mailto:jonathan.ander...@cl.cam.ac.uk] Sent: Wednesday, March 14, 2012 3:14 PM To: Adrian Chadd Cc: Gabor Kovesdan; freebsd-current@freebsd.org; Oleg Moskalenko; freebsd-po...@freebsd.org Subject: Re: CFT: new BSD-licensed sort available On 14 Mar 2012, at 21:10, Adrian Chadd wrote: Hi, This makes me think of the whole debian-y way of replacing the mailer programs using some magic alias program. So you could intall gnusort, bsdsort, and then some config file would determine which was used. 'sort' would then be a symlink to said magic program, that'd look at its argv[0], look at the contents of that file, and exec() the right one. In fact, the runtime behaviour of the Debian alternatives system is simpler than that: http://segfault.in/2010/04/using-the-debian-alternatives-system/ The custom Perl script with a config file is used to set up symlinks, which at runtime are... well, just symlinks. For instance, /usr/bin/vim is a symlink to /etc/alternatives/vim, which is itself a symlink to a binary like vim.gtk (example shamelessly stolen from the linked page, since I no longer have any Debian boxes to check for myself on :). No magic binaries or argv[0] fu. In one way, it's an elegant solution. On the other, it's a classic example of Wheeler's Law in action. :) Jon -- Jonathan Anderson Research Student, Security Group Computer Laboratory University of Cambridge +44 (1223) 763747 jonathan.ander...@cl.cam.ac.ukmailto:jonathan.ander...@cl.cam.ac.uk ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
