Re: FILE's _file can only hold a short
On Wednesday, October 31, 2012 02:12:55 PM m...@freebsd.org wrote: I seem to recall a thread earlier on this limitation, but looking at actual libc/stdio sources, the 4 year old check for open(2)'s fd being less than SHRT_MAX is still there. I thought I saw a patch to change this to an int, but it's not in the tree. Was this in a PR or a mailing list thread or am I just imagining things? We've run into this limitation at work, where some processes have around 32k open file descriptors and then try to use the libc FILE interface. Since we control ABI we can just change this to int, but I had been hoping there was a FreeBSD revision we could pull instead of having another diff. I had been working on a port-exp run. The problem I have run into is that perl actually reaches inside of FILE directly to clear out _file so it can control when the fd is actually closed (really gross). I have extended my stuff so that old Perl binaries should still work, but wanted to figure out how to prevent future Perl binaries from growing the same dependency. Also, I haven't had a chance to do a follow-up to find what else out in ports-land tries to use _file directly from FILE. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FILE's _file can only hold a short
On Fri, Jan 11, 2013 at 7:44 AM, John Baldwin j...@freebsd.org wrote: On Wednesday, October 31, 2012 02:12:55 PM m...@freebsd.org wrote: I seem to recall a thread earlier on this limitation, but looking at actual libc/stdio sources, the 4 year old check for open(2)'s fd being less than SHRT_MAX is still there. I thought I saw a patch to change this to an int, but it's not in the tree. Was this in a PR or a mailing list thread or am I just imagining things? We've run into this limitation at work, where some processes have around 32k open file descriptors and then try to use the libc FILE interface. Since we control ABI we can just change this to int, but I had been hoping there was a FreeBSD revision we could pull instead of having another diff. I had been working on a port-exp run. The problem I have run into is that perl actually reaches inside of FILE directly to clear out _file so it can control when the fd is actually closed (really gross). I have extended my stuff so that old Perl binaries should still work, but wanted to figure out how to prevent future Perl binaries from growing the same dependency. Also, I haven't had a chance to do a follow-up to find what else out in ports-land tries to use _file directly from FILE. John, I posted a problem I am having with a perl script that opens multiple filehandles with output to a pipe (open $fh,command|;). The script has worked fine for over a decade, but an upgrade of FreeBSD from 7.2 to 8.3 caused the program to fail to transfer the entire output (between 14 and 30KB of data in lines of between 1 and 80 bytes. (Output of show bgp summary on a Juniper.) The sub-process just hangs until my watchdog sends a sigterm after 15 seconds. When I read the filehadle, I get only a truncated output from these processes. Others complete normally with full output. the failures tend to be ones with larger outputs, but not always. I have worked around this, but I would love to know why it broke in 8.3 after working since at least v5 and probably 4.3. Since this is a common technique or running multiple commands in parallel in perl, it would be nice if I could at least understand what is going on. This looks a bit like what this thread is discussing, but I am far from certain. -- R. Kevin Oberman, Network Engineer E-mail: kob6...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FILE's _file can only hold a short
On Thu, Nov 1, 2012 at 7:41 PM, Eitan Adler li...@eitanadler.com wrote: On 1 November 2012 10:40, Ian Lepore free...@damnhippie.dyndns.org wrote: On Wed, 2012-10-31 at 11:12 -0700, m...@freebsd.org wrote: I seem to recall a thread earlier on this limitation, but looking at actual libc/stdio sources, the 4 year old check for open(2)'s fd being less than SHRT_MAX is still there. I thought I saw a patch to change this to an int, but it's not in the tree. Was this in a PR or a mailing list thread or am I just imagining things? We've run into this limitation at work, where some processes have around 32k open file descriptors and then try to use the libc FILE interface. Since we control ABI we can just change this to int, but I had been hoping there was a FreeBSD revision we could pull instead of having another diff. FWIW, I also remember some discussion recently (this year) on some mailing list about this, but I can't find it now. I thought it was somehow related to in-lib versus external uses of the funopen() function, but I may be conflating two unrelated discusssions in my head. Perhaps http://freebsd.1045724.n5.nabble.com/stdio-and-short-file-descriptors-revisited-td5747703.html ? Yes, that was it exactly. Thanks! My (quick) search had not been fruitful. Thanks, matthew ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FILE's _file can only hold a short
On Wed, 2012-10-31 at 11:12 -0700, m...@freebsd.org wrote: I seem to recall a thread earlier on this limitation, but looking at actual libc/stdio sources, the 4 year old check for open(2)'s fd being less than SHRT_MAX is still there. I thought I saw a patch to change this to an int, but it's not in the tree. Was this in a PR or a mailing list thread or am I just imagining things? We've run into this limitation at work, where some processes have around 32k open file descriptors and then try to use the libc FILE interface. Since we control ABI we can just change this to int, but I had been hoping there was a FreeBSD revision we could pull instead of having another diff. FWIW, I also remember some discussion recently (this year) on some mailing list about this, but I can't find it now. I thought it was somehow related to in-lib versus external uses of the funopen() function, but I may be conflating two unrelated discusssions in my head. -- Ian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FILE's _file can only hold a short
On 1 November 2012 10:40, Ian Lepore free...@damnhippie.dyndns.org wrote: On Wed, 2012-10-31 at 11:12 -0700, m...@freebsd.org wrote: I seem to recall a thread earlier on this limitation, but looking at actual libc/stdio sources, the 4 year old check for open(2)'s fd being less than SHRT_MAX is still there. I thought I saw a patch to change this to an int, but it's not in the tree. Was this in a PR or a mailing list thread or am I just imagining things? We've run into this limitation at work, where some processes have around 32k open file descriptors and then try to use the libc FILE interface. Since we control ABI we can just change this to int, but I had been hoping there was a FreeBSD revision we could pull instead of having another diff. FWIW, I also remember some discussion recently (this year) on some mailing list about this, but I can't find it now. I thought it was somehow related to in-lib versus external uses of the funopen() function, but I may be conflating two unrelated discusssions in my head. Perhaps http://freebsd.1045724.n5.nabble.com/stdio-and-short-file-descriptors-revisited-td5747703.html ? -- Eitan Adler ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
FILE's _file can only hold a short
I seem to recall a thread earlier on this limitation, but looking at actual libc/stdio sources, the 4 year old check for open(2)'s fd being less than SHRT_MAX is still there. I thought I saw a patch to change this to an int, but it's not in the tree. Was this in a PR or a mailing list thread or am I just imagining things? We've run into this limitation at work, where some processes have around 32k open file descriptors and then try to use the libc FILE interface. Since we control ABI we can just change this to int, but I had been hoping there was a FreeBSD revision we could pull instead of having another diff. Thanks, matthew ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
