Re: getline vs linebuffer
On 26 September 2011 23:24, Eric Blake ebl...@redhat.com wrote: So _both_ linebuffer and getdelim would benefit from a conversion to freadptr. I just tried using getndelim2 instead of getline for a program which slurps text line by line but spends most of its time regexp-searching the text, and it made no detectable difference, so I guess it really does have to be hardcore slurping. Is there some reason that getline getdelim can't just be wrappers to getndelim2? -- http://rrt.sc3d.org
Re: getline vs linebuffer
On 27 September 2011 22:45, Reuben Thomas r...@sc3d.org wrote: On 26 September 2011 23:24, Eric Blake ebl...@redhat.com wrote: So _both_ linebuffer and getdelim would benefit from a conversion to freadptr. I just tried using getndelim2 instead of getline for a program which slurps text line by line but spends most of its time regexp-searching the text, and it made no detectable difference, so I guess it really does have to be hardcore slurping. Or perhaps glibc's getline is efficient? -- http://rrt.sc3d.org
Re: getline vs linebuffer
On 09/27/2011 11:12 PM, Reuben Thomas wrote: On 27 September 2011 22:45, Reuben Thomas r...@sc3d.org wrote: On 26 September 2011 23:24, Eric Blake ebl...@redhat.com wrote: So _both_ linebuffer and getdelim would benefit from a conversion to freadptr. I just tried using getndelim2 instead of getline for a program which slurps text line by line but spends most of its time regexp-searching the text, and it made no detectable difference, so I guess it really does have to be hardcore slurping. Or perhaps glibc's getline is efficient? I found it very efficient http://www.pixelbeat.org/programming/readline/ Note when testing io functions ensure that gnulib/lib/unlocked-io.h is used appropriately, especially for getc using functions like getndelim2() I previously did an itemization of the line processing in coreutils: http://lists.gnu.org/archive/html/bug-coreutils/2009-03/msg00116.html cheers, Pádraig.
Re: getline vs linebuffer
2011/9/27 Pádraig Brady p...@draigbrady.com: On 09/27/2011 11:12 PM, Reuben Thomas wrote: Note when testing io functions ensure that gnulib/lib/unlocked-io.h is used appropriately, especially for getc using functions like getndelim2() The point Eric was making is that getndelim2 doesn't use getc (if it can help it) any more. -- http://rrt.sc3d.org
getline vs linebuffer
Is linebuffer obsoleteable in the face of standardized getline and friends? -- http://rrt.sc3d.org
Re: getline vs linebuffer
On 09/26/2011 03:47 PM, Reuben Thomas wrote: Is linebuffer obsoleteable in the face of standardized getline and friends? Not necessarily, because linebuffer provides a slightly different interface that might be easier to use. However, I do argue that both linebuffer and getdelim can be written MUCH more efficiently, by doing it in terms of freadahead and friends, rather than by one getc() at a time. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
Re: getline vs linebuffer
On 26 September 2011 23:04, Eric Blake ebl...@redhat.com wrote: On 09/26/2011 03:47 PM, Reuben Thomas wrote: Is linebuffer obsoleteable in the face of standardized getline and friends? Not necessarily, because linebuffer provides a slightly different interface that might be easier to use. If others agree with this, I'll write a patch which cross-refers the two and mentions that linebuffer can be easier to use. But I'd rather it disappeared! -- http://rrt.sc3d.org
Re: getline vs linebuffer
Hi Reuben, Is linebuffer obsoleteable in the face of standardized getline and friends? I'm not the maintainer of either of these modules, but I don't consider 'linebuffer' to be obsolete. Whenever I have to understand the calling convention of getline(), I get dizzy. What will happen if lineptr == NULL? And what if *lineptr == NULL? And what if *lineptr != NULL but *linesize == 0? Can I pass the result of a getline() call to another getline() call? Whereas 'linebuffer', through its object-oriented design, is much easier to understand: You call initbuffer(), and then readlinebuffer() will reuse the buffer as best as it can. Probably there are situations where getline() can be used and 'linebuffer' is not suitable. But for those situations where 'linebuffer' is suitable -- and I bet they are significant -- it is easier to use than getline(). Bruno -- In memoriam Pavlos Bakoyannis http://en.wikipedia.org/wiki/Pavlos_Bakoyannis
Re: getline vs linebuffer
On 09/26/2011 04:06 PM, Reuben Thomas wrote: On 26 September 2011 23:04, Eric Blakeebl...@redhat.com wrote: On 09/26/2011 03:47 PM, Reuben Thomas wrote: Is linebuffer obsoleteable in the face of standardized getline and friends? Not necessarily, because linebuffer provides a slightly different interface that might be easier to use. If others agree with this, I'll write a patch which cross-refers the two and mentions that linebuffer can be easier to use. But I'd rather it disappeared! Coreutils uses linebuffer far more than getline, and for good reason. Before you propose removing linebuffer, try writing a patch to make coreutils stop using it, and see how hard that is. Gnulib is allowed to provide both standard interfaces (getline) and useful GNU interfaces (linebuffer), where you can then choose which interface better suits your needs. But a patch for both modules that references the other may indeed be worthwhile. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
Re: getline vs linebuffer
On 26 September 2011 23:19, Eric Blake ebl...@redhat.com wrote: But a patch for both modules that references the other may indeed be worthwhile. Great, I'll write something (and I find your arguments convincing). At the moment the short descriptions in the module files are identical, which is confusing. The consensus seems to be that linebuffer is easier to use and possibly more flexible. (BTW, getline *is* a GNU interface, just one that happened to be standardized, at least, according to the man page.) -- http://rrt.sc3d.org
Re: getline vs linebuffer
On 09/26/2011 04:04 PM, Eric Blake wrote: On 09/26/2011 03:47 PM, Reuben Thomas wrote: Is linebuffer obsoleteable in the face of standardized getline and friends? Not necessarily, because linebuffer provides a slightly different interface that might be easier to use. However, I do argue that both linebuffer and getdelim can be written MUCH more efficiently, by doing it in terms of freadahead and friends, rather than by one getc() at a time. More to the point, I did the same conversion to getndelim2 back in 2008, and it made a very nice difference to cut: https://lists.gnu.org/archive/html/bug-coreutils/2008-04/msg00372.html So _both_ linebuffer and getdelim would benefit from a conversion to freadptr. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
getline on AIX 7.1BETA
The log says this: checking whether getline is declared... no checking for getline... yes I'm updating the doc: 2010-07-30 Bruno Haible br...@clisp.org getline: Update regarding AIX. * doc/posix-functions/getline.texi: Mention bug on AIX 7.1. Reported by Rainer Tammer. --- doc/posix-functions/getline.texi.orig Fri Jul 30 21:23:04 2010 +++ doc/posix-functions/getline.texiFri Jul 30 11:45:31 2010 @@ -13,7 +13,7 @@ MacOS X 10.3, FreeBSD 6.0, NetBSD 3.0, OpenBSD 3.8, AIX 5.1, HP-UX 11, IRIX 6.5, OSF/1 5.1, Solaris 10, mingw, Interix 3.5. @item This function is missing a declaration on some platforms: -BeOS. +AIX 7.1, BeOS. @item Some platforms provide a function by this name but with the wrong signature, for example in -linet.
getline on mingw
A different case is this error: ../gllib/stdio.h:803: error: `getline' was not declared in this scope Here we really want to do #define getline rpl_getline even if the system does not declare the function, for the reason mentioned in the documentation: Some platforms provide a function by this name but with the wrong signature, for example in -linet. So the fix is this: 2010-04-03 Bruno Haible br...@clisp.org stdio: Make C++ tests work on mingw. * lib/stdio.in.h (getline): Don't use _GL_CXXALIASWARN if the system does not declare the function. --- lib/stdio.in.h.orig Sat Apr 3 14:33:08 2010 +++ lib/stdio.in.h Sat Apr 3 14:31:26 2010 @@ -525,7 +525,9 @@ _GL_CXXALIAS_SYS (getline, ssize_t, (char **lineptr, size_t *linesize, FILE *stream)); # endif +# if @HAVE_DECL_GETLINE@ _GL_CXXALIASWARN (getline); +# endif #elif defined GNULIB_POSIXCHECK # undef getline # if HAVE_RAW_DECL_GETLINE
Re: [Bug-tar] getline() differences FreeBSD / GNU libc
Eric Blake wrote:
FreeBSD is buggy. Rewording the POSIX requirement slightly: if lineptr is
NULL, then it is irrelevant that linecapp is non-zero.
I agree.
Gnulib needs to work around this portability flaw, and guarantee a working
getline implementation even on fBSD, at which point, your patch to tar to
explicitly initialize bufsize to 0 is unnecessary.
Implemented as follows. Since I don't have access to a machine with FreeBSD 8.0,
I would appreciate if someone could test this on FreeBSD 8.0:
$ ./gnulib-tool --create-testdir --dir=/tmp/testdir --with-tests getline
$ cd /tmp/testdir
$ ./configure CPPFLAGS=-Wall
# What is the configure output regarding getline and getdelim here?
$ make; make check
Bruno
2010-01-31 Bruno Haible br...@clisp.org
Work around getline() bug on FreeBSD 8.0.
* m4/getline.m4 (gl_FUNC_GETLINE): Also test result for a NULL buffer
and a non-zero size.
* tests/test-getline.c (main): Likewise.
* doc/posix-functions/getline.texi: Mention the FreeBSD bug.
Reported by Dennis noord...@cs.helsinki.fi via Eric Blake.
--- doc/posix-functions/getline.texi.orig Sun Jan 31 17:20:48 2010
+++ doc/posix-functions/getline.texiSun Jan 31 16:49:18 2010
@@ -17,6 +17,10 @@
@item
Some platforms provide a function by this name but with the wrong
signature, for example in -linet.
+...@item
+This function crashes when passed a pointer to a NULL buffer together with a
+pointer to a non-zero buffer size on some platforms:
+FreeBSD 8.0.
@end itemize
Portability problems not fixed by Gnulib:
--- m4/getline.m4.orig Sun Jan 31 17:20:48 2010
+++ m4/getline.m4 Sun Jan 31 17:07:24 2010
@@ -1,4 +1,4 @@
-# getline.m4 serial 20
+# getline.m4 serial 21
dnl Copyright (C) 1998-2003, 2005-2007, 2009-2010 Free Software Foundation,
dnl Inc.
@@ -24,26 +24,39 @@
gl_getline_needs_run_time_check=no
AC_CHECK_FUNC([getline],
-dnl Found it in some library. Verify that it works.
-gl_getline_needs_run_time_check=yes,
-am_cv_func_working_getline=no)
+[dnl Found it in some library. Verify that it works.
+ gl_getline_needs_run_time_check=yes],
+[am_cv_func_working_getline=no])
if test $gl_getline_needs_run_time_check = yes; then
AC_CACHE_CHECK([for working getline function],
[am_cv_func_working_getline],
-[echo fooN |tr -d '\012'|tr N '\012' conftest.data
+[echo fooNbarN | tr -d '\012' | tr N '\012' conftest.data
AC_RUN_IFELSE([AC_LANG_SOURCE([[
#include stdio.h
#include stdlib.h
#include string.h
int main ()
-{ /* Based on a test program from Karl Heuer. */
- char *line = NULL;
- size_t siz = 0;
- int len;
+{
FILE *in = fopen (./conftest.data, r);
if (!in)
return 1;
- len = getline (line, siz, in);
- exit ((len == 4 line strcmp (line, foo\n) == 0) ? 0 : 1);
+ {
+/* Test result for a NULL buffer and a zero size.
+ Based on a test program from Karl Heuer. */
+char *line = NULL;
+size_t siz = 0;
+int len = getline (line, siz, in);
+if (!(len == 4 line strcmp (line, foo\n) == 0))
+ return 1;
+ }
+ {
+/* Test result for a NULL buffer and a non-zero size.
+ This crashes on FreeBSD 8.0. */
+char *line = NULL;
+size_t siz = (size_t)(~0) / 4;
+if (getline (line, siz, in) == -1)
+ return 1;
+ }
+ return 0;
}
]])], [am_cv_func_working_getline=yes] dnl The library version works.
, [am_cv_func_working_getline=no] dnl The library version does NOT work.
--- tests/test-getline.c.orig Sun Jan 31 17:20:48 2010
+++ tests/test-getline.cSun Jan 31 17:20:22 2010
@@ -33,13 +33,13 @@
main (void)
{
FILE *f;
- char *line = NULL;
- size_t len = 0;
+ char *line;
+ size_t len;
ssize_t result;
/* Create test file. */
f = fopen (test-getline.txt, wb);
- if (!f || fwrite (a\nbc\nd\0f, 1, 8, f) != 8 || fclose (f) != 0)
+ if (!f || fwrite (a\nA\nbc\nd\0f, 1, 10, f) != 10 || fclose (f) != 0)
{
fputs (Failed to create sample file.\n, stderr);
remove (test-getline.txt);
@@ -54,13 +54,24 @@
}
/* Test initial allocation, which must include trailing NUL. */
+ line = NULL;
+ len = 0;
result = getline (line, len, f);
ASSERT (result == 2);
ASSERT (strcmp (line, a\n) == 0);
ASSERT (2 len);
+ free (line);
- /* Test growth of buffer, must not leak. */
+ /* Test initial allocation again, with line = NULL and len != 0. */
+ line = NULL;
+ len = (size_t)(~0) / 4;
+ result = getline (line, len, f);
+ ASSERT (result == 2);
+ ASSERT (strcmp (line, A\n) == 0);
+ ASSERT (2 len);
free (line);
+
+ /* Test growth of buffer, must not leak. */
line = malloc (1);
len = 0;
result = getline (line, len, f);
Re: [Bug-tar] getline() differences FreeBSD / GNU libc
2010-01-31 Bruno Haible br...@clisp.org
Work around getline() bug on FreeBSD 8.0.
* m4/getline.m4 (gl_FUNC_GETLINE): Also test result for a NULL buffer
and a non-zero size.
* tests/test-getline.c (main): Likewise.
* doc/posix-functions/getline.texi: Mention the FreeBSD bug.
Reported by Dennis noord...@cs.helsinki.fi via Eric Blake.
And similarly for getdelim.
2010-01-31 Bruno Haible br...@clisp.org
Work around getdelim() bug on FreeBSD 8.0.
* m4/getdelim.m4 (gl_FUNC_GETDELIM): Test whether getdelim supports an
initially NULL line. Set REPLACE_GETDELIM if getdelim exists but does
not work.
* lib/stdio.in.h (getdelim): Define as an alias if REPLACE_GETDELIM
is 1.
* m4/stdio_h.m4 (gl_STDIO_H_DEFAULTS): Initialize REPLACE_GETDELIM.
* modules/stdio (Makefile.am): Also substitute REPLACE_GETDELIM.
* tests/test-getdelim.c (main): Also test result for a NULL buffer and
a non-zero size.
* doc/posix-functions/getdelim.texi: Mention the FreeBSD bug.
--- doc/posix-functions/getdelim.texi.orig Sun Jan 31 17:33:06 2010
+++ doc/posix-functions/getdelim.texi Sun Jan 31 17:30:59 2010
@@ -14,6 +14,10 @@
@item
This function is missing a declaration on some platforms:
BeOS.
+...@item
+This function crashes when passed a pointer to a NULL buffer together with a
+pointer to a non-zero buffer size on some platforms:
+FreeBSD 8.0.
@end itemize
Portability problems not fixed by Gnulib:
--- lib/stdio.in.h.orig Sun Jan 31 17:33:06 2010
+++ lib/stdio.in.h Sun Jan 31 17:30:59 2010
@@ -340,7 +340,11 @@
#endif
#if @GNULIB_GETDELIM@
-# if !...@have_decl_getdelim@
+# if @REPLACE_GETDELIM@
+# undef getdelim
+# define getdelim rpl_getdelim
+# endif
+# if !...@have_decl_getdelim@ || @REPLACE_GETDELIM@
/* Read input, up to (and including) the next occurrence of DELIMITER, from
STREAM, store it in *LINEPTR (and NUL-terminate it).
*LINEPTR is a pointer returned from malloc (or NULL), pointing to *LINESIZE
--- m4/getdelim.m4.orig Sun Jan 31 17:33:06 2010
+++ m4/getdelim.m4 Sun Jan 31 17:30:59 2010
@@ -1,6 +1,6 @@
-# getdelim.m4 serial 5
+# getdelim.m4 serial 6
-dnl Copyright (C) 2005, 2006, 2007, 2009, 2010 Free Software Foundation, Inc.
+dnl Copyright (C) 2005-2007, 2009-2010 Free Software Foundation, Inc.
dnl
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -15,16 +15,71 @@
dnl Persuade glibc stdio.h to declare getdelim().
AC_REQUIRE([AC_USE_SYSTEM_EXTENSIONS])
- AC_REPLACE_FUNCS([getdelim])
AC_CHECK_DECLS_ONCE([getdelim])
- if test $ac_cv_func_getdelim = no; then
-gl_PREREQ_GETDELIM
+ AC_CHECK_FUNCS_ONCE([getdelim])
+ if test $ac_cv_func_getdelim = yes; then
+dnl Found it in some library. Verify that it works.
+AC_CACHE_CHECK([for working getdelim function],
[gl_cv_func_working_getdelim],
+[echo fooNbarN | tr -d '\012' | tr N '\012' conftest.data
+AC_RUN_IFELSE([AC_LANG_SOURCE([[
+#include stdio.h
+#include stdlib.h
+#include string.h
+int main ()
+{
+ FILE *in = fopen (./conftest.data, r);
+ if (!in)
+return 1;
+ {
+/* Test result for a NULL buffer and a zero size.
+ Based on a test program from Karl Heuer. */
+char *line = NULL;
+size_t siz = 0;
+int len = getdelim (line, siz, '\n', in);
+if (!(len == 4 line strcmp (line, foo\n) == 0))
+ return 1;
+ }
+ {
+/* Test result for a NULL buffer and a non-zero size.
+ This crashes on FreeBSD 8.0. */
+char *line = NULL;
+size_t siz = (size_t)(~0) / 4;
+if (getdelim (line, siz, '\n', in) == -1)
+ return 1;
+ }
+ return 0;
+}
+]])], [gl_cv_func_working_getdelim=yes] dnl The library version works.
+, [gl_cv_func_working_getdelim=no] dnl The library version does NOT work.
+, dnl We're cross compiling. Assume it works on glibc2 systems.
+ [AC_EGREP_CPP([Lucky GNU user],
+ [
+#include features.h
+#ifdef __GNU_LIBRARY__
+ #if (__GLIBC__ = 2)
+ Lucky GNU user
+ #endif
+#endif
+ ],
+ [gl_cv_func_working_getdelim=yes],
+ [gl_cv_func_working_getdelim=no])]
+)])
+ else
+gl_cv_func_working_getdelim=no
fi
if test $ac_cv_have_decl_getdelim = no; then
HAVE_DECL_GETDELIM=0
fi
+
+ if test $gl_cv_func_working_getdelim = no; then
+if test $ac_cv_func_getdelim = yes; then
+ REPLACE_GETDELIM=1
+fi
+AC_LIBOBJ([getdelim])
+gl_PREREQ_GETDELIM
+ fi
])
# Prerequisites of lib/getdelim.c.
--- m4/stdio_h.m4.orig Sun Jan 31 17:33:06 2010
+++ m4/stdio_h.m4 Sun Jan 31 17:30:59 2010
@@ -1,4 +1,4 @@
-# stdio_h.m4 serial 24
+# stdio_h.m4 serial 25
dnl Copyright (C) 2007-2010 Free Software Foundation, Inc.
dnl This file is free software; the Free
Re: [Bug-tar] getline() differences FreeBSD / GNU libc
[adding bug-gnulib, as the source of tar's getline replacement] According to noordsij on 1/21/2010 6:37 AM: Dear GNU tar maintainer(s) / fBSD gtar port maintainer(s), The problem appears to be a difference in getline() behaviour in the fBSD and GNU libc. fBSD: The caller may provide a pointer to a malloc buffer for the line in *linep, and the capacity of that buffer in *linecapp; if *linecapp is 0, then *linep is treated as NULL. GNU: If *lineptr is NULL, then getline() will allocate a buffer for storing the line, which should be freed by the user program. (The value in *n is ignored.) POSIX states: The application shall ensure that *lineptr is a valid argument that could be passed to the free() function. If *n is non-zero, the application shall ensure that *lineptr either points to an object of size at least *n bytes, or is a null pointer. So fBSD libc looks at the value of linecapp, whereas GNU libc looks at the value of lineptr, to determine whether to allocate a new buffer or use a provided one. In the tar source, linecapp (bufsize) is not initialized, lineptr (buf) is. The fix is to simply initialize bufsize to 0 as well, to make (line 1232-1233): char *buf = 0; size_t bufsize = 0; FreeBSD is buggy. Rewording the POSIX requirement slightly: if lineptr is NULL, then it is irrelevant that linecapp is non-zero. Gnulib needs to work around this portability flaw, and guarantee a working getline implementation even on fBSD, at which point, your patch to tar to explicitly initialize bufsize to 0 is unnecessary. That said, your patch to tar doesn't hurt; and besides, some implementations of getline use linecapp as a hint of how large to make the initial allocation, so passing uninitialized bufsize will potentially cause different allocation behavior (although any decent implementation that uses bufsize as an initial hint should also be prepared to ignore that hint if it is too far out of bounds, so it should never provoke ENOMEM by itself). -- Don't work too hard, make some time for fun as well! Eric Blake e...@byu.net signature.asc Description: OpenPGP digital signature
Re: close vs socket vs getline
Bruno Haible br...@clisp.org writes: Some ideas: 1) Should the sockets.m4 module unconditionally add LIBSOCKET to LDADD? 2) Should the gnulib-tool generated Makefile.am add LIBSOCKET to LDADD? No, LDADD has an effect on all executables. I don't want to link all my programs against -lws2_32 just because one of them uses sockets. No, I don't want to add LIBSOCKET to the link dependencies of the 'close' and 'fclose' either. It would also have the effect that all or nearly my programs would link against -lws2_32. Dynamic loading of ws2_32,dll is not the solution either; it would still have the effect of loading an extra library in programs that don't need it. Right. Is there any other options? I propose to move the sockets related part of close() into the 'sockets' module, and connect the close() code with the sockets related code at runtime rather than at compile time. Like this (tested on mingw). Nice idea! It adds some complexity, but it makes the other code easier to understand once you have understood close-hook. I have tested it, and it seems to work for me on mingw and normal x86, and solves the problem. Please push it. /Simon
Re: close vs socket vs getline
Simon Josefsson wrote:
This may have been discussed before, but I don't recall a solution, so
maybe a new report will help.
Yes, in the meantime we all had time to think about it.
i586-mingw32msvc-gcc -g -O2 -o test-getdelim.exe test-getdelim.o
../gllib/libgnu.a
../gllib/libgnu.a(close.o): In function `_gl_close_fd_maybe_socket':
/home/jas/src/gnulib/m/gllib/close.c:39: undefined reference to
`_wsaenumnetworkeve...@12'
/home/jas/src/gnulib/m/gllib/close.c:46: undefined reference to
`_closesoc...@4'
../gllib/libgnu.a(close.o): In function `set_winsock_errno':
/home/jas/src/gnulib/m/gllib/w32sock.h:34: undefined reference to
`_wsagetlaster...@0'
/home/jas/src/gnulib/m/gllib/w32sock.h:35: undefined reference to
`_wsasetlaster...@4'
The logic behind it is correct: some functions in the package open sockets,
and test-getdelim calls fclose(), which calls close(), which - given the unknown
origin of the file descriptor - needs to call WSAEnumNetworkEvents(), which
requires -lws2_32.
Some ideas:
1) Should the sockets.m4 module unconditionally add LIBSOCKET to LDADD?
2) Should the gnulib-tool generated Makefile.am add LIBSOCKET to LDADD?
No, LDADD has an effect on all executables. I don't want to link all my
programs against -lws2_32 just because one of them uses sockets.
No, I don't want to add LIBSOCKET to the link dependencies of the 'close'
and 'fclose' either. It would also have the effect that all or nearly my
programs would link against -lws2_32.
Dynamic loading of ws2_32,dll is not the solution either; it would still
have the effect of loading an extra library in programs that don't need it.
Is there any other options?
I propose to move the sockets related part of close() into the 'sockets'
module, and connect the close() code with the sockets related code at runtime
rather than at compile time. Like this (tested on mingw).
This uses the hook design pattern. It would be possible to optimize it,
knowing that at most one hook will be installed, but there's not much
point in doing so.
2009-03-20 Bruno Haible br...@clisp.org
Remove dependency from 'close' module to -lws2_32 on native Windows.
* lib/close-hook.h: New file.
* lib/close-hook.c: New file.
* lib/close.c: Include close-hook.h. Don't include sys/socket.h,
w32sock.h.
(_gl_close_fd_maybe_socket): Remove function.
(rpl_close): Invoke execute_all_close_hooks instead of
_gl_close_fd_maybe_socket.
* lib/sockets.c: Include close-hook.h, w32sock.h.
(close_fd_maybe_socket): New function, essentially from lib/close.c.
(close_sockets_hook): New variable.
(gl_sockets_startup): Register close_fd_maybe_socket as a hook.
(gl_sockets_cleanup): Unregister it.
* lib/unistd.in.h (HAVE__GL_CLOSE_FD_MAYBE_SOCKET): Remove macro.
* m4/close.m4 (gl_REPLACE_CLOSE): Undo 2009-02-05 change.
* modules/close-hook: New file.
* modules/close (Files): Remove lib/w32sock.h.
(Depends-on): Add close-hook.
(Link): Remove section.
* modules/sockets (Files): Add lib/w32sock.h.
(Depends-on): Add close-hook.
* modules/sys_socket (configure.ac): Remove gl_MODULE_INDICATOR
invocation.
* NEWS: Mention that LIB_CLOSE is gone.
== lib/close-hook.h ==
/* Hook for making the close() function extensible.
Copyright (C) 2009 Free Software Foundation, Inc.
This program is free software: you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with this program. If not, see http://www.gnu.org/licenses/. */
#ifndef CLOSE_HOOK_H
#define CLOSE_HOOK_H
#ifdef __cplusplus
extern C {
#endif
/* Currently, this entire code is only needed for the handling of sockets
on native Windows platforms. */
#if WINDOWS_SOCKETS
/* An element of the list of close hooks.
The fields of this structure are considered private. */
struct close_hook
{
/* Doubly linked list. */
struct close_hook *private_next;
struct close_hook *private_prev;
/* Function that treats the types of FD that it knows about and calls
execute_close_hooks (FD, REMAINING_LIST) as a fallback. */
int (*private_fn) (int fd, const struct close_hook *remaining_list);
};
/* This type of function closes FD, applying special knowledge for the FD
types it knows about, and calls execute_close_hooks (FD, REMAINING_LIST)
for the other
close vs socket vs getline
This may have been discussed before, but I don't recall a solution, so maybe a new report will help. Gsasl and gnutls uses the close, socket and getline modules from gnulib. I get a problem when building gnulib's self-tests for mingw. You can reproduce it by using: rm -rf m gnulib-tool --create-testdir --with-tests --dir m close getdelim socket cd m ./configure --host=i586-mingw32msvc --build=i686-pc-linux-gnu make check The error is: i586-mingw32msvc-gcc -g -O2 -o test-getdelim.exe test-getdelim.o ../gllib/libgnu.a ../gllib/libgnu.a(close.o): In function `_gl_close_fd_maybe_socket': /home/jas/src/gnulib/m/gllib/close.c:39: undefined reference to `_wsaenumnetworkeve...@12' /home/jas/src/gnulib/m/gllib/close.c:46: undefined reference to `_closesoc...@4' ../gllib/libgnu.a(close.o): In function `set_winsock_errno': /home/jas/src/gnulib/m/gllib/w32sock.h:34: undefined reference to `_wsagetlaster...@0' /home/jas/src/gnulib/m/gllib/w32sock.h:35: undefined reference to `_wsasetlaster...@4' What appears to be missing is that -lws2_32 is added to the linker line. If I link the test-getdelim.exe program with -lws2_32 and re-run make check, everything works fine. Some ideas: 1) Should the sockets.m4 module unconditionally add LIBSOCKET to LDADD? 2) Should the gnulib-tool generated Makefile.am add LIBSOCKET to LDADD? Is there any other options? Thanks, /Simon
new module 'calloc-posix' (was: Re: getline() behaviour change)
Eric Blake wrote:
I would somewhat like this idea - it is much nicer assuming that malloc
reliably sets errno to ENOMEM on failure than having to patch all callers
of malloc to do the same.
Jim Meyering confirmed:
Yes. This is a fundamental goal of gnulib:
If there is some portability problem, don't penalize *all* software and
all systems. Instead, keep the client software clean, and penalize
only the systems that have the problem. The losing system (mingw,
in this case) can endure a little overhead in malloc/realloc wrappers.
Hmm, I was hesitating, because adding runtime overhead to _all_ malloc/realloc
calls for the (rare) case of failure seems like overkill. But now I stumble
on the need to set errno = ENOMEM even in totally unsuspected modules like
'xreadlink', so I come to agree that handling this in all callers is
practically infeasible.
So I propose to add three modules 'malloc-posix', 'realloc-posix',
'calloc-posix' (using the '-posix' suffix that we already know from
'fnmatch-posix'/'fnmatch-gnu' and 'printf-posix').
Since 'calloc' is a little simpler to handle than 'malloc' and 'realloc',
here is first a proposal for 'calloc'.
Also, I would propose to rename the modules 'malloc' - 'malloc-gnu',
'realloc' - 'realloc-gnu', 'calloc' - 'calloc-gnu', similar to the naming
of the fnmatch-* modules. (Only the modules. The file names and autoconf
macro names can be left untouched.) Objections?
2007-09-02 Bruno Haible [EMAIL PROTECTED]
* modules/calloc-posix: New file.
* lib/calloc.c: Include errno.h.
(rpl_calloc): Merge the requirements of a glibc-compatible calloc
and a POSIX-compatible calloc into a single function. Set ENOMEM
when returning NULL.
* m4/calloc.m4 (gl_FUNC_CALLOC_POSIX): New macro.
* doc/functions/calloc.texi: Mention the calloc-posix module.
* lib/stdlib_.h (calloc): New declaration.
* m4/stdlib_h.m4 (gl_STDLIB_H_DEFAULTS): Initialize
GNULIB_CALLOC_POSIX and HAVE_CALLOC_POSIX.
* modules/stdlib (stdlib.h): Substitute also GNULIB_CALLOC_POSIX
and HAVE_CALLOC_POSIX.
=== modules/calloc-posix =
Description:
calloc() function: allocate memory with indefinite extent.
Files:
lib/calloc.c
m4/calloc.m4
Depends-on:
stdlib
configure.ac:
gl_FUNC_CALLOC_POSIX
gl_STDLIB_MODULE_INDICATOR([calloc-posix])
Makefile.am:
Include:
stdlib.h
License:
LGPL
Maintainer:
Bruno Haible
==
*** lib/calloc.c13 Sep 2006 22:38:14 - 1.6
--- lib/calloc.c2 Sep 2007 23:16:14 -
***
*** 1,6
/* calloc() function that is glibc compatible.
!This wrapper function is required at least on Tru64 UNIX 5.1.
!Copyright (C) 2004, 2005, 2006 Free Software Foundation, Inc.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
--- 1,6
/* calloc() function that is glibc compatible.
!This wrapper function is required at least on Tru64 UNIX 5.1 and mingw.
!Copyright (C) 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
***
*** 16,44
along with this program; if not, write to the Free Software Foundation,
Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */
! /* written by Jim Meyering */
#include config.h
! #undef calloc
#include stdlib.h
/* Allocate and zero-fill an NxS-byte block of memory from the heap.
If N or S is zero, allocate and zero-fill a 1-byte block. */
void *
rpl_calloc (size_t n, size_t s)
{
! size_t bytes;
if (n == 0 || s == 0)
! return calloc (1, 1);
!
! /* Defend against buggy calloc implementations that mishandle
! size_t overflow. */
! bytes = n * s;
! if (bytes / s != n)
! return NULL;
! return calloc (n, s);
}
--- 16,68
along with this program; if not, write to the Free Software Foundation,
Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */
! /* written by Jim Meyering and Bruno Haible*/
#include config.h
! /* Only the AC_FUNC_CALLOC macro defines 'calloc' already in config.h. */
! #ifdef calloc
! # define NEED_CALLOC_GNU
! # undef calloc
! #endif
+ /* Specification. */
#include stdlib.h
+ #include errno.h
+
/* Allocate and zero-fill an NxS-byte block of memory from the heap.
If N or S is zero, allocate and zero-fill a 1-byte block. */
void *
rpl_calloc (size_t n, size_t s)
{
! void *result;
+ #ifdef NEED_CALLOC_GNU
if (n == 0 || s == 0)
! {
! n = 1;
! s = 1;
! }
! else
! {
! /* Defend against buggy calloc implementations that mishandle
!
Re: getline() behaviour change
Eric Blake [EMAIL PROTECTED] wrote: According to Ben Pfaff on 8/22/2007 5:19 PM: Bruno Haible [EMAIL PROTECTED] writes: Jim Meyering wrote: A *lot* of code expects malloc and realloc to set errno when they fail. Such code is not portable to plain ISO C 99 systems. We could extend gnulib's existing malloc/realloc wrappers to ensure that malloc/realloc set errno when they fail, if it is worthwhile to do so. I would somewhat like this idea - it is much nicer assuming that malloc reliably sets errno to ENOMEM on failure than having to patch all callers of malloc to do the same. Yes. This is a fundamental goal of gnulib: If there is some portability problem, don't penalize *all* software and all systems. Instead, keep the client software clean, and penalize only the systems that have the problem. The losing system (mingw, in this case) can endure a little overhead in malloc/realloc wrappers. That's far better than polluting all malloc-using application code with obscure, seemingly-unnecessary code to set errno upon malloc failure.
do not set errno after failing malloc [Re: getline() behaviour change
Bruno Haible [EMAIL PROTECTED] wrote: Jim Meyering wrote: Are you advocating support for non-POSIX malloc/realloc? Sure. gnulib supports - to a large extent - mingw. It uses a malloc implementation from msvcrt.dll. This malloc does not set errno. You're dumbing down gnulib to accommodate _mingw_?!? Please, don't do that. Add a wrapper, rather than polluting all application code. This is a fundamental tenet of gnulib: If there's some portability problem, gnulib allows all of its client code to assume the desired behavior. Nonconforming hosts like mingw can endure the run-time overhead of wrappers or replacement functions. That's far cleaner than polluting malloc-using applications with obscure code to set errno upon malloc failure.
Re: getline() behaviour change
Ben Pfaff [EMAIL PROTECTED] wrote: Bruno Haible [EMAIL PROTECTED] writes: Jim Meyering wrote: A *lot* of code expects malloc and realloc to set errno when they fail. Such code is not portable to plain ISO C 99 systems. We could extend gnulib's existing malloc/realloc wrappers to ensure that malloc/realloc set errno when they fail, if it is worthwhile to do so. It is definitely worthwhile to do so.
Re: getline() behaviour change
Hi Eric,
The patch includes a couple of bug fixes in getdelim.c:
It is valid on entrance for *lineptr to be non-NULL but *n to be 0 (ie.
via malloc(0)), but your implementation was leaking that memory. Also,
your implementation failed to set errno to EOVERFLOW when it detects that
it needs to malloc more than SSIZE_MAX bytes.
--- lib/getdelim.c29 Oct 2006 21:52:55 - 1.7
+++ lib/getdelim.c22 Aug 2007 00:04:01 -
@@ -62,10 +67,10 @@ getdelim (char **lineptr, size_t *n, int
flockfile (fp);
- if (*lineptr == NULL || *n == 0)
+ if (*n == 0)
This is a behaviour change: Previously when *lineptr == NULL, *n did not
need to be initialized. Now it needs to be initialized to 0. Should be
mentioned in NEWS.
{
*n = 120;
- *lineptr = (char *) malloc (*n);
+ *lineptr = (char *) realloc (*lineptr, 120);
if (*lineptr == NULL)
{
result = -1;
It's more maintainable to write *n instead of 120. Compilers optimize this
memory access anyway.
@@ -97,6 +102,7 @@ getdelim (char **lineptr, size_t *n, int
if (cur_len + 1 = needed)
{
result = -1;
+ errno = EOVERFLOW;
goto unlock_return;
}
The POSIX spec implies that errno should be set when getdelim fails. Therefore
errno needs to be set not only here, with EOVERFLOW. In lines 77 and 113
errno should be set to ENOMEM, since errno is undefined upon return from
malloc() or realloc().
Bruno
Re: getline() behaviour change
Jim Meyering wrote:
Are you advocating support for non-POSIX malloc/realloc?
Sure. gnulib supports - to a large extent - mingw. It uses a malloc
implementation from msvcrt.dll. This malloc does not set errno.
A *lot* of code expects malloc and realloc to set errno when they fail.
Such code is not portable to plain ISO C 99 systems.
A lot of code also expects malloc and realloc (and other library calls)
to leave errno untouched when they succeed [1]. This is also wrong, even
on POSIX systems.
Here are the two blocks of code you mention:
*lineptr = (char *) realloc (*lineptr, 120);
if (*lineptr == NULL)
{
result = -1;
goto unlock_return;
}
...
new_lineptr = (char *) realloc (*lineptr, needed);
if (new_lineptr == NULL)
{
result = -1;
goto unlock_return;
}
in each case, realloc fails, so errno is defined.
Errno is not defined in this case, on mingw.
unlock_return:
funlockfile (fp);
return result;
}
Upon failure, on a system with the funlockfile function,
the errno value from a failed realloc may be clobbered
by that funlockfile call.
Yes. Well spotted!
Bruno
[1] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=7312
Re: getline() behaviour change
Bruno Haible [EMAIL PROTECTED] writes: Jim Meyering wrote: A *lot* of code expects malloc and realloc to set errno when they fail. Such code is not portable to plain ISO C 99 systems. We could extend gnulib's existing malloc/realloc wrappers to ensure that malloc/realloc set errno when they fail, if it is worthwhile to do so. -- Ben Pfaff http://benpfaff.org
Re: getline() behaviour change
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Bruno Haible on 8/22/2007 3:25 PM: - if (*lineptr == NULL || *n == 0) + if (*n == 0) This is a behaviour change: Previously when *lineptr == NULL, *n did not need to be initialized. Now it needs to be initialized to 0. Should be mentioned in NEWS. Hmm, I agree. But the wording the draft 3 of POSIX 200x requires *n to be initialized - if it is non-zero, then *lineptr must point to that many bytes, so *n must be 0 if *lineptr is NULL when used in a manner compliant with POSIX. Maybe it is worth a question on the Austin group mailing list as to whether this is intentional? Or do we go ahead and be generous by resetting *n to a sane value when *lineptr is NULL (ie. revert this one line)? - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGzNA784KuGfSFAYARAmQeAJ41dT8abUd991cOKHxciQIro8dCfgCgqNsv gxWCa3449T1Cv7vIpHiVU7s= =cqbs -END PGP SIGNATURE-
Re: getline() behaviour change
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Ben Pfaff on 8/22/2007 5:19 PM: Bruno Haible [EMAIL PROTECTED] writes: Jim Meyering wrote: A *lot* of code expects malloc and realloc to set errno when they fail. Such code is not portable to plain ISO C 99 systems. We could extend gnulib's existing malloc/realloc wrappers to ensure that malloc/realloc set errno when they fail, if it is worthwhile to do so. I would somewhat like this idea - it is much nicer assuming that malloc reliably sets errno to ENOMEM on failure than having to patch all callers of malloc to do the same. - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGzNCA84KuGfSFAYARAt1gAJ9/wJ1wo481LMUSJMdxyPSkzOTlogCcDwUt rh03w3E/qiRJWvYw58xfvK0= =PWXo -END PGP SIGNATURE-
getline: protect prototype better
How about this? If for some reason HAVE_DECL_GETLINE is not defined at all, the header file fail to parse. This uses the idiom that is suggested by the autoconf manual. 2005-08-23 Simon Josefsson [EMAIL PROTECTED] * getline.h (getline): Protect prototype better. --- getline.h 17 Jul 2005 11:39:14 +0200 1.16 +++ getline.h 23 Aug 2005 13:28:13 +0200 @@ -23,6 +23,6 @@ # include stdio.h # include sys/types.h -#if !HAVE_DECL_GETLINE +#if defined(HAVE_DECL_GETLINE) !HAVE_DECL_GETLINE ssize_t getline (char **lineptr, size_t *n, FILE *stream); #endif /* !HAVE_GETLINE */ ___ bug-gnulib mailing list bug-gnulib@gnu.org http://lists.gnu.org/mailman/listinfo/bug-gnulib
Re: getline: protect prototype better
Simon Josefsson [EMAIL PROTECTED] writes: How about this? If for some reason HAVE_DECL_GETLINE is not defined at all, the header file fail to parse. This uses the idiom that is suggested by the autoconf manual. 2005-08-23 Simon Josefsson [EMAIL PROTECTED] * getline.h (getline): Protect prototype better. --- getline.h 17 Jul 2005 11:39:14 +0200 1.16 +++ getline.h 23 Aug 2005 13:28:13 +0200 @@ -23,6 +23,6 @@ # include stdio.h # include sys/types.h -#if !HAVE_DECL_GETLINE +#if defined(HAVE_DECL_GETLINE) !HAVE_DECL_GETLINE A symbol that is not defined expands in the preprocessor to 0, so the former should parse properly. If it does not, then the latter will not parse properly either. The latter can be preferred because of its different meaning, not because it should parse properly when the former does not. Furthermore, in gnulib we know that the test will be performed, so I don't see a reason to prefer the latter. -- Ben Pfaff email: [EMAIL PROTECTED] web: http://benpfaff.org ___ bug-gnulib mailing list bug-gnulib@gnu.org http://lists.gnu.org/mailman/listinfo/bug-gnulib
Re: [bug-gnulib] getline: protect prototype better
Simon Josefsson wrote: How about this? If for some reason HAVE_DECL_GETLINE is not defined at all, the header file fail to parse. What's the point in trying to use lib/getline.c without m4/getline.m4 ? We engage ourselves to maintain both in sync, since they are part of a single gnulib module. Anyone using only the .c file but not the .m4 will have endless maintenance problems anyway. Bruno ___ bug-gnulib mailing list bug-gnulib@gnu.org http://lists.gnu.org/mailman/listinfo/bug-gnulib
Re: getline: protect prototype better
Bruno Haible [EMAIL PROTECTED] writes: Simon Josefsson wrote: This was also discovered while porting GNU SASL to uClinux. Perhaps I should investigate how difficult it would be to make uClinux run ./configure properly instead of maintaining the HAVE_* symbols manually. Definitely. uClinux has a shell and a compilation environment (and you can also cross-compile to it). That's exactly what autoconf was designed for. Right, although now I was incorporating GNU SASL into the uClinux build framework. All autoconf based programs that have been ported to uClinux appear to use hand-written Makefile's. The reason I went through the hassle of incorporating GNU SASL into the uClinux framework was to get the tools included in the uClinux generated ROMFS root file system automatically. But it is likely much simpler to cross-compile it first (which, incidentally, I've already done), and manually move the tools on to the root file system. I'll likely do that, thanks for the heads-up. Regards, Simon ___ bug-gnulib mailing list bug-gnulib@gnu.org http://lists.gnu.org/mailman/listinfo/bug-gnulib
Re: getline: protect prototype better
Simon Josefsson wrote: This was also discovered while porting GNU SASL to uClinux. Perhaps I should investigate how difficult it would be to make uClinux run ./configure properly instead of maintaining the HAVE_* symbols manually. Definitely. uClinux has a shell and a compilation environment (and you can also cross-compile to it). That's exactly what autoconf was designed for. Bruno ___ bug-gnulib mailing list bug-gnulib@gnu.org http://lists.gnu.org/mailman/listinfo/bug-gnulib
Re: getline
Paul Eggert [EMAIL PROTECTED] writes: That patch (in http://lists.gnu.org/archive/html/bug-gnulib/2005-07/msg00172.html) looks reasonable to me, though I haven't had time to test it myself. Please install it, and we can deal with any porting problems later. I installed the patch. Thanks, Simon ___ bug-gnulib mailing list bug-gnulib@gnu.org http://lists.gnu.org/mailman/listinfo/bug-gnulib
Re: getline
Simon Josefsson [EMAIL PROTECTED] writes: 1) Port the libc LGPL getdelim into a gnulib module. 2a) Port the libc LGPL getline into a gnulib module. 2b) Write a gnulib LGPL getline module that use the gnulib LGPL getdelim. 3) Keep the current getndelim2 as a GPL module. The libc getline seem somewhat different from libc getdelim; presumably it is faster than a simple getline implementation that use getdelim. I'm not sure this matter for gnulib though, so I prefer 2b for simplicity. 2b might be simpler in the short run, but will it be in the long run? Or are you assuming that libc getline won't ever change much? Finally, I'm not sure we can ever hope to sync these files between gnulib and libc. The libc implementation look inside the FILE structure. Gnulib code can't do that. It could do so conditionally. For example, it might use the stdio_ext.h interface on Solaris. http://docs.sun.com/app/docs/doc/816-5168/6mbb3hr5j?q=stdio_exta=view. However, you're the expert here, and if you've judged that 2b) is less work overall, perhaps we should just do that. We can always switch to 2a) later if need be. ___ bug-gnulib mailing list bug-gnulib@gnu.org http://lists.gnu.org/mailman/listinfo/bug-gnulib
Re: getline
This is a rough start, to let you see my thinking. If this is OK, I'll write ChangeLog entries for it. The getdelim in glibc appear to suffer from a missing overflow check in the realloc call, but I may be mistaken. I fixed that while porting the code below. 1. There are separate modules for getline and getdelim. 2. The getndelim2 module is not modified. 3. I'm not sure the M4 magic is kosher. 4. The M4 tests and building appear to work on GNU/Linux, NetBSD, DEC and HPUX. I haven't tested that the implementation works. Index: modules/getline === RCS file: /cvsroot/gnulib/gnulib/modules/getline,v retrieving revision 1.9 diff -u -p -r1.9 getline --- modules/getline 21 Mar 2005 22:07:25 - 1.9 +++ modules/getline 17 Jul 2005 10:09:58 - @@ -4,25 +4,21 @@ Read a line from a stream. Files: lib/getline.h lib/getline.c -lib/getndelim2.h -lib/getndelim2.c m4/getline.m4 -m4/getndelim2.m4 -m4/ssize_t.m4 Depends-on: +getdelim configure.ac: -AM_FUNC_GETLINE +gl_FUNC_GETLINE Makefile.am: -EXTRA_DIST += getndelim2.h getndelim2.c Include: getline.h License: -GPL +LGPL Maintainer: -all +Simon Josefsson Index: modules/getdelim === RCS file: modules/getdelim diff -N modules/getdelim --- /dev/null 1 Jan 1970 00:00:00 - +++ modules/getdelim17 Jul 2005 10:09:58 - @@ -0,0 +1,23 @@ +Description: +Read character delimited data from a stream. + +Files: +lib/getdelim.h +lib/getdelim.c +m4/getdelim.m4 + +Depends-on: + +configure.ac: +gl_FUNC_GETDELIM + +Makefile.am: + +Include: +getdelim.h + +License: +LGPL + +Maintainer: +Simon Josefsson Index: m4/getdelim.m4 === RCS file: m4/getdelim.m4 diff -N m4/getdelim.m4 --- /dev/null 1 Jan 1970 00:00:00 - +++ m4/getdelim.m4 17 Jul 2005 10:09:58 - @@ -0,0 +1,30 @@ +# getdelim.m4 serial 1 + +dnl Copyright (C) 2005 Free Software dnl Foundation, Inc. +dnl +dnl This file is free software; the Free Software Foundation +dnl gives unlimited permission to copy and/or distribute it, +dnl with or without modifications, as long as this notice is preserved. + +AC_PREREQ(2.52) + +AC_DEFUN([gl_FUNC_GETDELIM], +[ + AC_LIBSOURCES([getdelim.c, getdelim.h]) + + dnl Persuade glibc stdio.h to declare getdelim(). + AC_REQUIRE([AC_GNU_SOURCE]) + + AC_REPLACE_FUNCS(getdelim) + AC_CHECK_DECLS_ONCE(getdelim) + + if test $ac_cv_func_getdelim = no; then +gl_PREREQ_GETDELIM + fi +]) + +# Prerequisites of lib/getdelim.c. +AC_DEFUN([gl_PREREQ_GETDELIM], +[ + : +]) Index: m4/getline.m4 === RCS file: /cvsroot/gnulib/gnulib/m4/getline.m4,v retrieving revision 1.15 diff -u -p -r1.15 getline.m4 --- m4/getline.m4 21 Mar 2005 22:06:27 - 1.15 +++ m4/getline.m4 17 Jul 2005 10:09:58 - @@ -1,4 +1,4 @@ -# getline.m4 serial 12 +# getline.m4 serial 13 dnl Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2005 Free Software dnl Foundation, Inc. @@ -13,19 +13,21 @@ dnl See if there's a working, system-sup dnl We can't just do AC_REPLACE_FUNCS(getline) because some systems dnl have a function by that name in -linet that doesn't have anything dnl to do with the function we need. -AC_DEFUN([AM_FUNC_GETLINE], +AC_DEFUN([gl_FUNC_GETLINE], [ AC_LIBSOURCES([getline.c, getline.h]) - dnl Persuade glibc stdio.h to declare getline() and getdelim(). + dnl Persuade glibc stdio.h to declare getline(). AC_REQUIRE([AC_GNU_SOURCE]) - am_getline_needs_run_time_check=no + AC_CHECK_DECLS([getline]) + + gl_getline_needs_run_time_check=no AC_CHECK_FUNC(getline, dnl Found it in some library. Verify that it works. - am_getline_needs_run_time_check=yes, + gl_getline_needs_run_time_check=yes, am_cv_func_working_getline=no) - if test $am_getline_needs_run_time_check = yes; then + if test $gl_getline_needs_run_time_check = yes; then AC_CACHE_CHECK([for working getline function], am_cv_func_working_getline, [echo fooN |tr -d '\012'|tr N '\012' conftest.data AC_TRY_RUN([ @@ -57,20 +59,12 @@ AC_DEFUN([AM_FUNC_GETLINE], [Define to a replacement function name for getline().]) AC_LIBOBJ(getline) -# Avoid multiple inclusions of getndelim2.o into LIBOBJS. -# This hack won't be needed after gnulib requires Autoconf 2.58 or later. -case $LIB@[EMAIL PROTECTED] in -* getndelim2.$ac_objext * ) ;; -*) AC_LIBOBJ(getndelim2);; -esac - gl_PREREQ_GETLINE -gl_PREREQ_GETNDELIM2 fi ]) # Prerequisites of lib/getline.c. AC_DEFUN([gl_PREREQ_GETLINE], [ - AC_CHECK_FUNCS(getdelim) + gl_FUNC_GETDELIM ]) Index: lib/getdelim.c === RCS file: lib/getdelim.c diff -N lib/getdelim.c --- /dev/null 1 Jan
Re: getline
I'm following up on this thread, it may be useful (or not..) to rehash
it:
http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/2710
As you can see, I gave up and ported the glibc implementation into a
gnulib module and used that.
I read the libc getline and getdelim implementations now, and they use
memchr. gnulib's getndelim2 uses a loop, and may be slower for other
reasons too. I think libc getline/getdelim would be slowed down if we
moved getndelim2 into libc and made libc getline/getdelim use it.
That was Paul's plan, I think, assuming it was possible without
slow-down.
So here is what I propose:
1) Port the libc LGPL getdelim into a gnulib module.
2a) Port the libc LGPL getline into a gnulib module.
2b) Write a gnulib LGPL getline module that use the gnulib LGPL getdelim.
3) Keep the current getndelim2 as a GPL module.
The libc getline seem somewhat different from libc getdelim;
presumably it is faster than a simple getline implementation that use
getdelim. I'm not sure this matter for gnulib though, so I prefer 2b
for simplicity.
Finally, I'm not sure we can ever hope to sync these files between
gnulib and libc. The libc implementation look inside the FILE
structure. Gnulib code can't do that.
Incidentally, implementing this plan appear to be exactly what I did
for lessergnulib in December. The getdelim module (which is the only
substantial code needed in the above plan) look as below. Fairly
untested.
Is this an OK plan? If you agree, I'll propose a complete patch.
Specifically:
1. Split current getline module into a getndelim2 and getline.
2. Make the getline module LGPL by using getdelim code below, and
simple getline wrapper to call getdelim.
Thanks.
/* Copyright (C) 1994, 1996, 1997, 1998, 2001, 2003, 2004, 2005 Free
* Software Foundation, Inc.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
* 02111-1307, USA. */
/* Ported from libc by Simon Josefsson. */
#ifdef HAVE_CONFIG_H
# include config.h
#endif
/* Get specification. */
#include lgetdelim.h
/* Get malloc. */
#include stdlib.h
/* Read up to (and including) a TERMINATOR from FP into *LINEPTR
(and null-terminate it). *LINEPTR is a pointer returned from malloc (or
NULL), pointing to *N characters of space. It is realloc'ed as
necessary. Returns the number of characters read (not including the
null terminator), or -1 on error or EOF. */
ssize_t
getdelim (char **lineptr, size_t *n, int delimiter, FILE *fp)
{
ssize_t cur_len = 0;
if (lineptr == NULL || n == NULL)
return -1;
if (*lineptr == NULL || *n == 0)
{
*n = 120;
*lineptr = (char *) malloc (*n);
if (*lineptr == NULL)
return -1;
}
for (;;)
{
char *t;
int i;
i = getc (fp);
if (i == EOF)
break;
/* Make enough space for curlen+1 bytes plus last NUL. */
if (cur_len + 1 = *n)
{
size_t needed = 2 * (cur_len + 1) + 1; /* Be generous. */
char *new_lineptr;
if (needed cur_len)
return -1; /* overflow */
new_lineptr = realloc (*lineptr, needed);
if (new_lineptr == NULL)
return -1;
*lineptr = new_lineptr;
*n = needed;
}
(*lineptr)[cur_len] = i;
cur_len++;
if (i == delimiter)
break;
}
(*lineptr)[cur_len] = '\0';
return cur_len;
}
___
bug-gnulib mailing list
bug-gnulib@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-gnulib
