[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091
--- Comment #9 from CVS Commits ---
The releases/gcc-8 branch has been updated by Jonathan Wakely
:
https://gcc.gnu.org/g:56b6c2ba3a45c768a33f726b130a56fc19cca650
commit r8-10084-g56b6c2ba3a45c768a33f726b130a56fc19cca650
Author: Jonathan Wakely
Date: Thu Jan 9 13:38:43 2020 +
Build filesystem library with large file support
Enable AC_SYS_LARGEFILE to set the macros needed for large file APIs to
be used by default. We do not want to define those macros in the
public headers that users include. The values of the macros are copied
to a separate file that is only included by the filesystem sources
during the build, and then the macros in are renamed
so that they don't have any effect in user code including our headers.
Also use larger type for result of filesystem::file_size to avoid
truncation of large values on 32-bit systems (PR 91947).
Backport from mainlne
2019-10-04 Jonathan Wakely
PR libstdc++/81091
PR libstdc++/91947
* configure.ac: Use AC_SYS_LARGEFILE to enable 64-bit file APIs.
* config.h.in: Regenerate:
* configure: Regenerate:
* include/Makefile.am (${host_builddir}/largefile-config.h): New
target to generate config header for filesystem library.
(${host_builddir}/c++config.h): Rename macros for large file support.
* include/Makefile.in: Regenerate.
* src/c++17/fs_dir.cc: Include new config header.
* src/c++17/fs_ops.cc: Likewise.
(filesystem::file_size): Use uintmax_t for size.
* src/filesystem/dir.cc: Include new config header.
* src/filesystem/ops.cc: Likewise.
(experimental::filesystem::file_size): Use uintmax_t for size.
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091 Jonathan Wakely changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution|--- |FIXED --- Comment #8 from Jonathan Wakely --- Fixed for GCC 9.3, but I don't plan to fix this on the gcc-8 branch.
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091
--- Comment #7 from CVS Commits ---
The releases/gcc-9 branch has been updated by Jonathan Wakely
:
https://gcc.gnu.org/g:6cb662745d38e680a1a46fa04b108734cbc3df58
commit r9-8130-g6cb662745d38e680a1a46fa04b108734cbc3df58
Author: Jonathan Wakely
Date: Thu Jan 9 13:38:43 2020 +
Build filesystem library with large file support
Enable AC_SYS_LARGEFILE to set the macros needed for large file APIs to
be used by default. We do not want to define those macros in the
public headers that users include. The values of the macros are copied
to a separate file that is only included by the filesystem sources
during the build, and then the macros in are renamed
so that they don't have any effect in user code including our headers.
Also use larger type for result of filesystem::file_size to avoid
truncation of large values on 32-bit systems (PR 91947).
Backport from mainlne
2019-10-04 Jonathan Wakely
PR libstdc++/81091
PR libstdc++/91947
* configure.ac: Use AC_SYS_LARGEFILE to enable 64-bit file APIs.
* config.h.in: Regenerate:
* configure: Regenerate:
* include/Makefile.am (${host_builddir}/largefile-config.h): New
target to generate config header for filesystem library.
(${host_builddir}/c++config.h): Rename macros for large file support.
* include/Makefile.in: Regenerate.
* src/c++17/fs_dir.cc: Include new config header.
* src/c++17/fs_ops.cc: Likewise.
(filesystem::file_size): Use uintmax_t for size.
* src/filesystem/dir.cc: Include new config header.
* src/filesystem/ops.cc: Likewise.
(experimental::filesystem::file_size): Use uintmax_t for size.
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091 Jonathan Wakely changed: What|Removed |Added Target Milestone|10.0|9.3
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091 Jonathan Wakely changed: What|Removed |Added Target Milestone|--- |10.0 --- Comment #6 from Jonathan Wakely --- Fixed on trunk so far. This should be OK to backport too, but I'll let it bake on trunk for a while.
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091
--- Comment #5 from Jonathan Wakely ---
Author: redi
Date: Fri Oct 4 15:08:23 2019
New Revision: 276585
URL: https://gcc.gnu.org/viewcvs?rev=276585=gcc=rev
Log:
Build filesystem library with large file support
Enable AC_SYS_LARGEFILE to set the macros needed for large file APIs to
be used by default. We do not want to define those macros in the
public headers that users include. The values of the macros are copied
to a separate file that is only included by the filesystem sources
during the build, and then the macros in are renamed
so that they don't have any effect in user code including our headers.
Also use larger type for result of filesystem::file_size to avoid
truncation of large values on 32-bit systems (PR 91947).
PR libstdc++/81091
PR libstdc++/91947
* configure.ac: Use AC_SYS_LARGEFILE to enable 64-bit file APIs.
* config.h.in: Regenerate:
* configure: Regenerate:
* include/Makefile.am (${host_builddir}/largefile-config.h): New
target to generate config header for filesystem library.
(${host_builddir}/c++config.h): Rename macros for large file support.
* include/Makefile.in: Regenerate.
* src/c++17/fs_dir.cc: Include new config header.
* src/c++17/fs_ops.cc: Likewise.
(filesystem::file_size): Use uintmax_t for size.
* src/filesystem/dir.cc: Include new config header.
* src/filesystem/ops.cc: Likewise.
(experimental::filesystem::file_size): Use uintmax_t for size.
Modified:
trunk/libstdc++-v3/ChangeLog
trunk/libstdc++-v3/config.h.in
trunk/libstdc++-v3/configure
trunk/libstdc++-v3/configure.ac
trunk/libstdc++-v3/include/Makefile.am
trunk/libstdc++-v3/include/Makefile.in
trunk/libstdc++-v3/src/c++17/fs_dir.cc
trunk/libstdc++-v3/src/c++17/fs_ops.cc
trunk/libstdc++-v3/src/filesystem/dir.cc
trunk/libstdc++-v3/src/filesystem/ops.cc
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091 --- Comment #4 from Richard Biener --- (In reply to Jonathan Wakely from comment #3) > (In reply to Richard Biener from comment #0) > > libstdc++ seems to lack AC_SYS_LARGEFILE in configury and thus uses > > fopen/open > > in fstream and friends that can fail not only because of large files but > > files with large inode numbers depending on the underlying filesystem. > > Richi, are you sure about that? I think I looked at all unresolved symbols of libstdc++.so and there were some not using the LFS interfaces. I didn't actually see you are "manually" selecting the API.
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091 Jonathan Wakely changed: What|Removed |Added Status|NEW |ASSIGNED Assignee|unassigned at gcc dot gnu.org |redi at gcc dot gnu.org
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091
--- Comment #3 from Jonathan Wakely ---
(In reply to Richard Biener from comment #0)
> libstdc++ seems to lack AC_SYS_LARGEFILE in configury and thus uses
> fopen/open
> in fstream and friends that can fail not only because of large files but
> files with large inode numbers depending on the underlying filesystem.
Richi, are you sure about that?
config/io/basic_file_stdio.cc does:
#ifdef _GLIBCXX_USE_LFS
if ((_M_cfile = fopen64(__name, __c_mode)))
#else
if ((_M_cfile = fopen(__name, __c_mode)))
#endif
And that macro should be set during configure, via:
dnl
dnl Check whether LFS support is available.
dnl
AC_DEFUN([GLIBCXX_CHECK_LFS], [
AC_LANG_SAVE
AC_LANG_CPLUSPLUS
ac_save_CXXFLAGS="$CXXFLAGS"
CXXFLAGS="$CXXFLAGS -fno-exceptions"
AC_MSG_CHECKING([for LFS support])
AC_CACHE_VAL(glibcxx_cv_LFS, [
GCC_TRY_COMPILE_OR_LINK(
[#include
#include
#include
],
[FILE* fp;
fopen64("t", "w");
fseeko64(fp, 0, SEEK_CUR);
ftello64(fp);
lseek64(1, 0, SEEK_CUR);
struct stat64 buf;
fstat64(1, );],
[glibcxx_cv_LFS=yes],
[glibcxx_cv_LFS=no])
])
if test $glibcxx_cv_LFS = yes; then
AC_DEFINE(_GLIBCXX_USE_LFS, 1, [Define if LFS support is available.])
fi
AC_MSG_RESULT($glibcxx_cv_LFS)
CXXFLAGS="$ac_save_CXXFLAGS"
AC_LANG_RESTORE
])
So I think fstream is OK.
However, there are some bugs elsewhere due to not checking _GLIBCXX_USE_LFS and
so not using the largefile APIs:
src/c++11/random.cc:_M_fd = ::open(fname, O_RDONLY);
src/c++11/random.cc:_M_file = static_cast(std::fopen(fname, "rb"));
This should be OK in practice, because fname is guaranteed to be either
"/dev/random" or "/dev/urandom" (it seems unlikely that those devices will have
inode numbers that don't fit in 32 bits).
Nothing in src/filesystem and std/c++17/fs_* checks _GLIBCXX_USE_LFS or uses
the LARGEFILE64 APIs. I think those files should be built with
_FILE_OFFSET_BITS=64 so that stat, lstat, open, truncate etc. handle large
files automatically, without having to remember to use the LARGEFILE64 versions
explicitly.
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091 Jonathan Wakely changed: What|Removed |Added Status|UNCONFIRMED |NEW Last reconfirmed||2018-05-04 Ever confirmed|0 |1 --- Comment #2 from Jonathan Wakely --- I was concerned this would be an ABI change for 32-bit targets, but libstdc++ already ensures that std::streamoff is a 64-bit type even when ::off_t isn't. So now I think building it with large file support should not change the type of anything in the libstdc++ API or the exported ABI.
[Bug libstdc++/81091] libstdc++ not built with large file support
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81091 Jc Yang changed: What|Removed |Added CC||bandinfinite at gmail dot com --- Comment #1 from Jc Yang --- This bug also affects std::filesystem when the capacity of modern storage devices are progressing in a fast pace. Without 64bit _FILE_OFFSET_BITS, and consider that there's no trends among the hardware vendors to migrate to a block size larger than 4096 in the foreseeable future, not even the fixed in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85632 can tell the user of a 32bit system the correct size of a partition that is as large as 20TB. I think we need this as the ultimate fix and should force it no matter whether it is a 64bit build or not.
