Bug#901062: qtbase5-dev: qtbase5-dev: QFile::exists() returns false for existing files?!
Found the bug.
Docker uses seccomp security profiles to restrict e.g. system calls [1].
In docker 17.05 system call "statx" is blocked [2][3], in trunk and in
18.04 which is not released yet that system call will be allowed [4][5].
By running the docker container with "--security-opt seccomp=unconfined"
or "--privileged" the system call "statx" not blocked [5] and everything
works as excepted: QFile::exists() returns true for existing files.
Older revisions of QFile (e.g. qt 5.7 from debian stretch) do work as
excepted even when statx is blocked, because they do not use system call
"statx".
That system call is not necessary to find out if a file exists. Qt is
able to detect whether "statx" is supported. But it fails to detect that
its call is not permitted. The reason for that is in
src/corelib/io/qfilesystemengine_unix.cpp which is still present in
latest upstream revision [6]. But lets start from the beginning:
QFile::exists() returns true if ExistsFlag is set [7]. Attributes are
set in "QFSFileEnginePrivate::doStat" [10]. This function first reads
attributes with 2-args-function-overload
"QFileSystemEngine::fillMetaData" [11] and then loads the missing
attributes with 3-args-function-overload
"QFileSystemEngine::fillMetaData" [12]. Boths overloads suffer from the
same bug.
The 2-args-overload "bool QFileSystemEngine::fillMetaData(int fd,
QFileSystemMetaData )" [11] calls "qt_fstatx". QtCore in debian sid
was build with a kernel that supports the statx system call and thus
function "qt_fstatx" does indeed call "syscall(SYS_statx, ...)". If this
syscall is blocked, then "syscall(...)" returns -1 and errno is set to 1
(EPERM == "Operation not permitted") [13]. Back in
"QFileSystemEngine::fillMetaData" [11] look at this code snippet:
int ret = qt_fstatx(fd, );
if (ret != -ENOSYS) {
if (ret == 0) {
data.fillFromStatxBuf(statxBuffer);
return true;
}
return false;
}
If statx is blocked, then ret==-EPERM (negated in [14]), which means
"return false" is executed, thus no attributes are read.
"QFSFileEnginePrivate::doStat" now executes the 3-args-overload "bool
QFileSystemEngine::fillMetaData(const QFileSystemEntry ,
QFileSystemMetaData , QFileSystemMetaData::MetaDataFlags what)"
[12]. It contains this snippet:
statResult = qt_lstatx(nativeFilePath, );
if (statResult == -ENOSYS) {
//...
} else if (statResult == 0) {
//...
}
if (statResult >= 0) {
//...
} else {
// it doesn't exist
entryErrno = errno;
data.knownFlagsMask |= QFileSystemMetaData::ExistsAttribute;
}
When statx is blocked then statResult is -1 (-EPERM) and thus last
branch is executed. Thus 3-args-overload of fillMetaData says "i know
that file does not exist" (knownFlagsMask has ExistsAttribute but
entryFlags has not). The code "data.entryFlags |= flag |
QFileSystemMetaData::ExistsAttribute;" later in that function [16] is
never executed because entryErrno equals -1 (-EPERM).
Thus QFile always returns false, for all files, even existing once, as
long as statx is supported but blocked via e.g. seccomp.
Cheers,
Jakob
[1] https://github.com/Microsoft/docker/blob/master/docs/security/seccomp.md
[2] debian package "docker-engine" currently available for stretch is
17.05.0~ce-0~debian-stretch
[3] https://github.com/moby/moby/blob/17.05.x/profiles/seccomp/default.json
[4] https://github.com/moby/moby/blob/master/profiles/seccomp/default.json
[5] https://github.com/docker/for-linux/issues/208
[6]
https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfilesystemengine_unix.cpp
[7] https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfile.cpp#n425
[8] -DELETED-
[9] -DELETED-
[10]
https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfsfileengine_unix.cpp#n419
[11]
https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfilesystemengine_unix.cpp#n439
[12]
https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfilesystemengine_unix.cpp#n931
[13] /usr/include/asm-generic/errno-base.h
[14]
https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfilesystemengine_unix.cpp#n350
[15]
https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfilesystemengine_unix.cpp#n979
[16]
https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfilesystemengine_unix.cpp#n1053
Am 08.06.2018 um 20:54 schrieb Dmitry Shachnev:
> Hi Jakob!
>
> On Fri, Jun 08, 2018 at 04:01:54PM +0200, Wearenotalone wrote:
>> did a fresh install of debian sid within docker and stumbled upon this
>> bug in QFile:
>> QFile's method exists() returns false even if that file exists.
> Thanks for your bug report and the test case.
>
> Can you please obtain strace of your example and attach it?
> I am not able to reproduce this issue, but I would like to compare yours
> and mine strace outputs.
>
> --
> Dmitry Shachnev
Bug#901062: qtbase5-dev: qtbase5-dev: QFile::exists() returns false for existing files?!
[big snip] >> >> > in 4.13. A fallback mechanism might be codable, but someone needs to do >> the >> > job. >> >> I think you are mistaken here. Qt depends on getrandom(2) syscall that was >> added in kernel 3.17 (see #895718), but that is not related to >> QFile::exists() >> in any way. >> >> Regarding this issue, I cannot reproduce it yet. > > > I was told it produces another "interesting" side effects which do not > appear to be related. > > But testing with a newer kernel should be enough. > Sorry, I clearly mixed kernel versions. Please forget what I've said. >
Bug#901062: qtbase5-dev: qtbase5-dev: QFile::exists() returns false for existing files?!
Hi Jakob! On Fri, Jun 08, 2018 at 04:01:54PM +0200, Wearenotalone wrote: > did a fresh install of debian sid within docker and stumbled upon this > bug in QFile: > QFile's method exists() returns false even if that file exists. Thanks for your bug report and the test case. Can you please obtain strace of your example and attach it? I am not able to reproduce this issue, but I would like to compare yours and mine strace outputs. -- Dmitry Shachnev signature.asc Description: PGP signature
Bug#901062: qtbase5-dev: qtbase5-dev: QFile::exists() returns false for existing files?!
El vie., 8 de jun. de 2018 15:44, Dmitry Shachnev escribió: > Hi Lisandro! > > On Fri, Jun 08, 2018 at 01:34:12PM -0300, Lisandro Damián Nicanor Pérez > Meyer wrote: > > -- System Information: > > > Debian Release: buster/sid > > > APT prefers unstable > > > APT policy: (500, 'unstable') > > > Architecture: amd64 (x86_64) > > > > > > Kernel: Linux 4.9.0-6-amd64 (SMP w/4 CPU cores) > > > > Old kernel. Qt requests an interface that, if I'm not mistaken, was added > > in 4.13. A fallback mechanism might be codable, but someone needs to do > the > > job. > > I think you are mistaken here. Qt depends on getrandom(2) syscall that was > added in kernel 3.17 (see #895718), but that is not related to > QFile::exists() > in any way. > > Regarding this issue, I cannot reproduce it yet. I was told it produces another "interesting" side effects which do not appear to be related. But testing with a newer kernel should be enough.
Bug#901062: qtbase5-dev: qtbase5-dev: QFile::exists() returns false for existing files?!
Hi Lisandro! On Fri, Jun 08, 2018 at 01:34:12PM -0300, Lisandro Damián Nicanor Pérez Meyer wrote: > -- System Information: > > Debian Release: buster/sid > > APT prefers unstable > > APT policy: (500, 'unstable') > > Architecture: amd64 (x86_64) > > > > Kernel: Linux 4.9.0-6-amd64 (SMP w/4 CPU cores) > > Old kernel. Qt requests an interface that, if I'm not mistaken, was added > in 4.13. A fallback mechanism might be codable, but someone needs to do the > job. I think you are mistaken here. Qt depends on getrandom(2) syscall that was added in kernel 3.17 (see #895718), but that is not related to QFile::exists() in any way. Regarding this issue, I cannot reproduce it yet. -- Dmitry Shachnev signature.asc Description: PGP signature
Bug#901062: qtbase5-dev: qtbase5-dev: QFile::exists() returns false for existing files?!
Hi! El vie., 8 de jun. de 2018 11:12, Wearenotalone escribió: > Package: qtbase5-dev > Version: 5.10.1+dfsg-7 > Severity: normal > > > Dear Maintainer, > > did a fresh install of debian sid within docker and stumbled upon this > bug in QFile: > QFile's method exists() returns false even if that file exists. > [snip] BUT surprisingly KDE apps like akregator have successfully been build > for debian sid but fail for me/my sid install. They should have suffered > the same problems during build because they also use desktoptojson > during build. > > Any ideas? > Yes -- System Information: > Debian Release: buster/sid > APT prefers unstable > APT policy: (500, 'unstable') > Architecture: amd64 (x86_64) > > Kernel: Linux 4.9.0-6-amd64 (SMP w/4 CPU cores) > Old kernel. Qt requests an interface that, if I'm not mistaken, was added in 4.13. A fallback mechanism might be codable, but someone needs to do the job.
Bug#901062: qtbase5-dev: qtbase5-dev: QFile::exists() returns false for existing files?!
Package: qtbase5-dev
Version: 5.10.1+dfsg-7
Severity: normal
Dear Maintainer,
did a fresh install of debian sid within docker and stumbled upon this
bug in QFile:
QFile's method exists() returns false even if that file exists.
STEPS TO REPRODUCE:
$ cat << 'EOF' >> /tmp/CMakeLists.txt
cmake_minimum_required(VERSION 3.0)
project(test)
find_package(Qt5Core REQUIRED)
add_executable(tgt main.cpp)
target_link_libraries(tgt Qt5::Core)
EOF
$ cat << 'EOF' >> /tmp/main.cpp
#include
#include
#include
#include
int main() {
QString path{"/bin/sh"};
QFile qf{path};
QFileInfo qfi{path};
std::cout << "TRUE:=" << true << std::endl;
std::cout << "QFile::exists() == " << qf.exists() << std::endl;
std::cout << "QFileInfo::exists() == " << qfi.exists() << std::endl;
return 0;
}
EOF
$ cmake .
-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/
$ make
Scanning dependencies of target tgt
[ 50%] Building CXX object CMakeFiles/tgt.dir/main.cpp.o
[100%] Linking CXX executable tgt
[100%] Built target tgt
$ ./tgt
TRUE:=1
QFile::exists() == 0
QFileInfo::exists() == 1
EXPECTED RESULT (same as in debian stretch):
$ ./tgt
TRUE:=1
QFile::exists() == 1
QFileInfo::exists() == 1
QFile is quite fundamental to Qt and KDE. For me it prevented
compilation of kdevelop from git because kdevelop uses desktoptojson
(from libkf5coreaddons-dev-bin) during build which fails due to this bug
(QFile is used in line 312 in file
kcoreaddons-5.46.0/src/lib/plugin/desktopfileparser.cpp)..
So far I was unable to locate the bug. QFile's code
(src/corelib/io/qfile.cpp) didn't change much from
qtbase-opensource-src-5.7.1 (stretch) to qtbase-opensource-src-5.10.1
(sid), so my guess is its somewhere in
/src/corelib/io/qabstractfileengine.cpp or even lower.
BUT surprisingly KDE apps like akregator have successfully been build
for debian sid but fail for me/my sid install. They should have suffered
the same problems during build because they also use desktoptojson
during build.
Any ideas?
Cheers
Jakob
-- System Information:
Debian Release: buster/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)
Kernel: Linux 4.9.0-6-amd64 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968), LANGUAGE=C
(charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash
Init: unable to detect
Versions of packages qtbase5-dev depends on:
ii libgl1-mesa-dev [libgl-dev] 18.0.5-1
ii libglu1-mesa-dev [libglu-dev] 9.0.0-2.1
ii libqt5concurrent5 5.10.1+dfsg-7
ii libqt5core5a 5.10.1+dfsg-7
ii libqt5dbus5 5.10.1+dfsg-7
ii libqt5gui5 5.10.1+dfsg-7
ii libqt5network5 5.10.1+dfsg-7
ii libqt5printsupport5 5.10.1+dfsg-7
ii libqt5sql5 5.10.1+dfsg-7
ii libqt5test5 5.10.1+dfsg-7
ii libqt5widgets5 5.10.1+dfsg-7
ii libqt5xml5 5.10.1+dfsg-7
ii libxext-dev 2:1.3.3-1+b2
ii qt5-qmake 5.10.1+dfsg-7
ii qtbase5-dev-tools 5.10.1+dfsg-7
ii qtchooser 64-ga1b6736-5
Versions of packages qtbase5-dev recommends:
ii libqt5opengl5-dev 5.10.1+dfsg-7
Versions of packages qtbase5-dev suggests:
ii default-libmysqlclient-dev 1.0.4
pn firebird-dev
ii libegl1-mesa-dev 18.0.5-1
ii libgl1-mesa-dev 18.0.5-1
ii libpq-dev 10.4-2
ii libsqlite3-dev 3.23.1-1
ii unixodbc-dev 2.3.6-0.1
-- no debconf information
