Re: [PATCH] git_open_noatime: return with errno=0 on success
On Tue, Aug 4, 2015 at 11:03 PM, Junio C Hamano gits...@pobox.com wrote: I would agree it is a good idea to clear it after seeing the first open fail due to lack of O_NOATIME before trying open for the second time, iow, more like this? So I don't think this is _wrong_ per se, but I think the deeper issue is that somebody cares about 'errno' here in the first place. A stale 'errno' generally shouldn't matter, because we either (a) return success (and nobody should look at errno) or (b) return an error later, without setting errno for that _later_ error. and I think either of those two situations are the real bug, and this clear stale errno is just a workaround. But as mentioned, I don't think clearign errno is wrong, so I'm not objecting to the patch. I just suspect there's something else goign on too.. Linus -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git_open_noatime: return with errno=0 on success
On Wed, Aug 05, 2015 at 10:59:09AM +0200, Linus Torvalds wrote: On Tue, Aug 4, 2015 at 11:03 PM, Junio C Hamano gits...@pobox.com wrote: I would agree it is a good idea to clear it after seeing the first open fail due to lack of O_NOATIME before trying open for the second time, iow, more like this? Looks good to me. So I don't think this is _wrong_ per se, but I think the deeper issue is that somebody cares about 'errno' here in the first place. A stale 'errno' generally shouldn't matter, because we either (a) return success (and nobody should look at errno) or (b) return an error later, without setting errno for that _later_ error. and I think either of those two situations are the real bug, and this clear stale errno is just a workaround. I agree. But I do not see how to get there easily. We are trying to read an object. We first try to read from a pack. We may encounter broken pack files, missing index files, unreadable files, but those errors are not necessarily fatal since we may still be able to read the object from the next pack file or from a sha1 file. If finally we do not find the object anywhere, in read_sha1_file_extended we try our best to die with an appropriate error message, for example by looking at errno, and otherwise we just return NULL. Most callers seem to die explicitly or they dereference the null pointer. I think we should instead output error messages closer to the source, like for example in map_sha1_file, but continue anyway. In particular we should immediately report failures due to EPERM or unexpected ENOENT. In the end we may return NULL without another message, but at least the user should have some hints about what went wrong along the way. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git_open_noatime: return with errno=0 on success
Clemens Buchacher clemens.buchac...@intel.com writes: On Wed, Aug 05, 2015 at 10:59:09AM +0200, Linus Torvalds wrote: ... A stale 'errno' generally shouldn't matter, because we either (a) return success (and nobody should look at errno) or (b) return an error later, without setting errno for that _later_ error. and I think either of those two situations are the real bug, and this clear stale errno is just a workaround. I agree. But I do not see how to get there easily. We are trying to read an object. We first try to read from a pack. We may encounter broken pack files, missing index files, unreadable files, but those errors are not necessarily fatal since we may still be able to read the object from the next pack file or from a sha1 file. If finally we do not find the object anywhere, in read_sha1_file_extended we try our best to die with an appropriate error message, for example by looking at errno, and otherwise we just return NULL. Most callers seem to die explicitly or they dereference the null pointer. I think we should instead output error messages closer to the source, like for example in map_sha1_file, but continue anyway. Hmm, if we find one data source unreadable but an alternative usable, do we really want that error message? What should it say? error: cannot read from pack? Such a message, unless we later give info: but we managed to read it from elsewhere and make sure these two messages are clearly associated with each other, would make things unnecessarily alarming, wouldn't it? Perhaps we should not rely so heavily on 'errno', but explicitly pass around error code (or enough information to formulate an intelligent message at the end) in the callchain instead. Then the earlier part can notice EPERM on a pack, for example, and return to the caller, and after consulting an alternate data source (e.g. loose object file), the caller can then choose to say we managed to read the data, but FYI, you may want to check the permission bits of this pack, or choose to stay silent. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git_open_noatime: return with errno=0 on success
In read_sha1_file_extended we die if read_object fails with a fatal
error. We detect a fatal error if errno is non-zero and is not
ENOENT. If the object could not be read because it does not exist,
this is not considered a fatal error and we want to return NULL.
Somewhere down the line, read_object calls git_open_noatime to open
a pack index file, for example. We first try open with O_NOATIME.
If O_NOATIME fails with EPERM, we retry without O_NOATIME. When the
second open succeeds, errno is however still set to EPERM from the
first attempt. When we finally determine that the object does not
exist, read_object returns NULL and read_sha1_file_extended dies
with a fatal error:
fatal: failed to read object sha1: Operation not permitted
Fix this by resetting errno to zero before we call open again.
Cc: Linus Torvalds torva...@linux-foundation.org
Signed-off-by: Clemens Buchacher clemens.buchac...@intel.com
---
This is a re-submission without changes except for a typo fix in the
comments (thanks Eric). The original submission received no other
comments, but I think it is a clear improvement and I hope it was just
missed the first time.
Best regards,
Clemens
sha1_file.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/sha1_file.c b/sha1_file.c
index 77cd81d..62b7ad6 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1453,6 +1453,7 @@ int git_open_noatime(const char *name)
static int sha1_file_open_flag = O_NOATIME;
for (;;) {
+ errno = 0;
int fd = open(name, O_RDONLY | sha1_file_open_flag);
if (fd = 0)
return fd;
--
1.9.4
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git_open_noatime: return with errno=0 on success
Clemens Buchacher clemens.buchac...@intel.com writes:
diff --git a/sha1_file.c b/sha1_file.c
index 77cd81d..62b7ad6 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1453,6 +1453,7 @@ int git_open_noatime(const char *name)
static int sha1_file_open_flag = O_NOATIME;
for (;;) {
+ errno = 0;
int fd = open(name, O_RDONLY | sha1_file_open_flag);
Please avoid decl-after-stmt, which this codebase does not accept.
if (fd = 0)
return fd;
More importantly, is this the right place to clear errno?
I would agree it is a good idea to clear it after seeing the first
open fail due to lack of O_NOATIME before trying open for the second
time, iow, more like this?
sha1_file.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/sha1_file.c b/sha1_file.c
index 1cee438..bf2f229 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1467,6 +1467,7 @@ int git_open_noatime(const char *name)
/* Might the failure be due to O_NOATIME? */
if (errno != ENOENT sha1_file_open_flag) {
+ errno = 0;
sha1_file_open_flag = 0;
continue;
}
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git_open_noatime: return with errno=0 on success
In read_sha1_file_extended we die if read_object fails with a fatal
error. We detect a fatal error if errno is non-zero and is not
ENOENT. If the object could not be read because it does not exist,
this is not considered a fatal error and we want to return NULL.
Somewhere down the line, read_object calls git_open_noatime to open
a pack index file, for example. We first try open with O_NOATIME.
If O_NOATIME fails with EPERM, we retry without O_NOATIME. When the
second open succeeds, errno is however still set to EPERM from the
first attemt. When we finally determine that the object does not
exist, read_object returns NULL and read_sha1_file_extended dies
with a fatal error:
fatal: failed to read object sha1: Operation not permitted
Fix this by resetting errno to zero before we call open again.
Cc: Linus Torvalds torva...@linux-foundation.org
Signed-off-by: Clemens Buchacher clemens.buchac...@intel.com
Helped-by: Martin Schröder martin.h.schroe...@intel.com
---
sha1_file.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/sha1_file.c b/sha1_file.c
index 77cd81d..62b7ad6 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1453,6 +1453,7 @@ int git_open_noatime(const char *name)
static int sha1_file_open_flag = O_NOATIME;
for (;;) {
+ errno = 0;
int fd = open(name, O_RDONLY | sha1_file_open_flag);
if (fd = 0)
return fd;
--
1.9.4
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git_open_noatime: return with errno=0 on success
On Wed, Jul 8, 2015 at 8:38 AM, Clemens Buchacher clemens.buchac...@intel.com wrote: In read_sha1_file_extended we die if read_object fails with a fatal error. We detect a fatal error if errno is non-zero and is not ENOENT. If the object could not be read because it does not exist, this is not considered a fatal error and we want to return NULL. Somewhere down the line, read_object calls git_open_noatime to open a pack index file, for example. We first try open with O_NOATIME. If O_NOATIME fails with EPERM, we retry without O_NOATIME. When the second open succeeds, errno is however still set to EPERM from the first attemt. When we finally determine that the object does not s/attemt/attempt/ exist, read_object returns NULL and read_sha1_file_extended dies with a fatal error: fatal: failed to read object sha1: Operation not permitted Fix this by resetting errno to zero before we call open again. Cc: Linus Torvalds torva...@linux-foundation.org Signed-off-by: Clemens Buchacher clemens.buchac...@intel.com Helped-by: Martin Schröder martin.h.schroe...@intel.com -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
