Re: [Valgrind-users] helgrind/drd annotations for one statement
Here are the threads I have:
{
pthread_rwlock_rdlock();
x = 1;
pthread_rwlock_rdunlock();
}
{
pthread_rwlock_wrlock();
if (x) {
x = 0;
} else {
foo();
}
pthread_rwlock_wrunlock();
}
Boehm's paper did not convince me of a bug.
Since pthread locks will include proper mfences I believe this to be correct
even on non-x86 machines.
Sent from my iPhone
On Dec 5, 2012, at 13:17, Brian Budge brian.bu...@gmail.com wrote:
It would be better to use atomics (with proper memory order), which can solve
this kind of problem without unintended consequences. As mentioned before,
this still may not be as maintenance friendly as locking, but at least it
will be correct.
Brian
On Wed, Dec 5, 2012 at 9:29 AM, Julian Seward jsew...@acm.org wrote:
On Tuesday, December 04, 2012, Patrick J. LoPresti wrote:
There is no such thing as a benign data race. Ever.
soapbox
Well said. I couldn't have put any of this better myself.
Having spent a considerable amount of time working on Helgrind and then
using it to chase races in some big hairy C++ codes, I became very
skeptical of the oh it's only a harmless race arguments. However,
I gave up shouting about it after a while since it just made me look
like a tiresome pedant hellbent on criticising people's clever go-faster-
by-avoiding-locking schemes.
IMO, the idea that some races are harmless is a dangerous myth that
we should try to stamp out.
I also noticed that threaded code that relies on unsynchronised
accesses is hard for newcomers to understand and reason about, so it
tends to be a maintenance hazard.
In some ways, the fact that the Intel architecture guarantees to
deliver stores in-order (x86-TSO), and is therefore somewhat tolerant
of such racery, is a disadvantage. I think people would take this
stuff more seriously if racey code got trashed more often by machines
with memory systems that do reordering, such as Power7.
/soapbox
J
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Valgrind-users mailing list
Valgrind-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/valgrind-users
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Valgrind-users mailing list
Valgrind-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/valgrind-users
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d___
Valgrind-users mailing list
Valgrind-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] helgrind/drd annotations for one statement
Sent from my iPhone
On Dec 5, 2012, at 15:38, Julian Seward jsew...@acm.org wrote:
On Wednesday, December 05, 2012, Leif Walsh wrote:
Here are the threads I have:
{
pthread_rwlock_rdlock();
x = 1;
pthread_rwlock_rdunlock();
}
{
pthread_rwlock_wrlock();
if (x) {
x = 0;
} else {
foo();
}
pthread_rwlock_wrunlock();
}
Boehm's paper did not convince me of a bug.
Since pthread locks will include proper mfences I believe this to be
correct even on non-x86 machines.
Well, but what's your definition of correct? That's central to the
issue at hand.
Imagine that pthread_rwlock_rdunlock contains a one-sided fence,
that causes all writes done by other processors to be visible on
this processor, but has no effect on migrating writes done by this
processor to others. (Why should it? Holding a reader lock doesn't
allow you to write shared memory, so a conforming program should
have no writes that need to be propagated to other processors at
lock release time.)
The important synchronization point isn't the rdunlock, it's the wrlock.
I have a hard time believing that you can take a pthread write lock and then
look at a value some other processor wrote before you took the lock and not get
that value. That sounds like pthread is broken or I am being exceedingly thick.
But everything I have read on this since I started asking around says that my
code is fine.
I agree that races that seem benign can easily stop being benign. I just don't
think this is a race according to the spec.
The above is just a guess. I'm not saying your code will fail on current
implementations. Merely that (as Patrick points out), non-adherence
to the standard puts the program pretty much in the Russian Roulette
department w.r.t. future hardware and compiler developments.
You may remember the gcc -fstrict-aliasing wars of the early 2000s,
in which the gcc developers starting using a relatively obscure aspect
of the C/C++ standard to improve optimisation. As a result vast reams
of previously-working code (including Valgrind) started to fail in
obscure ways.
Same thing happened on a smaller scale a couple of years later when
gcc started to assume that signed integer arithmetic would never overflow.
So .. seen from a historical context, the compiler crews are an extremely
ingenious bunch, and I would not be at all surprised to see newer compilers
causing non-compliant (racey) code to fail more often.
J
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Valgrind-users mailing list
Valgrind-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] helgrind/drd annotations for one statement
Rdunlock happens before wrlock. Sent from my iPhone On Dec 5, 2012, at 16:51, David Faure fa...@kde.org wrote: On Wednesday 05 December 2012 16:39:47 Leif Walsh wrote: The important synchronization point isn't the rdunlock, it's the wrlock. Well, you need two locks, for a happens-before relationship to be established. If you remove the rdlock/rdunlock completely (since it could basically be a no-op for a write operation anyway, as others pointed out earlier), then this will be more clear: this write might never become visible to the other thread. I have a hard time believing that you can take a pthread write lock and then look at a value some other processor wrote before you took the lock and not get that value. You say before, but this assumes a global ordering, which you don't get, when not using atomics or the proper locks. Each CPU can have a different notion of before, without the correct synchronization primitives. I recommend reading C++ Concurrency in action by Anthony Williams, it taught me a lot on all this all works... definitely not a simple topic. -- David Faure, fa...@kde.org, http://www.davidfaure.fr Working on KDE, in particular KDE Frameworks 5 -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] helgrind/drd annotations for one statement
Sent from my iPhone
On Dec 5, 2012, at 16:31, Patrick J. LoPresti lopre...@gmail.com wrote:
On Wed, Dec 5, 2012 at 10:29 AM, Leif Walsh leif.wa...@gmail.com wrote:
Here are the threads I have:
{
pthread_rwlock_rdlock();
x = 1;
pthread_rwlock_rdunlock();
}
{
pthread_rwlock_wrlock();
if (x) {
x = 0;
} else {
foo();
}
pthread_rwlock_wrunlock();
}
I am going to repeat myself a bit because this is an important point.
Then I will give up.
First, this code smells on its surface. If I were interviewing a
candidate who submitted this as a code sample, it would be an easy
strong no hire decision.
Ouch.
Grabbing a reader lock to write a
variable? What the heck is that?
It's an eviction heuristic. It doesn't even need to be perfectly accurate.
No matter how smart you think you are, sooner or later, you will work
with people who lack your genius. Truly smart programmers write code
that any idiot can prove correct. This does not qualify, to put it
mildly.
Second, as we keep trying to explain, concurrent writes to the same
location are *undefined behavior* under every common threading model
(including, in particular, POSIX and C++11). That means a compiler
looking at this code can PROVE that only one thread ever runs the
first block concurrently. If that is not true, then you have
introduced a contradiction into the compiler's reasoning. From a
contradiction, anything follows. How badly this code breaks depends
only on how smart your compiler is, and they get smarter all the time.
Try to sell a compiler that generates a random executable just because it
proves undefined behavior.
I'm not trying to be (that) snarky, just realistic. I understand the risk
you're describing and I appreciate you taking the time to do so. But adding
extra fences here just because some theoretical compiler might turn this code
into a giraffe drawing program is not a useful way for me or the CPU running my
program to spend our time.
Boehm's paper did not convince me of a bug.
When dealing with multi-threading, the right question is not Do I
fail to see the problem? The right question is Can I create a
simple proof that there is no problem?
In my experience, most programmers either get this right away, or they
never will.
Since pthread locks will include proper mfences I believe this to be correct
even on non-x86 machines.
(a) You believe incorrectly, because undefined behavior can result in
anything whatsoever.
(b) Even if you were correct, the chain of reasoning to prove it is
too long for maintainable code.
Anyway, Helgrind is perfectly correct to warn about this bug.
It is.
- Pat
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Valgrind-users mailing list
Valgrind-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] helgrind/drd annotations for one statement
Sent from my iPhone On Dec 4, 2012, at 4:49, Julian Seward jsew...@acm.org wrote: On Monday, December 03, 2012, Leif Walsh wrote: I have a memory location whose behavior I'd like to verify with helgrind and/or drd, but which I access in a racy way sometimes. I would like to ignore the race I know about but still have it checked for other accesses. Is this possible? The race is that, while a thread has a read lock, it may set the value to 1, but it does not read the value. While a thread has a write lock, it may read the value and it may set the value to 0. The race is that many threads can set it to 1 if they all have the read lock. DRD and helgrind tend to view this as a race. Correctly, IMO. Why do you think this behaviour is OK ? If you have a read lock, you can read the location, but not write it. It is a race, but one that has the same result as if everything were serialized. If three threads all set this bit to 1, that's ok. It's ok because they're all trying to write the same thing. You're right, it is correctly flagged as a race, I'm just hoping to ask it to not report this one. Were you trying to achieve some larger goal, such as (at a guess) the thread with the write lock creates a description of some work to be done (in an object in memory) and multiple threads with read locks compete to do that work, and write into the location you mentioned, in order to show that they have completed it? J -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] helgrind/drd annotations for one statement
Sent from my iPhone On Dec 4, 2012, at 9:33, Julian Seward jsew...@acm.org wrote: It is a race, but one that has the same result as if everything were serialized. If three threads all set this bit to 1, that's ok. It's ok because they're all trying to write the same thing. Do you have suitable memory fences in place, so it won't break in mysterious ways on targets that deliver stores out-of-order to other processors, eg Power7? I'm ashamed to say I don't quite understand what can fail here but I'm not worried until I start supporting such architectures. Thanks for the warning though. Anyway .. You're right, it is correctly flagged as a race, I'm just hoping to ask it to not report this one. You might be best off removing the magic macros shown in your first posting, and instead using Valgrind's suppression mechanism to hide precisely the error(s) you don't want to see. Ok, thanks. J -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
[Valgrind-users] helgrind/drd annotations for one statement
I have a memory location whose behavior I'd like to verify with helgrind
and/or drd, but which I access in a racy way sometimes. I would like to
ignore the race I know about but still have it checked for other accesses.
Is this possible?
The race is that, while a thread has a read lock, it may set the value to
1, but it does not read the value. While a thread has a write lock, it may
read the value and it may set the value to 0. The race is that many
threads can set it to 1 if they all have the read lock. DRD and helgrind
tend to view this as a race.
I tried something like this, but wasn't surprised that it didn't work:
{
VALGRIND_HG_DISABLE_CHECKING(var, sizeof var);
DRD_IGNORE_VAR(var);
var = 1;
DRD_STOP_IGNORING_VAR(var);
VALGRIND_HG_ENABLE_CHECKING(var, sizeof var);
}
I can eliminate the race report if I disable checking on the variable
always, but I don't want to lose all that checking if I can avoid it.
--
Cheers,
Leif
--
Keep yourself connected to Go Parallel:
BUILD Helping you discover the best ways to construct your parallel projects.
http://goparallel.sourceforge.net___
Valgrind-users mailing list
Valgrind-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] Variable Tracing
Maybe you can construct something with gdb watch points. I haven't tried scripting an event to occur on a watch point or breakpoint but it seems like the sort of thing gdb should be good at Sent from my iPhone On Oct 3, 2012, at 10:47, Pansy Arafa par...@uwaterloo.ca wrote: Hi, How can I use Valgrind to trace variables? i.e. extract the values of certain variables during the program execution. I know I can use gdb with Valgrind, but I don't want to interrupt the execution. I want Valgrind to write the value of the variable to console or a log file once the variable gets updated. Thanks. -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
[Valgrind-users] DRD client request to disable CondRaceErr
I want to signal a pthread condition variable without holding the associated mutex. DRD (correctly) flags this as an error, but I want to suppress the error. I'd like to do it with a client request like DRD_IGNORE_VAR. I tried putting this on the cond and the mutex, and the error is still reported. Using --gen-suppressions won't work because there are lots of different stacks that trigger this, and the optimizer, especially with -flto, messes with the function names in a different way each time we build. Is there any other way to suppress this error? I see ANNOTATE_CONDVAR_* but they are all no-ops. -- Cheers, Leif -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] DRD client request to disable CondRaceErr
On Tue, Aug 14, 2012 at 2:55 PM, Julian Seward jsew...@acm.org wrote: Hmm, you really really want to do that and 110% understand the possible races that could result? Yep, I really do want to do that. I really do not care about the race. I suppose I could switch to a semaphore or something and hope that makes it go away, but I would rather tell valgrind what I want than change my code to fit valgrind. I looked at a situation like this in NSPR some time back and found it extremely difficult to convince myself that the code was really safe. J -- Cheers, Leif -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] DRD client request to disable CondRaceErr
This may work, I forgot you could use wildcards in function names. I'll come back if it doesn't. On Tue, Aug 14, 2012 at 3:46 PM, Julian Seward jsew...@acm.org wrote: On Tuesday, August 14, 2012, Leif Walsh wrote: because there are lots of different stacks that trigger this, and the optimizer, especially with -flto, messes with the function names in a different way each time we build. Does it mess with the names so much that you can't use the * and ? wildcards, and the ... frame level wildcard, to reduce the number of suppressions to a manageable number? J -- Cheers, Leif -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
