[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277031#comment-16277031
 ] 

ASF subversion and git services commented on PROTON-1540:
-

Commit 25efd103a84a666c9da5e05cad5fbd2d12edf6f4 in qpid-proton's branch 
refs/heads/master from [~aconway]
[ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=25efd10 ]

PROTON-1540: [go] disabled race detection by default (again)

NOTE: go test -race flag is not included by default, it causes problems on 
several platforms:
- ubuntu up to trust: link errors
- ubuntu from xenial: requires extra package golang-race-detector-runtime
- fedora with gccgo: complains about "import cycles"
(Works well on fedora with original go)
Enable manually with -DGO_TEST_FLAGS="-v -race"


> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Daněk
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[Alan Conway (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158816#comment-16158816
 ] 

Alan Conway commented on PROTON-1540:
-

The san RPM's don't install a plain libasan.so symlink, and there's no -devel 
RPM that does.  Cmake can't find the correct libsan.so.X version by the normal 
means. I guess that is to avoid accidentally preloading the wrong version.

The go binding had the race detector running before, it was removed because of 
CI problems on Ubuntu. I'll re-enable it and see if it is still a problem,  if 
it is I'll disable it selectively.


> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[Jiri Danek (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158770#comment-16158770
 ] 

Jiri Danek commented on PROTON-1540:


[~aconway] The only way I can think of is to create a "custom python 
interpreter" using libpython, which would run the tests. It should be possible 
to modify some embedded python tutorial to do this. This custom interpreter can 
then be linked with the sanitizers and should not need LD_PRELOAD.

I do not know how to detect path to sanitizer libs in CMake, in a portable way.

In case of go, there is https://golang.org/doc/articles/race_detector.html. It 
should be possible to enable that somehow in the build. I will try to have a 
look at it.

> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[Alan Conway (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158744#comment-16158744
 ] 

Alan Conway commented on PROTON-1540:
-

[~jdanek] The python, ruby and go bindings need LD_PRELOAD because the 
interpreters are not linked directly by clang/gcc (as our tests and examples 
are) which automatically adds the required link to lib*san when -fsanitize 
flags are present.

Maybe we can avoid the PRELOAD by explicitly linking those libs ourselves 
somewhere - but after a few experiments I'm not sure where. 
So PRELOAD is a good solution until we find a better one. We could add it to 
CMake but I'm worried about guessing  the library paths wrong and confusing 
everybody, since a bad LD_PRELOAD will cause an error message from literally 
every executable you run, not just our tests/examples. So for now I think it 
best to let the user set that.

> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157743#comment-16157743
 ] 

ASF subversion and git services commented on PROTON-1540:
-

Commit 742ccf9d9034ed0d2be1c459758cf2e93aae08f6 in qpid-proton's branch 
refs/heads/master from [~aconway]
[ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=742ccf9 ]

PROTON-1540: Don't disable bindings with sanitizers

Some of the bindings (GO, RUBY) currently don't work with sanitizers on,
but don't forcibly disable them - let the user do it.

Going forward we can probably fix the link problems and enable sanitizers with
these bindings.


> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[Jiri Danek (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156617#comment-16156617
 ] 

Jiri Danek commented on PROTON-1540:


Even with your options propagation fix, I still have this message coming from 
python-qpid-proton tests. Which I think is completely expected. There Python 
starts first, and it then loads the Proton lib when it is imported from Python 
code.

{noformat}
1: ==488==ASan runtime does not come first in initial library list; you should 
either link runtime to your application or manually preload it with LD_PRELOAD.
{noformat}

So, ctest should be running with LD_PRELOAD, or at least the python that is 
running the 1: tests, and then the sanitizer can sanitize the Python binding 
too.

There seem to be leaks in that. See ?? for report and what I believe is helpful 
command to isolate the leaks to individual tests (otherwise all are reported at 
the end when Python exits).

> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[Alan Conway (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155638#comment-16155638
 ] 

Alan Conway commented on PROTON-1540:
-

[~jdanek] after a lot of fiddling I discovered that the sanitize options were 
not being propagated to all of our executables, which I think is the cause of 
the messages about  using LD_PRELOAD. It looks to me that it Just Works now 
without LD_PRELOAD, but if you are seeing cases where it doesn't we'll add it 
to the developer doc somewhere.


> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155630#comment-16155630
 ] 

ASF GitHub Bot commented on PROTON-1540:


Github user asfgit closed the pull request at:

https://github.com/apache/qpid-proton/pull/117


> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155628#comment-16155628
 ] 

ASF subversion and git services commented on PROTON-1540:
-

Commit 33645b91468c6bf8a4253727dd74cfc85c70de05 in qpid-proton's branch 
refs/heads/master from [~aconway]
[ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=33645b9 ]

PROTON-1540 Add options to enable Sanitizers to CMake build

Enable with:

cmake -DENABLE_SANITZERS # enables memory sanitizers
cmake -DENABLE_TSAN  # enables thread sanitizer

Note TSAN cannot be used together with the other sanitizers.

Tested to work with gcc 5.4.0, 7.1.0 and clang 4.0.1

Modified original patch from Jiri Danek to propagate the sanitize flags to

This closes #117


> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>  Labels: patch
> Fix For: proton-c-0.18.0
>
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[Jiri Danek (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154836#comment-16154836
 ] 

Jiri Danek commented on PROTON-1540:


One addition. It is best to always run sanitizers with LD_PRELOAD. ASan will 
refuse to run if it is not loaded as the first thing. TSan can live with that, 
but it will find less issues. This is relevant when there is mixing of Python 
and C (and Python starts first). Some of it is in Proton tests (for Proton 
Python).

{noformat}
LD_PRELOAD=/path/to/gcc-7.1.0-lib/lib/libtsan.so TSAN_OPTIONS="color=always 
second_deadlock_stack=1" ctest -VV
{noformat}

> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Alan Conway
>Priority: Minor
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (PROTON-1540) Add options to enable Sanitizers to CMake build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126410#comment-16126410
 ] 

ASF GitHub Bot commented on PROTON-1540:


GitHub user jdanekrh opened a pull request:

https://github.com/apache/qpid-proton/pull/117

PROTON-1540 Add options to enable Sanitizers to CMake build



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jdanekrh/qpid-proton jd_sanitizers

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/qpid-proton/pull/117.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #117


commit 3f5385c898196312caa33606b3b2ac3d9827f2a2
Author: Jiri Danek 
Date:   2017-08-14T20:08:16Z

NO-JIRA Silence clang 4 error: use of GNU statement expression extension

This silences the following kind of warnings

/home/jdanek/Work/repos/qpid-proton/proton-c/src/tests/engine.c:283:5: 
error: use of GNU statement expression extension 
[-Werror,-Wgnu-statement-expression]
assert(pn_link_current(rx));
^

/nix/store/qy94v105wag3z9rgy1rb34zk0x20lkwj-glibc-2.25-dev/include/assert.h:95:6:
 note: expanded from macro 'assert'
({

commit c96107c0768ec98a6b5e5af5db56f918ee108cc5
Author: Jiri Danek 
Date:   2017-08-14T20:11:59Z

PROTON-1540 Add options to enable Sanitizers to CMake build

Tested to work with gcc 5.4.0, 7.1.0 and clang 4.0.1




> Add options to enable Sanitizers to CMake build
> ---
>
> Key: PROTON-1540
> URL: https://issues.apache.org/jira/browse/PROTON-1540
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: build
>Reporter: Jiri Danek
>Assignee: Jiri Danek
>Priority: Minor
>
> To get the most out of sanitizers in Qpid Dispatch, it helps to compile 
> Proton with sanitizers as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org