[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2015-12-15 Thread wschmidt at gcc dot gnu.org
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975

--- Comment #39 from Bill Schmidt  ---
I believe this work has been completed.  Peter, do you concur?

[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2015-12-15 Thread bergner at gcc dot gnu.org
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975

Peter Bergner  changed:

   What|Removed |Added

 Status|UNCONFIRMED |RESOLVED
 CC||bergner at gcc dot gnu.org
 Resolution|--- |FIXED

--- Comment #40 from Peter Bergner  ---
(In reply to Bill Schmidt from comment #39)
> I believe this work has been completed.  Peter, do you concur?

Agreed.

[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-24 Thread wschmidt at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #37 from William J. Schmidt wschmidt at gcc dot gnu.org 
2013-01-24 13:30:53 UTC ---

(In reply to comment #36)

 Bill, tests in Instrumentation/AddressSanitizer are compiler-only tests

 and thus are mostly platform independent.

 The interesting part is the run-time tests that can be run with make

 check-asan

 (only in cmake build)

 These build on PPC64 today and some of them even pass, but not all.



OK.  Please let me know if we can assist setting up a PPC bot in the future, to

help maintain compatibility.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-24 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #38 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-24 
13:46:17 UTC ---

 OK.  Please let me know if we can assist setting up a PPC bot in the future, 
 to

 help maintain compatibility.



Oh, that'll be great even now.

If you can setup a bot that builds asan-runtime for PPC32 and PPC64 

and runs the test, it will be awesome! 



Just let the bot execute these steps in order: 

make -j 64 clang_rt.asan-powerpc64

make -j 64 clang_rt.asan-powerpc

make -j 64 Asan-powerpc64-Test

make -j 64 Asan-powerpc-Test

make -j 64 check-asan 



The last step is red now, but we'll fix it eventually. 



We may even need two bots, for 44-bit and 46-bit AS. 



(as it's taking us away from this bug, perhaps we could continue by e-mail)


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #25 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-23 
12:21:51 UTC ---

So, what is our decision? 



Are we just doing 

-  static const uptr kHighMemEnd = 0x0fffUL;

+  static const uptr kHighMemEnd = 0x3fffUL;

, leaving SHADOW_OFFSET=(1ULL  41)

and using ADD instead of OR when applying SHADOW_OFFSET? 



This seems to work on my ppc box (44-bit) with LLVM

(I've changed it to use ADD on PPC)


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #26 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-23 
12:26:30 UTC ---

(In reply to comment #25)

 So, what is our decision? 

 

 Are we just doing 

 -  static const uptr kHighMemEnd = 0x0fffUL;

 +  static const uptr kHighMemEnd = 0x3fffUL;

 , leaving SHADOW_OFFSET=(1ULL  41)

 and using ADD instead of OR when applying SHADOW_OFFSET? 

 

 This seems to work on my ppc box (44-bit) with LLVM

 (I've changed it to use ADD on PPC)



If that works, it is my preference.  But needs testing, also with GCC, and with

both 44-bit and 46-bit AS.



BTW, I wonder why clang generates larger and slower code with ADD instead of

OR, at least gcc seems to generate generally better code with ADD, plus on

i?86/x86_64 it even has better HW support for that (for ADD can use both

add{l,q} and leal, allowing RA to generate better code).  GCC for asan

generates always ADD, on all architectures, right now.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #27 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-23 
12:51:40 UTC ---

 BTW, I wonder why clang generates larger and slower code with ADD instead of 
 OR

I did not investigate. I noticed that the code size with OR is smaller than

with ADD. 



One more thing to take care of with powerpc: the new allocator needs a change

like this:

--- asan/asan_allocator2.cc(revision 173248)

+++ asan/asan_allocator2.cc(working copy)

@@ -57,7 +57,11 @@

 };



 #if SANITIZER_WORDSIZE == 64

+#if defined(__powerpc64__)

+const uptr kAllocatorSpace =  0xa00ULL;

+#else

 const uptr kAllocatorSpace = 0x6000ULL;

+#endif

 const uptr kAllocatorSize  =  0x100ULL;  // 1T.

 typedef DefaultSizeClassMap SizeClassMap;

 typedef SizeClassAllocator64kAllocatorSpace, kAllocatorSize, 0 /*metadata*/,





And one more thing: if we set kHighMemEnd = 0x3fffUL

asan will stop working in 44-bit AS with unlimited stack: 



% (ulimit -s unlimited; clang -fsanitize=address use-after-free.cc ;

ASAN_OPTIONS=verbosity=1 ./a.out )

==37704== Parsed ASAN_OPTIONS: verbosity=1

==37704== AddressSanitizer: libc interceptors initialized

|| `[0x0a00, 0x3fff]` || HighMem||

|| `[0x0340, 0x09ff]` || HighShadow ||

|| `[0x0240, 0x033f]` || ShadowGap  ||

|| `[0x0200, 0x023f]` || LowShadow  ||

|| `[0x, 0x01ff]` || LowMem ||

MemToShadow(shadow): 0x0240 0x0247 0x0268

0x033f

red_zone=16

malloc_context_size=30

SHADOW_SCALE: 3

SHADOW_GRANULARITY: 8

SHADOW_OFFSET: 200

==37704== Shadow memory range interleaves with an existing memory mapping. ASan

cannot proceed correctly. ABORTING.

==37704== Process memory map follows:

0x1000-0x1003   /tmp/a.out

0x1003-0x1005   /tmp/a.out

0x1005-0x12db   

0x0400-0x0403   /lib64/ld-2.13.so

0x0403-0x0404   /lib64/ld-2.13.so

0x0404-0x0406   [vdso]

0x0407-0x0409   /lib64/libpthread-2.13.so

0x0409-0x040a   /lib64/libpthread-2.13.so

0x040a-0x040b   

0x040b-0x040c   /lib64/libdl-2.13.so

0x040c-0x040d   /lib64/libdl-2.13.so

0x040d-0x040f   /lib64/libgcc_s.so.1

0x040f-0x0410   /lib64/libgcc_s.so.1

0x0410-0x0429   /lib64/libc-2.13.so

0x0429-0x042a   /lib64/libc-2.13.so

0x042a-0x042c   /lib64/libc-2.13.so

0x042c-0x0432   

0x0fffd2df-0x0fffd2e2   [stack]

==37704== End of process memory map.



So, perhaps we still need dynamic kHighMemEnd.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #28 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-23 
13:05:57 UTC ---

Why doesn't it error for unlimited stack say on x86_64?  If the stack mapping

size is unlimited, it also potentially overlaps with the shadow memory.  If you

have a growsdown mapping, it simply needs to be capped by the end of the shadow

memory area resp. start of the high memory region.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #29 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-23 
13:19:30 UTC ---

 Why doesn't it error for unlimited stack say on x86_64? 

Because on x86_64 the stack is still high enough (we are lucky). 



Note: I would not generally care about unlimited stack, 

but for some obscure reason GNU make spawns its sub-processes with unlimited

stack.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #30 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-23 
13:30:19 UTC ---

What I mean, is a stack PROT_GROWSDOWN RLIM_INFINITY RLIMIT_STACK mapping is

defined to be a mapping from the top of that mapping down to the first mapping

of something else below it.  Whether it is the shadow memory or something else

doesn't matter.  There is just no overlap in that case, so the error looks

wrong to me.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #31 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-23 
13:32:11 UTC ---

I've committed an upstream change

http://llvm.org/viewvc/llvm-project?view=revrevision=173260 that makes

kHighMemEnd dynamic. 

Hopefully, it will simplify further changes. 



I'd appreciate if someone who has access to a 46-bit AS machine 

propose a patch that works for both 44 and 46 AS w/ and w/o unlimited stack.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #32 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-23 
13:33:45 UTC ---

I take it back, seems it is because the kernel mmaps the shared libraries in

that range in this case.  So indeed, you probably need a dynamic mapping, sorry

for the noise.  All that just to grow VA by 2 bits?  Urgh.  On x86_64 at least

the address space was growing much more many years ago.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #33 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-23 
13:40:57 UTC ---

(In reply to comment #31)

 I've committed an upstream change

 http://llvm.org/viewvc/llvm-project?view=revrevision=173260 that makes

 kHighMemEnd dynamic. 

 Hopefully, it will simplify further changes. 

 

 I'd appreciate if someone who has access to a 46-bit AS machine 

 propose a patch that works for both 44 and 46 AS w/ and w/o unlimited stack.



Do you really want to make kHighMemEnd dynamic always?  Shouldn't it be dynamic

only when needed (i.e. for configurations like ppc64 which now require it)?

Otherwise it will slow down many of the inlines that use this heavily.

Furthermore, in configurations where kHighMemEnd is dynamic, also IMHO all

asan_mapping.h defines that are based on it should also be global variables

to avoid unnecessary runtime computations everywhere.

I mean kHighMemBeg, kHighShadowBeg, kHighShadowEnd, kShadowGapEnd.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #34 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-23 
13:50:31 UTC ---

 Do you really want to make kHighMemEnd dynamic always?  Shouldn't it be 
 dynamic

 only when needed (i.e. for configurations like ppc64 which now require it)?



If we make this dynamic on PPC and constant elsewhere, we risk breaking

PPC frequently because we typically don't test on PPC (we don't even have a

bot!).

We can easily introduce a change that will assume kHighMemEnd is a constant. 



 Otherwise it will slow down many of the inlines that use this heavily.



Hopefully, not that bad. 

unless an application is malloc-bound, we don't enter asan-rt too often. 

If it is malloc-bound, we are in trouble anyway, because the allocator is very

expensive in all other parts. 



 Furthermore, in configurations where kHighMemEnd is dynamic, also IMHO all

 asan_mapping.h defines that are based on it should also be global variables

 to avoid unnecessary runtime computations everywhere.

 I mean kHighMemBeg, kHighShadowBeg, kHighShadowEnd, kShadowGapEnd.



Hm... I would not change that w/o some proof (benchmarks results). 

When e.g. we access two of these variables, we will be doing two loads instead

of one load and arithmetic. Not a clear choice.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread wschmidt at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



William J. Schmidt wschmidt at gcc dot gnu.org changed:



   What|Removed |Added



 CC||wschmidt at gcc dot gnu.org



--- Comment #35 from William J. Schmidt wschmidt at gcc dot gnu.org 
2013-01-23 16:25:27 UTC ---

(In reply to comment #34)

 If we make this dynamic on PPC and constant elsewhere, we risk breaking

 PPC frequently because we typically don't test on PPC (we don't even have a

 bot!).



Well, although the ppc64 LNT bot is not quite functional yet, there are many of

us who run the LLVM unit test suite several times a day.  So as long as the

changes are covered with tests in Instrumentation/{Address,Thread}Sanitizer,

we'll definitely notice PPC64 breakage and let you know. ;)



Bill


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-23 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #36 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-24 
07:19:00 UTC ---

 Well, although the ppc64 LNT bot is not quite functional yet, there are many 
 of

 us who run the LLVM unit test suite several times a day.  So as long as the

 changes are covered with tests in Instrumentation/{Address,Thread}Sanitizer,

 we'll definitely notice PPC64 breakage and let you know. ;)



Bill, tests in Instrumentation/AddressSanitizer are compiler-only tests

and thus are mostly platform independent.

The interesting part is the run-time tests that can be run with make

check-asan

(only in cmake build)

These build on PPC64 today and some of them even pass, but not all.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-21 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #24 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-21 
09:11:22 UTC ---

I've tried adding the offset instead of OR-ing it and got 2%-5% slowdown and

same code size increase on SPEC on x86_64 (using fresh clang -02). 

So, yes, if we want to keep the shadow offset 2^41 on PPC, we need to add the

offset, while still OR-ing it on other architectures. 



It's sad we can't use zero base on PPC.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #14 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-18 
08:26:00 UTC ---

Because of this: 

#define kLowShadowBeg   SHADOW_OFFSET

#define kLowShadowEnd   MEM_TO_SHADOW(kLowMemEnd)



If we leave kLowShadowEnd as before, we will have too little shadow for the 44

bit address space. 



Maybe it's still better to make kHighMemEnd dynamic...


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #15 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-18 
11:40:57 UTC ---

 There are two off-by-one:



In LLVM trunk: http://llvm.org/viewvc/llvm-project?rev=172807view=rev



Initially I wanted to do a merge to gcc this week, but I think I better wait a

bit -- there were too many intrusive changes in the trunk last week. 



BTW, many things will get simpler if PowerPC-specific bugs are 

tracked against the upstream version (file bugs in 

https://code.google.com/p/address-sanitizer/issues/list against the LLVM

version of the run-time)


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #16 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-18 
12:30:11 UTC ---

-#define SHADOW_OFFSET (1ULL  41)

+#define SHADOW_OFFSET (1ULL  43)



On my ppc box which uses 44 bit address space this does not work: 



% ASAN_OPTIONS=verbosity=1 ./a.out

|| `[0x1000, 0x3fff]` || HighMem||

|| `[0x0a00, 0x0fff]` || HighShadow ||

|| `[0x0900, 0x09ff]` || ShadowGap  ||

|| `[0x0800, 0x08ff]` || LowShadow  ||

|| `[0x, 0x07ff]` || LowMem ||

==36991== Shadow memory range interleaves with an existing memory mapping. ASan

cannot proceed correctly. ABORTING.

==36991== Process memory map follows:

...

0x04ad-0x04b0   [stack]





Note: 

0x0fff HighShadow

0x04b0 [stack]



So, we need a bit more flexible solution.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #17 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-18 
13:02:41 UTC ---

But why doesn't the 1  41 version work with both 44 and 46-bit VA?

It should be:

   || `[0x0a00, 0x3fff]` || HighMem||

   || `[0x0340, 0x09ff]` || HighShadow ||

   || `[0x0240, 0x033f]` || ShadowGap  ||

   || `[0x0200, 0x023f]` || LowShadow  ||

   || `[0x, 0x01ff]` || LowMem ||

then, which IMHO should work just fine even if 0x1000 through

0x3fff is unavailable.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread sch...@linux-m68k.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #18 from Andreas Schwab sch...@linux-m68k.org 2013-01-18 13:21:02 
UTC ---

(In reply to comment #15)

 In LLVM trunk: http://llvm.org/viewvc/llvm-project?rev=172807view=rev



That only happens when running under gdb which appears to disable address

randomisation.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread sch...@linux-m68k.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #19 from Andreas Schwab sch...@linux-m68k.org 2013-01-18 13:25:42 
UTC ---

MEM_TO_SHODOW does not work when (SHADOW_OFFSET  SHADOW_SHIFT) is less than

kHighMemEnd.



$ ASAN_OPTIONS=verbosity=1 ./clone-test-1.exe

==16778== Parsed ASAN_OPTIONS: verbosity=1

==16778== AddressSanitizer: libc interceptors initialized

|| `[0x0800, 0x3fff]` || HighMem||

|| `[0x0300, 0x07ff]` || HighShadow ||

|| `[0x0240, 0x02ff]` || ShadowGap  ||

|| `[0x0200, 0x023f]` || LowShadow  ||

|| `[0x, 0x01ff]` || LowMem ||

MemToShadow(shadow): 0x0240 0x0247 0x0260

0x02ff

red_zone=128

malloc_context_size=30

SHADOW_SCALE: 3

SHADOW_GRANULARITY: 8

SHADOW_OFFSET: 200

==16778== Installed the sigaction for signal 11

==16778== T0: stack [0x3fffcebc9000,0x3fffcf3c9000) size 0x80;

local=0x3fffcf3c6990

==16778== AddressSanitizer Init done

ASAN:SIGSEGV

=

==16778== ERROR: AddressSanitizer: SEGV on unknown address 0x09fff9e78d62 (pc

0x1d10 sp 0x3fffcf3c6a70 bp 0x3fffcf3c6a70 T0)

AddressSanitizer can not provide additional info.

#0 0x1d0c

(/home/andreas/src/gcc/ppc/gcc/testsuite/gcc/clone-test-1.exe+0x1d0c)

Stats: 0M malloced (0M for red zones) by 0 calls

Stats: 0M realloced by 0 calls

Stats: 0M freed by 0 calls

Stats: 0M really freed by 0 calls

Stats: 0M (0M-0M) mmaped; 0 maps, 0 unmaps

  mmaps   by size class: 

  mallocs by size class: 

  frees   by size class: 

  rfrees  by size class: 

Stats: malloc large: 0 small slow: 0

Stats: StackDepot: 0 ids; 0M mapped

==16778== ABORTING


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #20 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-18 
13:52:02 UTC ---

Ah yes, it can't be | SHADOW_OFFSET then, but has to be + SHADOW_OFFSET.  + is

what gcc emits (the reason for that was primarily that it resulted in better

code on x86_64/i686).  So perhaps for targets where SHADOW_OFFSET is non-zero

and below (or equal to) 12.5% of maximum user address space, we are free to use

either | or +, but for other targets (for now ppc64) it must use +?


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #21 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-18 
15:33:33 UTC ---

While we are at it... 

It is possible to use zero shadow offset of powerpc somehow? 



on x86_64 linux, when I build the code with -fPIC -pie

the mapping looks like this: 



7fb6dda49000-7fb6ddbfe000 r-xp  fc:00 1208459   

/lib/x86_64-linux-gnu/libc-2.15.so

...

7fff01eba000-7fff01edb000 rw-p  00:00 0 

[stack]

7fff01f15000-7fff01f16000 r-xp  00:00 0  [vdso]

ff60-ff601000 r-xp  00:00 0 

[vsyscall]



which allows us to occupy 16T of memory starting from zero for the shadow. 



On my power pc box, however, I get this mapping: 

36ec-36ed r-xp  08:02 3213429   

/tmp/a.out

36ed-36ee rw-p  08:02 3213429   

/tmp/a.out

fff7f36-fff7f37 rw-p  00:00 0 

...

fffe70d-fffe710 rw-p  00:00 0   

[stack]



So, I can't use zero-based shadow. 



On x86_64 using zero-based shadow gives 5% speedup. It's worth trying on ppc as

well, I think.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #22 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-18 
15:41:09 UTC ---

Forcing everything to be built as -pie is a non-starter.  The 5% just aren't

worth the trouble.  Especially, how are you going to deal with say a

-fsanitize=address shared library built with 1  44 shadow offset and a PIE

binary built with 0 shadow offset?


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-18 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #23 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-18 
15:46:00 UTC ---

in our use-case 5% of run-time (and code size too) is a lot. 

We are currently in the process of migrating the chrome builds to zero based

offset (chrome already uses -pie).



Mixing objects build with different options is, of course, not supported. 



On Android (where we are especially restricted with code size and address

space, but where we also control our destiny) the zero based shadow is the only

choice.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-17 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #10 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-17 
08:30:05 UTC ---

Since I can't reproduce the failure,

please help me debug this (basically, print all local variables around the

assertion and the proc maps) or suggest me how to reproduce. 



I've got access to a ppc machine with 3.3.8 Linux, but it uses 

44 bit address space and basic asan functionality seems to work 

(I am working with LLVM trunk, not with GCC version, but that should not

matter)


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-17 Thread sch...@linux-m68k.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #11 from Andreas Schwab sch...@linux-m68k.org 2013-01-17 23:23:47 
UTC ---

There are two off-by-one:



diff --git a/libsanitizer/asan/asan_poisoning.cc

b/libsanitizer/asan/asan_poisoning.cc

index a00baff..bbbaf0a 100644

--- a/libsanitizer/asan/asan_poisoning.cc

+++ b/libsanitizer/asan/asan_poisoning.cc

@@ -23,7 +23,7 @@ void PoisonShadow(uptr addr, uptr size, u8 value) {

   CHECK(AddrIsAlignedByGranularity(addr));

   CHECK(AddrIsAlignedByGranularity(addr + size));

   uptr shadow_beg = MemToShadow(addr);

-  uptr shadow_end = MemToShadow(addr + size);

+  uptr shadow_end = MemToShadow(addr + size - 1) + 1;

   CHECK(REAL(memset) != 0);

   REAL(memset)((void*)shadow_beg, value, shadow_end - shadow_beg);

 }

diff --git a/libsanitizer/asan/asan_thread.cc

b/libsanitizer/asan/asan_thread.cc

index cc2e777..02f49dd 100644

--- a/libsanitizer/asan/asan_thread.cc

+++ b/libsanitizer/asan/asan_thread.cc

@@ -72,7 +72,7 @@ void AsanThread::Destroy() {

 void AsanThread::Init() {

   SetThreadStackTopAndBottom();

   CHECK(AddrIsInMem(stack_bottom_));

-  CHECK(AddrIsInMem(stack_top_));

+  CHECK(AddrIsInMem(stack_top_ - 1));

   ClearShadowForThreadStack();

   if (flags()-verbosity = 1) {

 int local = 0;


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-17 Thread sch...@linux-m68k.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #12 from Andreas Schwab sch...@linux-m68k.org 2013-01-17 23:52:39 
UTC ---

Created attachment 29199

  -- http://gcc.gnu.org/bugzilla/attachment.cgi?id=29199

make check-gcc RUNTESTFLAGS='--target_board=unix/-m64 asan.exp'



diff --git a/gcc/config/rs6000/rs6000.c b/gcc/config/rs6000/rs6000.c

index 46bb0b4..23c8c7f 100644

--- a/gcc/config/rs6000/rs6000.c

+++ b/gcc/config/rs6000/rs6000.c

@@ -27609,7 +27609,7 @@ rs6000_final_prescan_insn (rtx insn, rtx *operand

ATTRIBUTE_UNUSED,

 static unsigned HOST_WIDE_INT

 rs6000_asan_shadow_offset (void)

 {

-  return (unsigned HOST_WIDE_INT) 1  (TARGET_64BIT ? 41 : 29);

+  return (unsigned HOST_WIDE_INT) 1  (TARGET_64BIT ? 43 : 29);

 }

 #endif



diff --git a/libsanitizer/asan/asan_mapping.h

b/libsanitizer/asan/asan_mapping.h

index 54e21b7..6c22709 100644

--- a/libsanitizer/asan/asan_mapping.h

+++ b/libsanitizer/asan/asan_mapping.h

@@ -32,7 +32,7 @@ extern SANITIZER_INTERFACE_ATTRIBUTE uptr

__asan_mapping_offset;

 #   define SHADOW_OFFSET (1  29)

 #  else

 #   if defined(__powerpc64__)

-#define SHADOW_OFFSET (1ULL  41)

+#define SHADOW_OFFSET (1ULL  43)

 #   else

 #define SHADOW_OFFSET (1ULL  44)

 #   endif

@@ -46,7 +46,7 @@ extern SANITIZER_INTERFACE_ATTRIBUTE uptr

__asan_mapping_offset;



 #if SANITIZER_WORDSIZE == 64

 # if defined(__powerpc64__)

-  static const uptr kHighMemEnd = 0x0fffUL;

+  static const uptr kHighMemEnd = 0x3fffUL;

 # else

   static const uptr kHighMemEnd = 0x7fffUL;

 # endif


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-16 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



Kostya Serebryany kcc at gcc dot gnu.org changed:



   What|Removed |Added



Summary|FAIL:   |asan does not work with 46

   |c-c++-common/asan/global-ov |bit address space on

   |erflow-1.c  -O0  output |PowerPC64

   |pattern test|



--- Comment #6 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-16 
11:47:38 UTC ---

  we'd need to make size of shadow memory region and kHighMemEnd dynamic.



Probably so. 

We already have a macro ASAN_FLEXIBLE_MAPPING_AND_OFFSET that makes the 

SHADOW_SCALE and SHADOW_OFFSET dynamic. 

We'll soon make this the default since we want to use the zero base for asan 

on linux more widely (under a flag). 



What is the best way to compute kHighMemEnd at startup (anything better than

reading proc maps?)


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-16 Thread jakub at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #7 from Jakub Jelinek jakub at gcc dot gnu.org 2013-01-16 
11:50:47 UTC ---

I think for 44-46 bits we can still make it constant.  But generally, the

constructors of libasan are usually run from the stack of the initial thread,

so it should be enough to look at address of any local variable and check if it

is around (1  44) - epsilon or (1  46) - epsilon.


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-16 Thread kcc at gcc dot gnu.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #8 from Kostya Serebryany kcc at gcc dot gnu.org 2013-01-16 
11:54:36 UTC ---

Sounds good for both. 



Andreas, could you please try replacing 

kHighMemEnd = 0x0fffUL

with 

kHighMemEnd = 0x3fffUL

and see if it helps?


[Bug sanitizer/55975] asan does not work with 46 bit address space on PowerPC64

2013-01-16 Thread sch...@linux-m68k.org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55975



--- Comment #9 from Andreas Schwab sch...@linux-m68k.org 2013-01-16 19:12:01 
UTC ---

-FAIL: c-c++-common/asan/global-overflow-1.c  -O0  output pattern test, is

==9876== AddressSanitizer CHECK failed:

../../../../../gcc/libsanitizer/asan/asan_thread.cc:74

((AddrIsInMem(stack_bottom_))) != (0) (0x0, 0x0)

+FAIL: c-c++-common/asan/global-overflow-1.c  -O0  output pattern test, is

ASAN:SIGSEGV

 =

-==9876== ERROR: AddressSanitizer: unknown-crash on address 0x3fffc266bcc0 at

pc 0x3fff8c011b1c bp 0x3fffc266a7f0 sp 0x3fffc266a860

-WRITE of size 1 at 0x3fffc266bcc0 thread T0

-==9876== AddressSanitizer: while reporting a bug found another one.Ignoring.

+==15643== ERROR: AddressSanitizer: SEGV on unknown address 0x020002002471 (pc

0x1a4c sp 0x3fffe508dba0 bp 0x3fffe508dba0 T0)

+AddressSanitizer can not provide additional info.

+#0 0x1a48 in main

/home/andreas/src/gcc/gcc/gcc/testsuite/c-c++-common/asan/global-overflow-1.c:15

+Stats: 0M malloced (0M for red zones) by 0 calls

+Stats: 0M realloced by 0 calls

+Stats: 0M freed by 0 calls

+Stats: 0M really freed by 0 calls

+Stats: 0M (0M-0M) mmaped; 0 maps, 0 unmaps

+  mmaps   by size class: 

+  mallocs by size class: 

+  frees   by size class: 

+  rfrees  by size class: 

+Stats: malloc large: 0 small slow: 0

+Stats: StackDepot: 0 ids; 0M mapped

+==15643== ABORTING





 === gcc Summary for unix/-m64 ===



-# of expected passes210

-# of unexpected failures91

+# of expected passes214

+# of unexpected failures87

 # of unsupported tests17