[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread Ned Deily


Ned Deily  added the comment:


New changeset dcc0eb379613f279864af61023ea44c94aa0535c by Ned Deily (Miss 
Islington (bot)) in branch '3.7':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
https://github.com/python/cpython/commit/dcc0eb379613f279864af61023ea44c94aa0535c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread Ned Deily


Change by Ned Deily :


--
priority: release blocker -> 
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread Ned Deily


Ned Deily  added the comment:


New changeset 782854f90ad5f73f787f68693d535f2b05514e13 by Ned Deily (Miss 
Islington (bot)) in branch '3.6':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546) 
(GH-14549)
https://github.com/python/cpython/commit/782854f90ad5f73f787f68693d535f2b05514e13


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread miss-islington


miss-islington  added the comment:


New changeset bf82cd3124df94935c6e3190c7c40b76918d2174 by Miss Islington (bot) 
in branch '3.7':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
https://github.com/python/cpython/commit/bf82cd3124df94935c6e3190c7c40b76918d2174


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread miss-islington


miss-islington  added the comment:


New changeset bd92b94da93198c8385c06ca908407f172c7e8b2 by Miss Islington (bot) 
in branch '3.8':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
https://github.com/python/cpython/commit/bd92b94da93198c8385c06ca908407f172c7e8b2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread Ned Deily


Ned Deily  added the comment:


New changeset 5bbbc733e6cc0804f19b071944af8d4719e26ae6 by Ned Deily in branch 
'master':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
https://github.com/python/cpython/commit/5bbbc733e6cc0804f19b071944af8d4719e26ae6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread miss-islington


Change by miss-islington :


--
pull_requests: +14365
pull_request: https://github.com/python/cpython/pull/14549

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread miss-islington


Change by miss-islington :


--
pull_requests: +14363
pull_request: https://github.com/python/cpython/pull/14548

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread miss-islington


Change by miss-islington :


--
pull_requests: +14361
pull_request: https://github.com/python/cpython/pull/14547

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread Ned Deily


Change by Ned Deily :


--
pull_requests: +14359
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/14546

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-07-02 Thread Ned Deily


Ned Deily  added the comment:

> So we should keep an eye open for reports of segfaults running tests as 
> originally reported in Issue18075.

Now there are such reports for 3.7.4rc1, at least. Release builds seem to be OK 
but at least four tests now fail on macOS 10.14.5 when built --with-pydebug: 
test_exceptions test_json test_logging test_sys test_traceback.  Plus, changing 
the interpreters stack size can inhibit use of dtrace.

There are really two attempts at dealing with macOS's small default stack size. 
 The original workaround, dating back to 2002-12-02 
(bb48465273d2aa98fc7669e99b0d5fb1c57962de !!) added the runtime calls in 
regrtest to change RLIMIT_STACK. Years later, a different approach was taken in 
the original change for Issue18075 (335ab5b66f432ae3713840ed2403a11c368f5406) 
by increasing the default stack size when building the interpreter rather than 
at runtime.  So, I *think* that means that the original regrtest runtime 
workaround is really no longer needed.  So I think we should take the opposite 
approach to what I originally merged back in April, that is, we should go back 
to building the interpreter with the increased stack size and remove the 
ancient regrtest workaround attempt: that's what was failing here anyway.  The 
only place in the source base that resource.RLIMIT_STACK is used, outside of 
its functional test, is the old regrtest workaround.  It's still a bit of a 
mystery as what has 
 changed in 3.8 that seems to not hit the stack size limit but these kind of 
test failures on macOS have always been dependent on other factors, including 
compiler version and options.  I'd like to get this into 3.8.0b2 and 3.7.4rc2 
ao I'm temporarily making it a "release blocker"; the PR will follow 
momentarily.

--
nosy: +lukasz.langa
priority: normal -> release blocker
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
versions: +Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-04-29 Thread Ned Deily


Ned Deily  added the comment:

Thanks for the analyses everyone!  Also see the discussion in duplicate 
Issue36432.  I was never able to reproduce the failure on earlier versions of 
macOS but then it seemed to become a hard failure with the release of 10.14.4.  
I haven't gone back and tried running the tests on all supported older versions 
with this reversion in place but those I have did not exhibit any new failures. 
 So we should keep an eye open for reports of segfaults running tests as 
originally reported in Issue18075.  But better that then not being able to run 
any tests.  "Fixed" in 3.8.0a4, 3.7.4, and 3.6.9 (to allow tests to be run on 
macOS) by reverting the change for Issue18075.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-04-29 Thread Ned Deily


Ned Deily  added the comment:


New changeset fbe2a1394bf52f5a4455681e1b1f705a31559585 by Ned Deily (Miss 
Islington (bot)) in branch '3.6':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011) 
(GH-13014)
https://github.com/python/cpython/commit/fbe2a1394bf52f5a4455681e1b1f705a31559585


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-04-29 Thread miss-islington


Change by miss-islington :


--
pull_requests: +12936

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-04-29 Thread miss-islington


miss-islington  added the comment:


New changeset 52a5b71063af68c42b048095c4e555e93257f151 by Miss Islington (bot) 
in branch '3.7':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011)
https://github.com/python/cpython/commit/52a5b71063af68c42b048095c4e555e93257f151


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-04-29 Thread miss-islington


Change by miss-islington :


--
pull_requests: +12934

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-04-29 Thread Ned Deily


Ned Deily  added the comment:


New changeset 883dfc668f9730b00928730035b5dbd24b9da2a0 by Ned Deily in branch 
'master':
bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011)
https://github.com/python/cpython/commit/883dfc668f9730b00928730035b5dbd24b9da2a0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-04-29 Thread Ned Deily


Change by Ned Deily :


--
keywords: +patch
pull_requests: +12932
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-03-30 Thread Chih-Hsuan Yen


Chih-Hsuan Yen  added the comment:

I guess Inada Naoki was to say https://bugs.python.org/issue36432 in the last 
comment.

--
nosy: +yan12125

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2019-03-26 Thread Inada Naoki


Inada Naoki  added the comment:

https://bugs.python.org/issue34602 may be relating to this.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2018-09-08 Thread marche147


marche147  added the comment:

Thanks for the repro! It did help for pinpointing the issue.

So I took a little spare time and dived into xnu kernel code, here is my 
assumption based on what I found (N.B. : My assumption comes from a simple 
experiment and a brief skim of the source code within 15 minutes or less, it 
could be seriously wrong since I'm not an expert of XNU kernel, and I currently 
don't have the time to build and debug it.) :

In bsd/kern/kern_resource.c, there's a function `dosetrlimit` which handles the 
`setrlimit` request, and here's part of it:

```
  case RLIMIT_STACK:
// ...
if (limp->rlim_cur > alimp->rlim_cur) {
  user_addr_t addr;
  user_size_t size;

/* grow stack */
size = round_page_64(limp->rlim_cur);
size -= round_page_64(alimp->rlim_cur);

  addr = p->user_stack - round_page_64(limp->rlim_cur);
  kr = mach_vm_protect(current_map(),
   addr, size,
   FALSE, VM_PROT_DEFAULT);
  if (kr != KERN_SUCCESS) {
error =  EINVAL;
goto out;
  }
} // ...

```

As we can see, the kernel will try to `mprotect` the memory preceding the stack 
to `VM_PROT_DEFAULT` (presumably read & write). I then used `vmmap` to see the 
difference between two binaries compiled with different commands. And the 
results are : 

1. Binary compiled without default stack size:

```
- Before calling setrlimit

...
STACK GUARD7ffee76d9000-7ffeeaed9000 [ 56.0M 0K 0K  
   0K] ---/rwx SM=NUL  stack guard for thread 0
...
Stack  7ffeeaed9000-7ffeeb6d9000 [ 8192K20K20K  
   0K] rw-/rwx SM=PRV  thread 0
...
VIRTUAL RESIDENTDIRTY  SWAPPED VOLATILE   
NONVOLEMPTY   REGION
REGION TYPESIZE SIZE SIZE SIZE SIZE 
SIZE SIZECOUNT (non-coalesced)
=== === =  ===    
===  ===
Kernel Alloc Once8K   4K   4K   0K   0K 
  0K   0K2
MALLOC guard page   16K   0K   0K   0K   0K 
  0K   0K5
MALLOC metadata 60K  60K  60K   0K   0K 
  0K   0K6
MALLOC_SMALL  16.0M  16K  16K   0K   0K 
  0K   0K3 see MALLOC ZONE table below
MALLOC_TINY   2048K  32K  32K   0K   0K 
  0K   0K3 see MALLOC ZONE table below
STACK GUARD   56.0M   0K   0K   0K   0K 
  0K   0K2
Stack 8192K  20K  20K   0K   0K 
  0K   0K2
__DATA2324K1192K 208K   0K   0K 
  0K   0K   43
__LINKEDIT   192.7M21.7M   0K   0K   0K 
  0K   0K4
__TEXT9448K8224K   0K   0K   0K 
  0K   0K   48
shared memory8K   8K   8K   0K   0K 
  0K   0K3
=== === =  ===    
===  ===
TOTAL286.3M31.0M 348K   0K   0K 
  0K   0K  110
...

- After calling setrlimit

...
STACK GUARD7ffee76d9000-7ffee76da000 [4K 0K 0K  
   0K] ---/rwx SM=NUL  stack guard for thread 0
...
Stack  7ffee76da000-7ffeeaed9000 [ 56.0M 0K 0K  
   0K] rw-/rwx SM=NUL  thread 0
Stack  7ffeeaed9000-7ffeeb6d9000 [ 8192K20K20K  
   0K] rw-/rwx SM=PRV  thread 0
...
VIRTUAL RESIDENTDIRTY  SWAPPED VOLATILE   
NONVOLEMPTY   REGION
REGION TYPESIZE SIZE SIZE SIZE SIZE 
SIZE SIZECOUNT (non-coalesced)
=== === =  ===    
===  ===
Kernel Alloc Once8K   4K   4K   0K   0K 
  0K   0K2
MALLOC guard page   16K   0K   0K   0K   0K 
  0K   0K5
MALLOC metadata 60K  60K  60K   0K   0K 
  0K   0K6
MALLOC_SMALL  16.0M  20K  20K   0K   0K 
  0K   0K3 see MALLOC ZONE table below
MALLOC_TINY   2048K  32K  32K   0K   0K 
  0K   0K3 see MALLOC ZONE table below
STACK GUARD  4K   0K   0K   0K   0K 
  0K   0K2
Stack 64.0M  20K  20K   0K   0K 
  0K   0K   

[issue34602] python3 resource.setrlimit strange behaviour under macOS

2018-09-07 Thread Vladimir Matveev


Vladimir Matveev  added the comment:

I can repro it with a given sample file 
```
vladima-mbp $ cat test.c
#include 
#include 
#include 
#include 
#include 

int main() {
  struct rlimit rl;
  if(getrlimit(RLIMIT_STACK, ) < 0) {
perror("getrlimit");
exit(1);
  }

  rl.rlim_cur = rl.rlim_max;
  if(setrlimit(RLIMIT_STACK, ) < 0) {
perror("setrlimit");
exit(1);
  }
  return 0;
}vladima-mbp $ gcc   -Wl,-stack_size,100   -o test test.c
vladima-mbp $ ./test
setrlimit: Invalid argument
```
Similar settings were added to Python in 
https://github.com/python/cpython/commit/335ab5b66f4

--
nosy: +v2m

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2018-09-07 Thread Ronald Oussoren


Ronald Oussoren  added the comment:

I get the same error, also with python3.7. Both for homebrew and a python.org 
installer.

--
components: +macOS
nosy: +ned.deily, ronaldoussoren
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34602] python3 resource.setrlimit strange behaviour under macOS

2018-09-07 Thread marche147


New submission from marche147 :

Consider the following code:

```
import resource
s, h = resource.getrlimit(resource.RLIMIT_STACK)
resource.setrlimit(resource.RLIMIT_STACK, (h, h))
```

Running this under macOS with python 3.6.5 gives the following exception:

```
bash-3.2$ uname -a
Darwin arch-osx 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 
2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
bash-3.2$ cat test.py
import resource
s, h = resource.getrlimit(resource.RLIMIT_STACK)
resource.setrlimit(resource.RLIMIT_STACK, (h, h))
bash-3.2$ python3 test.py
Traceback (most recent call last):
  File "test.py", line 3, in 
resource.setrlimit(resource.RLIMIT_STACK, (h, h))
ValueError: current limit exceeds maximum limit
```

Nevertheless, when using python 2.7.10 under the same environment, this code 
works perfectly without exceptions being thrown. Additionally, neither of these 
operations fail under the same circumstances :

```
bash-3.2$ cat test.c
#include 
#include 
#include 
#include 
#include 

int main() {
  struct rlimit rl;
  if(getrlimit(RLIMIT_STACK, ) < 0) {
perror("getrlimit");
exit(1);
  }

  rl.rlim_cur = rl.rlim_max;
  if(setrlimit(RLIMIT_STACK, ) < 0) {
perror("setrlimit");
exit(1);
  }
  return 0;
}
bash-3.2$ gcc -o test test.c
bash-3.2$ ./test
```

```
bash-3.2$ ulimit -s -H
65532
bash-3.2$ ulimit -s
8192
bash-3.2$ ulimit -s 65532
bash-3.2$ ulimit -s
65532
```

I have also tried to run the above-mentioned python script on linux, also it 
does not generate exceptions both on python2 (2.7.10) & python3 (3.6.5).

--
components: Library (Lib)
messages: 324731
nosy: marche147
priority: normal
severity: normal
status: open
title: python3 resource.setrlimit strange behaviour under macOS
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com