Re: python and HUGE_STACK_SIZE
On Mar 25, 2010, at 1:07 AM, Adam Vande More wrote: On Wed, Mar 24, 2010 at 5:50 PM, Chuck Swiger cswi...@mac.com wrote: I've run and written quite a bit of Python (including Trac, Mailman, the Python IDE, our own custom stuff [like some log munging and web processing stuff], and even a few graphical Python games) without ever turning HUGE_STACK_SIZE on. I don't have any objection to turning it on, but it's not needed by default for most things. YMMV. Yes, I've had the same experience. When doing socket level python stuff, I've had to increase the buffer size, which seems to be at least indirectly related to stack size but setting it manually has been easy enough. Are there any negative repercussions to turning on huge ie like would scripts start using more memory, or is just giving them the ability to use it without explicitly setting it? So, it seems like most of the time python scripts will work with HUGE_STACK_SIZE turned off, but every once and a while some scripts will fail in non obvious ways that could leave a person scratching their head for weeks trying to get to the bottom of it To me, it seems like the best behavior would be to default to compiling with that set. I'll create a patch over the weekend and open a PR -- Steven Kreuzer http://www.exit2shell.com/~skreuzer ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: python and HUGE_STACK_SIZE
On Fri, Mar 26, 2010 at 3:03 PM, Steven Kreuzer skreu...@exit2shell.comwrote: To me, it seems like the best behavior would be to default to compiling with that set. I'll create a patch over the weekend and open a PR Django and twisted don't need it. If the only app which does is zope, defaulting to on seems to be overkill. No offense to zope users, but I imagine freebsd zope installs aren't on every corner. It would be interesting to hear more feedback from other sources, and your coworker with the original experience. Within python, stack size(in regards to sockets) errors are made quite evident. What zope does to it I have no idea, maybe there is some abstraction going on. My main objection that I use python a lot, especially for quick network apps. One example is that I have written a python based NMS, and if every thread is going to start using more memory, then I will have to do things differently. That's okay too, but if others are doing anything similar there could be more ripples down the road. -- Adam Vande More ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: python and HUGE_STACK_SIZE
On 03/26/10 14:03, Steven Kreuzer wrote: So, it seems like most of the time python scripts will work with HUGE_STACK_SIZE turned off, but every once and a while some scripts will fail in non obvious ways that could leave a person scratching their head for weeks trying to get to the bottom of it To me, it seems like the best behavior would be to default to compiling with that set. I'll create a patch over the weekend and open a PR From the discussion (not speaking from experience or python knowledge) it seems like an OPTION is the way to go, with the open question being defaults to on or defaults to off. What is the impact of HUGE_STACK_SIZE when it's compiled in, and how will it affect those running python stuff who don't actually need it? If it turns out that only a few ports need it and the impact is undesirable those ports that do need it could be adapted to test for it somehow and suggest that the user re-install python with the option. Doug -- ... and that's just a little bit of history repeating. -- Propellerheads Improve the effectiveness of your Internet presence with a domain name makeover!http://SupersetSolutions.com/ ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: python and HUGE_STACK_SIZE
Doug Barton wrote: On 03/26/10 14:03, Steven Kreuzer wrote: So, it seems like most of the time python scripts will work with HUGE_STACK_SIZE turned off, but every once and a while some scripts will fail in non obvious ways that could leave a person scratching their head for weeks trying to get to the bottom of it To me, it seems like the best behavior would be to default to compiling with that set. I'll create a patch over the weekend and open a PR From the discussion (not speaking from experience or python knowledge) it seems like an OPTION is the way to go, with the open question being defaults to on or defaults to off. What is the impact of HUGE_STACK_SIZE when it's compiled in, and how will it affect those running python stuff who don't actually need it? If it turns out that only a few ports need it and the impact is undesirable those ports that do need it could be adapted to test for it somehow and suggest that the user re-install python with the option. HUGE_STACK_SIZE actually refers to the stack size used for each thread other than the primary thread. As I understand it, these stacks are fully committed while the thread exists, which is different to the handling of the primary thread's stack (where only the pages used are committed). On 32 bit systems, careless use of large thread stacks can cause non-trivial address space wastage. People don't see the same issues on Linux as it has a much larger default thread stack size and I'm lead to believe that only the pages in use in the thread stacks are committed. Since Python 2.5, there has been a way to change the thread stack size at runtime: threading.stack_size(). Ports patches using these calls for packages likely to be affected (such as Zope) could make this easier for a lot of people, if support for them can't be encouraged upstream. -- - Andrew I MacIntyre These thoughts are mine alone... E-mail: andy...@bullseye.apana.org.au (pref) | Snail: PO Box 370 andy...@pcug.org.au (alt) |Belconnen ACT 2616 Web:http://www.andymac.org/ |Australia ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
python and HUGE_STACK_SIZE
A coworker of mine got bit by this bug on and off for the last few weeks and it appears he might not be the only one as documented in this blog: http://tomster.org/blog/archive/2006/09/27/size-does-matter If python needs to be compiled with HUGE_STACK_SIZE on FreeBSD, is there a reason to provide the option to not compile python with it, or at the very least, should it default to being on? I'm not a python guy, so I am just asking in case what is in the blog post is true. -- Steven Kreuzer http://www.exit2shell.com/~skreuzer -- Steven Kreuzer http://www.exit2shell.com/~skreuzer ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: python and HUGE_STACK_SIZE
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 [freebsd-python@ cc'ed] On 2010/03/24 15:20, Steven Kreuzer wrote: A coworker of mine got bit by this bug on and off for the last few weeks and it appears he might not be the only one as documented in this blog: http://tomster.org/blog/archive/2006/09/27/size-does-matter If python needs to be compiled with HUGE_STACK_SIZE on FreeBSD, is there a reason to provide the option to not compile python with it, or at the very least, should it default to being on? Ah, yes this is the thing I turn on on all systems I have myself... I'd vote for enabling it by default. Cheers, - -- Xin LI delp...@delphij.nethttp://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLqpwyAAoJEATO+BI/yjfBrwwIAJfEAeJ9QXTvgOeMRyAdekcU teYGqTVP1W3qE7adxvM/2MMQm4ogyJtAJBu9UQVAwiYtckNd1mpaIuQdfDvuc1fD vD5OT+8Sy+TXqxLi2A29HZkOCvOoYuZ03WwyF5YMpMU6uPHGa0fnx+gNtXGuOxSN V45ZWxlEwNNKv+WYZzehF5jJvIPLXadrPYP/y53loIXqxA5htV+ZPpiGeidwTDun u+o5AjLrcRKRM+hrf10X6gfCgJZpQyqeL88fDqhUX6uL1QIQJHg1nBHQ+6DbZrR9 R2cCadkhCF2X3nC3stTDi/KvuAZ4rZZo02lUf8MdxK7PX8T9cd4A7brHCltxoOU= =UaWe -END PGP SIGNATURE- ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: python and HUGE_STACK_SIZE
Hi-- On Mar 24, 2010, at 4:11 PM, Xin LI wrote: If python needs to be compiled with HUGE_STACK_SIZE on FreeBSD, is there a reason to provide the option to not compile python with it, or at the very least, should it default to being on? Ah, yes this is the thing I turn on on all systems I have myself... I'd vote for enabling it by default. I've run and written quite a bit of Python (including Trac, Mailman, the Python IDE, our own custom stuff [like some log munging and web processing stuff], and even a few graphical Python games) without ever turning HUGE_STACK_SIZE on. I don't have any objection to turning it on, but it's not needed by default for most things. YMMV. Regards, -- -Chuck ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: python and HUGE_STACK_SIZE
On Wed, Mar 24, 2010 at 5:50 PM, Chuck Swiger cswi...@mac.com wrote: I've run and written quite a bit of Python (including Trac, Mailman, the Python IDE, our own custom stuff [like some log munging and web processing stuff], and even a few graphical Python games) without ever turning HUGE_STACK_SIZE on. I don't have any objection to turning it on, but it's not needed by default for most things. YMMV. Yes, I've had the same experience. When doing socket level python stuff, I've had to increase the buffer size, which seems to be at least indirectly related to stack size but setting it manually has been easy enough. Are there any negative repercussions to turning on huge ie like would scripts start using more memory, or is just giving them the ability to use it without explicitly setting it? -- Adam Vande More ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
