Re: Stack overflow issue

2024-04-17 Thread Andres Freund
Hi, On 2024-04-17 14:39:14 +0300, Alexander Korotkov wrote: > On Wed, Apr 17, 2024 at 2:37 PM Alexander Korotkov > wrote: > > I've invested more time into polishing this. I'm intended to push > > this. Could you, please, take a look before? > > Just after sending this I spotted a typo

Re: Stack overflow issue

2024-04-17 Thread Alexander Korotkov
On Wed, Apr 17, 2024 at 2:37 PM Alexander Korotkov wrote: > On Tue, Apr 16, 2024 at 7:42 PM Alexander Korotkov > wrote: > > On Tue, Apr 16, 2024 at 6:35 PM Andres Freund wrote: > > > On 2024-04-16 15:45:42 +0300, Alexander Korotkov wrote: > > > > On Tue, Apr 16, 2024 at 1:48 AM Andres Freund

Re: Stack overflow issue

2024-04-17 Thread Alexander Korotkov
On Tue, Apr 16, 2024 at 7:42 PM Alexander Korotkov wrote: > > On Tue, Apr 16, 2024 at 6:35 PM Andres Freund wrote: > > On 2024-04-16 15:45:42 +0300, Alexander Korotkov wrote: > > > On Tue, Apr 16, 2024 at 1:48 AM Andres Freund wrote: > > > > On 2024-03-06 14:17:23 +0200, Alexander Korotkov

Re: Stack overflow issue

2024-04-16 Thread Alexander Korotkov
On Tue, Apr 16, 2024 at 6:35 PM Andres Freund wrote: > On 2024-04-16 15:45:42 +0300, Alexander Korotkov wrote: > > On Tue, Apr 16, 2024 at 1:48 AM Andres Freund wrote: > > > On 2024-03-06 14:17:23 +0200, Alexander Korotkov wrote: > > > > 0001 Turn tail recursion into iteration in

Re: Stack overflow issue

2024-04-16 Thread Andres Freund
Hi, On 2024-04-16 15:45:42 +0300, Alexander Korotkov wrote: > On Tue, Apr 16, 2024 at 1:48 AM Andres Freund wrote: > > On 2024-03-06 14:17:23 +0200, Alexander Korotkov wrote: > > > 0001 Turn tail recursion into iteration in CommitTransactionCommand() > > > I did minor revision of comments and

Re: Stack overflow issue

2024-04-16 Thread Alexander Korotkov
On Tue, Apr 16, 2024 at 1:48 AM Andres Freund wrote: > On 2024-03-06 14:17:23 +0200, Alexander Korotkov wrote: > > 0001 Turn tail recursion into iteration in CommitTransactionCommand() > > I did minor revision of comments and code blocks order to improve the > > readability. > > After sending >

Re: Stack overflow issue

2024-04-15 Thread Andres Freund
Hi, On 2024-03-06 14:17:23 +0200, Alexander Korotkov wrote: > 0001 Turn tail recursion into iteration in CommitTransactionCommand() > I did minor revision of comments and code blocks order to improve the > readability. After sending

Re: Stack overflow issue

2024-03-10 Thread Alexander Korotkov
On Thu, Mar 7, 2024 at 11:07 AM Alexander Korotkov wrote: > On Thu, Mar 7, 2024 at 9:53 AM Egor Chindyaskin wrote: > > > > > 6 march 2024 г., at 19:17, Alexander Korotkov > > > wrote: > > > > > > The revised set of remaining patches is attached. > > > > > > 0001 Turn tail recursion into

Re: Stack overflow issue

2024-03-08 Thread Alexander Korotkov
On Thu, Mar 7, 2024 at 1:49 AM Tom Lane wrote: > > Alexander Korotkov writes: > > Sorry for tediousness, but isn't pre-order a variation of depth-first order > > [1]? > > To me, depth-first implies visiting children before parents. > Do I have the terminology wrong? According to Wikipedia,

Re: Stack overflow issue

2024-03-07 Thread Alexander Korotkov
Hi, Egor! On Thu, Mar 7, 2024 at 9:53 AM Egor Chindyaskin wrote: > > > 6 march 2024 г., at 19:17, Alexander Korotkov wrote: > > > > The revised set of remaining patches is attached. > > > > 0001 Turn tail recursion into iteration in CommitTransactionCommand() > > I did minor revision of

Re: Stack overflow issue

2024-03-06 Thread Tom Lane
Alexander Korotkov writes: > Sorry for tediousness, but isn't pre-order a variation of depth-first order > [1]? To me, depth-first implies visiting children before parents. Do I have the terminology wrong? regards, tom lane

Re: Stack overflow issue

2024-03-06 Thread Alexander Korotkov
On Thu, Mar 7, 2024 at 12:52 AM Tom Lane wrote: > Alexander Korotkov writes: > > The revised set of remaining patches is attached. > > ... > > 0003 Avoid recursion in MemoryContext functions > > I've renamed MemoryContextTraverse() => MemoryContextTraverseNext(), > > which I think is a bit more

Re: Stack overflow issue

2024-03-06 Thread Tom Lane
Alexander Korotkov writes: > The revised set of remaining patches is attached. > ... > 0003 Avoid recursion in MemoryContext functions > I've renamed MemoryContextTraverse() => MemoryContextTraverseNext(), > which I think is a bit more intuitive. Also I fixed > MemoryContextMemConsumed(), which

Re: Stack overflow issue

2024-03-06 Thread Alexander Korotkov
On Wed, Feb 14, 2024 at 2:00 PM Alexander Korotkov wrote: > On Fri, Jan 12, 2024 at 11:00 PM Robert Haas wrote: > > On Fri, Jan 12, 2024 at 10:12 AM Heikki Linnakangas wrote: > > > Here's one goto-free attempt. It adds a local loop to where the > > > recursion was, so that if you have a chain

Re: Stack overflow issue

2024-02-14 Thread Alexander Korotkov
Hi! On Fri, Jan 12, 2024 at 11:00 PM Robert Haas wrote: > On Fri, Jan 12, 2024 at 10:12 AM Heikki Linnakangas wrote: > > Here's one goto-free attempt. It adds a local loop to where the > > recursion was, so that if you have a chain of subtransactions that need > > to be aborted in

Re: Stack overflow issue

2024-01-12 Thread Robert Haas
On Fri, Jan 12, 2024 at 10:12 AM Heikki Linnakangas wrote: > Here's one goto-free attempt. It adds a local loop to where the > recursion was, so that if you have a chain of subtransactions that need > to be aborted in CommitTransactionCommand, they are aborted iteratively. > The TBLOCK_SUBCOMMIT

Re: Stack overflow issue

2024-01-12 Thread Heikki Linnakangas
On 11/01/2024 19:37, Robert Haas wrote: On Wed, Jan 10, 2024 at 4:25 PM Heikki Linnakangas wrote: The problem with CommitTransactionCommand (or rather AbortCurrentTransaction() which has the same problem) and ShowTransactionStateRec is that they get called in a state where aborting can lead to

Re: Stack overflow issue

2024-01-11 Thread Robert Haas
On Wed, Jan 10, 2024 at 4:25 PM Heikki Linnakangas wrote: > The problem with CommitTransactionCommand (or rather > AbortCurrentTransaction() which has the same problem) > and ShowTransactionStateRec is that they get called in a state where > aborting can lead to a panic. If you add a

Re: Stack overflow issue

2024-01-10 Thread Heikki Linnakangas
On 05/01/2024 19:23, Robert Haas wrote: On Fri, Nov 24, 2023 at 10:47 AM Heikki Linnakangas wrote: What do you think? At least for 0001 and 0002, I think we should just add the stack depth checks. With regard to 0001, CommitTransactionCommand() and friends are hard enough to understand as

Re: Stack overflow issue

2024-01-05 Thread Robert Haas
On Fri, Jan 5, 2024 at 3:16 PM Andres Freund wrote: > On 2024-01-05 12:23:25 -0500, Robert Haas wrote: > > I agree that in the memory-context case it might be worth expending > > some more code to be more clever. But I probably wouldn't do that for > > MemoryContextStats(); check_stack_depth()

Re: Stack overflow issue

2024-01-05 Thread Andres Freund
Hi, On 2024-01-05 12:23:25 -0500, Robert Haas wrote: > I agree that in the memory-context case it might be worth expending > some more code to be more clever. But I probably wouldn't do that for > MemoryContextStats(); check_stack_depth() seems fine for that one. We run MemoryContextStats() when

Re: Stack overflow issue

2024-01-05 Thread Robert Haas
On Fri, Nov 24, 2023 at 10:47 AM Heikki Linnakangas wrote: > What do you think? At least for 0001 and 0002, I think we should just add the stack depth checks. With regard to 0001, CommitTransactionCommand() and friends are hard enough to understand as it is; they need "goto" like I need an

Re: Stack overflow issue

2023-12-22 Thread Egor Chindyaskin
On 24/11/2023 21:14, Heikki Linnakangas wrote: What do you think? Hello! Thank you for researching the problem! I'm more of a tester than a developer, so I was able to check the patches from that side. I've configured the server with CFLAGS=" -O0" and cassert enabled and checked the following

Re: Stack overflow issue

2023-11-24 Thread Heikki Linnakangas
On 21/06/2023 16:45, Egor Chindyaskin wrote: Hello! In continuation of the topic I would like to suggest solution. This patch adds several checks to the vulnerable functions above. I looked at this last patch. The depth checks are clearly better than segfaulting, but I think we can also avoid

Re: Stack overflow issue

2023-06-21 Thread Egor Chindyaskin
Hello! In continuation of the topic I would like to suggest solution. This patch adds several checks to the vulnerable functions above.diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index d85e313908..102d0e1574 100644 --- a/src/backend/access/transam/xact.c +++

Re: Stack overflow issue

2023-01-19 Thread Egor Chindyaskin
03.01.2023 22:45, Sascha Kuhl writes: Great work. Max Stack depth is memory dependent? Processor dependent? Hello! These situations are not specific to the x86_64 architecture, but also manifest themselves, for example, on aarch64 architecture. For example this query, ran on aarch64,

Re: Stack overflow issue

2023-01-19 Thread Egor Chindyaskin
Hello! In continuation of the topic, I, under the leadership of Alexander Lakhin, prepared patches that fix these problems. We decided that these checks would be enough and put them in the places we saw fit.diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index

Re: Stack overflow issue

2023-01-03 Thread Sascha Kuhl
Great work. Max Stack depth is memory dependent? Processor dependent? Егор Чиндяскин schrieb am Mi., 24. Aug. 2022, 11:51: > Hello, I recently got a server crash (bug #17583 [1]) caused by a stack > overflow. > > Tom Lane and Richard Guo, in a discussion of this bug, suggested that > there

Re: Stack overflow issue

2023-01-03 Thread Егор Чиндяскин
>Среда, 26 октября 2022, 21:47 +07:00 от Egor Chindyaskin : >  >24.08.2022 20:58, Tom Lane writes: >> Nice work! I wonder if you can make the regex crashes reachable by >> reducing the value of max_stack_depth enough that it's hit before >> reaching the "regular expression is too complex" limit.

Re: Stack overflow issue

2022-10-26 Thread Egor Chindyaskin
24.08.2022 20:58, Tom Lane writes: Nice work! I wonder if you can make the regex crashes reachable by reducing the value of max_stack_depth enough that it's hit before reaching the "regular expression is too complex" limit. regards, tom lane Hello everyone! It's been a

Re: Stack overflow issue

2022-08-30 Thread Richard Guo
them except > case-fold. This is closer to the pre-patch behavior, but gives up > the opportunity to avoid useless downstream processing of long words. This patch looks good to me. It avoids overly-long words (> 1000 bytes) going through the stemmer so the stack overflow issue in Turkish stemmer should not exist any more. Thanks Richard

Re: Stack overflow issue

2022-08-30 Thread Tom Lane
I wrote: > The upstream recommendation, which seems pretty sane to me, is to > simply reject any string exceeding some threshold length as not > possibly being a word. Apparently it's common to use thresholds > as small as 64 bytes, but in the attached I used 1000 bytes. On further thought: that

Re: Stack overflow issue

2022-08-30 Thread Tom Lane
I wrote: >> I think most likely we should report this to Snowball upstream >> and see what they think is an appropriate fix. > Done at [1], and I pushed the other fixes. Thanks again for the report! The upstream recommendation, which seems pretty sane to me, is to simply reject any string

Re: Stack overflow issue

2022-08-24 Thread Tom Lane
I wrote: > I think most likely we should report this to Snowball upstream > and see what they think is an appropriate fix. Done at [1], and I pushed the other fixes. Thanks again for the report! regards, tom lane [1]

Re: Stack overflow issue

2022-08-24 Thread Tom Lane
=?UTF-8?B?0JXQs9C+0YAg0KfQuNC90LTRj9GB0LrQuNC9?= writes: > Firstly, we decided to test the regex catalog functions and found 6 of them  > that lack the check_stach_depth() call. > zaptreesubs > markst > next > nfatree > numst > repeat I took a closer look at these. I think the markst, numst,

Re: Stack overflow issue

2022-08-24 Thread Tom Lane
Richard Guo writes: > Attached adds the checks in these places. But I'm not sure about the > snowball case. Can we edit src/backend/snowball/libstemmer/*.c directly? No, that file is generated code, as it says right at the top. I think most likely we should report this to Snowball upstream and

Re: Stack overflow issue

2022-08-24 Thread Tom Lane
=?UTF-8?B?0JXQs9C+0YAg0KfQuNC90LTRj9GB0LrQuNC9?= writes: > Therefore, Alexander Lakhin and I decided to deal with this issue and > Alexander developed a methodology. We processed src/backend/*/*.c with "clang > -emit-llvm  ... | opt -analyze -print-calgraph" to find all the functions > that

Re: Stack overflow issue

2022-08-24 Thread mahendrakar s
Hi Richard, Patch is looking good to me. Would request others to take a look at it as well. Thanks, Mahendrakar. On Wed, 24 Aug 2022 at 17:24, Richard Guo wrote: > > On Wed, Aug 24, 2022 at 7:12 PM Richard Guo > wrote: > >> >> On Wed, Aug 24, 2022 at 6:49 PM Alvaro Herrera >> wrote: >> >>>

Re: Stack overflow issue

2022-08-24 Thread Richard Guo
On Wed, Aug 24, 2022 at 7:12 PM Richard Guo wrote: > > On Wed, Aug 24, 2022 at 6:49 PM Alvaro Herrera > wrote: > >> On 2022-Aug-24, mahendrakar s wrote: >> >> > Hi, >> > Can we have a parameter to control the recursion depth in these cases to >> > avoid crashes? >> >> We already have one

Re: Stack overflow issue

2022-08-24 Thread Richard Guo
On Wed, Aug 24, 2022 at 6:49 PM Alvaro Herrera wrote: > On 2022-Aug-24, mahendrakar s wrote: > > > Hi, > > Can we have a parameter to control the recursion depth in these cases to > > avoid crashes? > > We already have one (max_stack_depth). The problem is lack of calling > the control function

Re: Stack overflow issue

2022-08-24 Thread mahendrakar s
Thanks. On Wed, 24 Aug, 2022, 4:19 pm Alvaro Herrera, wrote: > On 2022-Aug-24, mahendrakar s wrote: > > > Hi, > > Can we have a parameter to control the recursion depth in these cases to > > avoid crashes? > > We already have one (max_stack_depth). The problem is lack of calling > the control

Re: Stack overflow issue

2022-08-24 Thread Alvaro Herrera
On 2022-Aug-24, mahendrakar s wrote: > Hi, > Can we have a parameter to control the recursion depth in these cases to > avoid crashes? We already have one (max_stack_depth). The problem is lack of calling the control function in a few places. -- Álvaro Herrera 48°01'N 7°57'E —

Re: Stack overflow issue

2022-08-24 Thread mahendrakar s
Hi, Can we have a parameter to control the recursion depth in these cases to avoid crashes? Just a thought. Thanks, Mahendrakar. On Wed, 24 Aug, 2022, 3:21 pm Егор Чиндяскин, wrote: > Hello, I recently got a server crash (bug #17583 [1]) caused by a stack > overflow. > > Tom Lane and Richard

Stack overflow issue

2022-08-24 Thread Егор Чиндяскин
Hello, I recently got a server crash (bug #17583 [1]) caused by a stack overflow.    Tom Lane and Richard Guo, in a discussion of this bug, suggested that there could be more such places.  Therefore, Alexander Lakhin and I decided to deal with this issue and Alexander developed a methodology.