Re: [HACKERS] [sqlsmith] Infinite recursion in bitshift

2016-10-14 Thread Tom Lane
Merlin Moncure writes: > On Fri, Oct 14, 2016 at 3:31 PM, Tom Lane wrote: >> Poking around in varbit.c, I noticed some other places that were assuming >> that a typmod couldn't exceed VARBITMAXLEN. > Curious -- are there real world scenarios where this

Re: [HACKERS] [sqlsmith] Infinite recursion in bitshift

2016-10-14 Thread Merlin Moncure
On Fri, Oct 14, 2016 at 3:31 PM, Tom Lane wrote: > Andreas Seltenreich writes: >> Tom Lane writes: >>> Seems sane, though I wonder if it'd be better to use -INT_MAX rather >>> than -VARBITMAXLEN. > >> I am undecided between those two. -INT_MAX might be a

Re: [HACKERS] [sqlsmith] Infinite recursion in bitshift

2016-10-14 Thread Tom Lane
Andreas Seltenreich writes: > Tom Lane writes: >> Seems sane, though I wonder if it'd be better to use -INT_MAX rather >> than -VARBITMAXLEN. > I am undecided between those two. -INT_MAX might be a more precise fix > for the problem, but the extra distance to the danger zone

Re: [HACKERS] [sqlsmith] Infinite recursion in bitshift

2016-10-14 Thread Andreas Seltenreich
Tom Lane writes: >> This is due to an integer overflow in bitshiftright()/bitshiftleft() >> leading to them recursively calling each other. Patch attached. > > Seems sane, though I wonder if it'd be better to use -INT_MAX rather > than -VARBITMAXLEN. I am undecided between those two. -INT_MAX

Re: [HACKERS] [sqlsmith] Infinite recursion in bitshift

2016-10-14 Thread Tom Lane
Andreas Seltenreich writes: > sqlsmith just found another crasher: > select bit '1' >> (-2^31)::int; Nice catch :-) > This is due to an integer overflow in bitshiftright()/bitshiftleft() > leading to them recursively calling each other. Patch attached. Seems sane,