Re: Removing "long int"-related limit on hash table sizes

2021-07-27 Thread Andres Freund
Hi, On 2021-07-26 11:38:41 +0900, Michael Paquier wrote: > On Sun, Jul 25, 2021 at 12:28:04PM -0400, Tom Lane wrote: > > Andres Freund writes: > >> We really ought to just remove every single use of long. > > > > I have no objection to that as a long-term goal. But I'm not volunteering > > to

Re: Removing "long int"-related limit on hash table sizes

2021-07-26 Thread Alvaro Herrera
On 2021-Jul-26, Tom Lane wrote: > ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: > > We also have the (U)INT64CONST() macros, which are about about two > > thirds as common as the U?LL? suffixes. > > Yeah. Ideally we'd forbid direct use of the suffixes and insist > you go

Re: Removing "long int"-related limit on hash table sizes

2021-07-26 Thread Tom Lane
ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: > We also have the (U)INT64CONST() macros, which are about about two > thirds as common as the U?LL? suffixes. Yeah. Ideally we'd forbid direct use of the suffixes and insist you go through those macros, but I don't know of

Re: Removing "long int"-related limit on hash table sizes

2021-07-26 Thread Dagfinn Ilmari Mannsåker
Alvaro Herrera writes: > On 2021-Jul-25, Ranier Vilela wrote: > >> > BTW, one aspect of this that I'm unsure how to tackle is the >> > common usage of "L" constants; in particular, "work_mem * 1024L" >> > is a really common idiom that we'll need to get rid of. Not sure >> > that grep will be a

Re: Removing "long int"-related limit on hash table sizes

2021-07-26 Thread Alvaro Herrera
On 2021-Jul-25, Ranier Vilela wrote: > > BTW, one aspect of this that I'm unsure how to tackle is the > > common usage of "L" constants; in particular, "work_mem * 1024L" > > is a really common idiom that we'll need to get rid of. Not sure > > that grep will be a useful aid for finding those. >

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Michael Paquier
On Sun, Jul 25, 2021 at 12:28:04PM -0400, Tom Lane wrote: > Andres Freund writes: >> We really ought to just remove every single use of long. > > I have no objection to that as a long-term goal. But I'm not volunteering > to do all the work, and in any case it wouldn't be a back-patchable fix.

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Ranier Vilela
Em dom., 25 de jul. de 2021 às 15:53, Tom Lane escreveu: > Ranier Vilela writes: > > I think int64 is in most cases the counterpart of *long* on Windows. > > I'm not particularly on board with s/long/int64/g as a universal > solution. Sure, not a universal solution, I mean a start point. When

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Tom Lane
Ranier Vilela writes: > I think int64 is in most cases the counterpart of *long* on Windows. I'm not particularly on board with s/long/int64/g as a universal solution. I think that most of these usages are concerned with memory sizes and would be better off as "size_t". We might need int64 in

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Ranier Vilela
Em dom., 25 de jul. de 2021 às 13:28, Tom Lane escreveu: > Andres Freund writes: > > On 2021-07-23 17:15:24 -0400, Tom Lane wrote: > >> That's because they spill to disk where they did not before. The easy > >> answer of "raise hash_mem_multiplier" doesn't help, because on Windows > >> the

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Tom Lane
Andres Freund writes: > On 2021-07-23 17:15:24 -0400, Tom Lane wrote: >> That's because they spill to disk where they did not before. The easy >> answer of "raise hash_mem_multiplier" doesn't help, because on Windows >> the product of work_mem and hash_mem_multiplier is clamped to 2GB, >> thanks

Re: Removing "long int"-related limit on hash table sizes

2021-07-24 Thread Bruce Momjian
On Sat, Jul 24, 2021 at 06:25:53PM -0700, Andres Freund wrote: > > That's because they spill to disk where they did not before. The easy > > answer of "raise hash_mem_multiplier" doesn't help, because on Windows > > the product of work_mem and hash_mem_multiplier is clamped to 2GB, > > thanks to

Re: Removing "long int"-related limit on hash table sizes

2021-07-24 Thread Andres Freund
Hi, On 2021-07-23 17:15:24 -0400, Tom Lane wrote: > Per the discussion at [1], users on Windows are seeing nasty performance > losses in v13/v14 (compared to prior releases) for hash aggregations that > required somewhat more than 2GB in the prior releases. Ugh :(. > That's because they spill

Removing "long int"-related limit on hash table sizes

2021-07-23 Thread Tom Lane
Per the discussion at [1], users on Windows are seeing nasty performance losses in v13/v14 (compared to prior releases) for hash aggregations that required somewhat more than 2GB in the prior releases. That's because they spill to disk where they did not before. The easy answer of "raise