Re: Requiring LLVM 14+ in PostgreSQL 18

2024-05-16 Thread Nazir Bilal Yavuz
Y5EFHXLKCO_=p4pwfmhrovom_qse_7b48gpchfa...@mail.gmail.com [2] https://cirrus-ci.com/task/4969910856581120 -- Regards, Nazir Bilal Yavuz Microsoft From 976d2c7ad0e470b24875ee27171359f54078a761 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Mon, 13 May 2024 10:56:28 +0300 Subject: [PATCH v5 1/3] Upgrade D

Re: CREATE DATABASE with filesystem cloning

2024-05-16 Thread Nazir Bilal Yavuz
Hi, On Thu, 16 May 2024 at 15:40, Robert Haas wrote: > > On Thu, May 16, 2024 at 8:35 AM Nazir Bilal Yavuz wrote: > > I updated the documentation and put a comment on top of the copydir() > > function to inform that further changes and uses of this function may > > requi

Re: CREATE DATABASE with filesystem cloning

2024-05-16 Thread Nazir Bilal Yavuz
Hi, On Thu, 9 May 2024 at 19:29, Robert Haas wrote: > > On Wed, May 8, 2024 at 10:34 AM Nazir Bilal Yavuz wrote: > > > I'm not so sure about the GUC name. On the one hand, it feels like > > > createdb should be spelled out as create_database, but on the other > >

Re: Use streaming read API in ANALYZE

2024-05-15 Thread Nazir Bilal Yavuz
Hi, On Mon, 29 Apr 2024 at 18:41, Nazir Bilal Yavuz wrote: > > Hi, > > On Mon, 8 Apr 2024 at 04:21, Thomas Munro wrote: > > > > Pushed. Thanks Bilal and reviewers! > > I wanted to discuss what will happen to this patch now that > 27bc1772fc8 is reverted. I am

Re: Build the docs if there are changes in docs and don't run other tasks if the changes are only in docs

2024-05-15 Thread Nazir Bilal Yavuz
Hi, On Sun, 12 May 2024 at 14:53, Peter Eisentraut wrote: > > On 14.12.23 14:40, Nazir Bilal Yavuz wrote: > > On Fri, 6 Oct 2023 at 17:07, Tom Lane wrote: > >> > >> As a quick cross-check, I searched our commit log to see how many > >> README-only commit

Re: Fix parallel vacuum buffer usage reporting

2024-05-13 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 19:09, Masahiko Sawada wrote: > > On Fri, May 10, 2024 at 7:26 PM Nazir Bilal Yavuz wrote: > > > > Hi, > > > > Thank you for working on this! > > > > On Wed, 1 May 2024 at 06:37, Masahiko Sawada wrote: > > > > &g

Upgrade Debian CI images to Bookworm

2024-05-13 Thread Nazir Bilal Yavuz
/pull/91 [1] postgr.es/m/CAN55FZ0o9wqVoMTh_gJCmj_%2B4XbX9VXzQF8OySPZ0R1saxV3bA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft From 26f4b2425cb04dc7218142f24f151d3698f33191 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Mon, 13 May 2024 10:56:28 +0300 Subject: [PATCH v1] Upgrade

Re: CFbot does not recognize patch contents

2024-05-12 Thread Nazir Bilal Yavuz
Y1e%2B1%3DpB7hXJOyGj1dJOfgde%2BHmiSnv3gDKayUFJMA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 16:55, Nazir Bilal Yavuz wrote: > > Hi, > > On Fri, 10 May 2024 at 16:21, Nazir Bilal Yavuz wrote: > > > > Hi, > > > > On Fri, 10 May 2024 at 14:49, Alena Rybakina > > wrote: > > > > > > Hi! I coul

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 16:21, Nazir Bilal Yavuz wrote: > > Hi, > > On Fri, 10 May 2024 at 14:49, Alena Rybakina wrote: > > > > Hi! I could try to check it with the test, but I want to ask you about > > details, because I'm not sure that I completely underst

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
1 ... ... buffer usage: 29343 hits, 9580 misses in the previous version, 14165 misses in the patched version, 14262 dirtied Patched version counts 14165 misses but the previous version counts 9580 misses in this specific example. -- Regards, Nazir Bilal Yavuz Microsoft diff --git a/src/backe

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
* Skip this block if someone else has already completed it. If an * I/O is already in progress in another backend, this will wait for * the outcome: either done, or something went wrong and we will * retry. */ -- Regards, Nazir Bilal Yavuz Microsoft

Re: gcc 12.1.0 warning

2024-05-10 Thread Nazir Bilal Yavuz
pen in newer versions of postgres is that we aren't > using guc_var_compare() in the relevant places anymore... The fix is attached. It cleanly applies from REL_15_STABLE to REL_12_STABLE, fixes the warnings and the tests pass. -- Regards, Nazir Bilal Yavuz Microsoft From 588c99f5c402fc41414702133636e5

Re: CREATE DATABASE with filesystem cloning

2024-05-08 Thread Nazir Bilal Yavuz
Hi, On Wed, 8 May 2024 at 16:58, Robert Haas wrote: > > On Tue, May 7, 2024 at 8:00 AM Nazir Bilal Yavuz wrote: > > We had an off-list talk with Thomas and we thought making this option > > GUC instead of SQL command level could solve this problem. > > > > I am

Re: CREATE DATABASE with filesystem cloning

2024-05-08 Thread Nazir Bilal Yavuz
Hi, On Wed, 8 May 2024 at 15:23, Ranier Vilela wrote: > > Em qua., 8 de mai. de 2024 às 08:42, Nazir Bilal Yavuz > escreveu: >> >> Hi, >> >> On Wed, 8 May 2024 at 14:16, Ranier Vilela wrote: >> > >> > >> > Em qua., 8 de mai. de 2024

Re: CREATE DATABASE with filesystem cloning

2024-05-08 Thread Nazir Bilal Yavuz
Hi, On Wed, 8 May 2024 at 14:16, Ranier Vilela wrote: > > > Em qua., 8 de mai. de 2024 às 04:37, Nazir Bilal Yavuz > escreveu: >> >> Hi Ranier, >> >> Thanks for looking into this! >> >> I am not sure why but your reply does not show up in th

Re: CREATE DATABASE with filesystem cloning

2024-05-08 Thread Nazir Bilal Yavuz
ID %u already exists", dboid)); } else { /* Select an OID for the new database if is not explicitly configured. */ do { dboid = GetNewOidWithIndex(pg_database_rel, DatabaseOidIndexId, Anum_pg_database_oid);

Re: CREATE DATABASE with filesystem cloning

2024-05-07 Thread Nazir Bilal Yavuz
have the full responsibility. Any kind of feedback would be appreciated. -- Regards, Nazir Bilal Yavuz Microsoft From 404e301dbdb252c23ea9d451b817cf6e372d0d9a Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Tue, 7 May 2024 14:16:09 +0300 Subject: [PATCH v5] Use CLONE method with GUC on

Re: Use streaming read API in ANALYZE

2024-04-29 Thread Nazir Bilal Yavuz
Any kind of feedback would be appreciated. [1] https://www.postgresql.org/message-id/CAAKRu_ZxU6hucckrT1SOJxKfyN7q-K4KU1y62GhDwLBZWG%2BROg%40mail.gmail.com [2] https://www.postgresql.org/message-id/CAAKRu_YkphAPNbBR2jcLqnxGhDEWTKhYfLFY%3D0R_oG5LHBH7Gw%40mail.gmail.com [3] https://www.postgresql.org/message-id/flat/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft

Re: New committers: Melanie Plageman, Richard Guo

2024-04-26 Thread Nazir Bilal Yavuz
ss and few reverts! Congratulations to both of you! -- Regards, Nazir Bilal Yavuz Microsoft

Re: ci: Allow running mingw tests by default via environment variable

2024-04-25 Thread Nazir Bilal Yavuz
t; the automatically set variables. Right now that's CI_, but I'm not at all > wedded to that. I agree with your thoughts and CI_ prefix. I tested both patches and they work as expected. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-04-24 Thread Nazir Bilal Yavuz
Hi, On Fri, 19 Apr 2024 at 11:01, Nazir Bilal Yavuz wrote: > > On Thu, 18 Jan 2024 at 04:22, Michael Paquier wrote: > > > > > > On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote: > > > > I agree with your points. While the other I/O rela

Re: slightly misleading Error message in guc.c

2024-04-22 Thread Nazir Bilal Yavuz
nge for parameter "work_mem" (64 > kB .. 2147483647 kB) > ? > since the units for work_mem are { "B", "kB", "MB", "GB", and "TB"} > > search `outside the valid range for parameter`, > there are two occurrences in guc.c. Nice find. I agree it could cause confusion. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-04-19 Thread Nazir Bilal Yavuz
Hi, On Mon, 19 Feb 2024 at 10:28, Nazir Bilal Yavuz wrote: > > Hi, > > On Thu, 18 Jan 2024 at 04:22, Michael Paquier wrote: > > > > On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote: > > > I agree with your points. While the other I/O relat

Re: Use streaming read API in ANALYZE

2024-04-16 Thread Nazir Bilal Yavuz
Hi, On Wed, 3 Apr 2024 at 22:25, Nazir Bilal Yavuz wrote: > > Hi, > > Thank you for looking into this! > > On Wed, 3 Apr 2024 at 20:17, Heikki Linnakangas wrote: > > > > On 03/04/2024 13:31, Nazir Bilal Yavuz wrote: > > > Streaming API has been committe

Re: Typos in the code and README

2024-04-16 Thread Nazir Bilal Yavuz
words fixups, 0002 fixes a parameter > with > the wrong name in the prototype and 0003 removes a leftover prototype which > was > accidentally left in a refactoring. I realized two small typos: 'sgmr' -> 'smgr'. You may want to include them in 0001. -- Regards, Nazir Bilal Yavuz Microsoft

Use read streams in CREATE DATABASE command when the strategy is wal_log

2024-04-16 Thread Nazir Bilal Yavuz
) as i; [2] CREATE DATABASE test_1 STRATEGY 'wal_log' TEMPLATE test; [3] https://www.postgresql.org/message-id/CAN55FZ1yGvCzCW_aufu83VimdEYHbG_zuOY3J9JL-nBptyJyKA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft From edcfacec40a70a8747c5f18777b6c28b0fccba7a Mon Sep 17 00:00:00 2001 From

Re: Table AM Interface Enhancements

2024-04-15 Thread Nazir Bilal Yavuz
d to be edited as well since these functions assume block numbers are actual physical on-disk location, right? I mean if the block number is something different than the actual physical on-disk location, the acquire_sample_rows() function looks wrong to me before c6fc50cb4028 as well. -- Regards, Nazir Bilal Yavuz Microsoft

Re: gcc 12.1.0 warning

2024-04-15 Thread Nazir Bilal Yavuz
Hi, On Sat, 13 Apr 2024 at 05:25, Andres Freund wrote: > > Hi, > > On 2023-10-27 13:09:01 +0300, Nazir Bilal Yavuz wrote: > > I was testing 'upgrading CI Debian images to bookworm'. I tested the > > bookworm on REL_15, REL_16 and upstream. REL_16 and upstream f

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Nazir Bilal Yavuz
best way forward. > > > > As I wrote in [1], I didn't observe the issue with clang-18, so maybe it > is fixed already. > Perhaps it's worth rechecking... > > [1] > https://www.postgresql.org/message-id/d2bf3727-bae4-3aee-65f6-caec2c4ebaa8%40gmail.com I had this problem on my local

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Mon, 8 Apr 2024 at 00:01, Nazir Bilal Yavuz wrote: > > Hi, > > On Sun, 7 Apr 2024 at 20:33, Nazir Bilal Yavuz wrote: > > > > Hi, > > > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > > > I had been planning to commit v14 t

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Sun, 7 Apr 2024 at 20:33, Nazir Bilal Yavuz wrote: > > Hi, > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > I had been planning to commit v14 this morning but got cold feet with > > the BMR-based interface. Heikki didn't like it much, and in

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
am objects can be created only using Relations now. There could be read stream users which do not have a Relation but SMgrRelations. So, I created another constructor for the read streams which use SMgrRelations instead of Relations. Related patch is attached. -- Regards, Na

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-04-07 Thread Nazir Bilal Yavuz
Hi Andrey, On Sun, 7 Apr 2024 at 08:29, Andrey M. Borodin wrote: > > > > > On 15 Mar 2024, at 17:12, Nazir Bilal Yavuz wrote: > > > > I did not have the time to check other things you mentioned but I > > tested the read performance. The table size is 5.5GB, I

Re: Use streaming read API in ANALYZE

2024-04-04 Thread Nazir Bilal Yavuz
Hi, On Wed, 3 Apr 2024 at 23:44, Melanie Plageman wrote: > > > I've reviewed the patches inline below and attached a patch that has > some of my ideas on top of your patch. Thank you! > > > From 8d396a42186325f920d5a05e7092d8e1b66f3cdf Mon Sep 17 00:00:00 2001 > &

Re: Use streaming read API in ANALYZE

2024-04-03 Thread Nazir Bilal Yavuz
Hi, Thank you for looking into this! On Wed, 3 Apr 2024 at 20:17, Heikki Linnakangas wrote: > > On 03/04/2024 13:31, Nazir Bilal Yavuz wrote: > > Streaming API has been committed but the committed version has a minor > > change, the read_stream_begin_relation function take

Re: Use streaming read API in ANALYZE

2024-04-03 Thread Nazir Bilal Yavuz
Hi Jakub, Thank you for looking into this and doing a performance analysis. On Wed, 3 Apr 2024 at 11:42, Jakub Wartak wrote: > > On Tue, Apr 2, 2024 at 9:24 AM Nazir Bilal Yavuz wrote: > [..] > > v4 is rebased on top of v14 streaming read API changes. > > Hi Nazir,

Re: Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

2024-04-03 Thread Nazir Bilal Yavuz
ay and committed it. Thanks! Thanks for the commit! -- Regards, Nazir Bilal Yavuz Microsoft

Re: Use streaming read API in ANALYZE

2024-04-03 Thread Nazir Bilal Yavuz
Hi, On Tue, 2 Apr 2024 at 10:23, Nazir Bilal Yavuz wrote: > > v4 is rebased on top of v14 streaming read API changes. Streaming API has been committed but the committed version has a minor change, the read_stream_begin_relation function takes Relation instead of BufferManagerRelation n

Re: Use streaming read API in ANALYZE

2024-04-02 Thread Nazir Bilal Yavuz
Hi, Thanks for the review! On Wed, 27 Mar 2024 at 23:15, Melanie Plageman wrote: > > On Tue, Mar 26, 2024 at 02:51:27PM +0300, Nazir Bilal Yavuz wrote: > > Hi, > > > > On Wed, 28 Feb 2024 at 14:42, Nazir Bilal Yavuz wrote: > > > > > > > >

Re: Building with meson on NixOS/nixpkgs

2024-04-01 Thread Nazir Bilal Yavuz
Hi, >From your prior reply: On Thu, 21 Mar 2024 at 23:44, Wolfgang Walther wrote: > > Nazir Bilal Yavuz: > > 0001 & 0002: Adding code comments to explain why they have fallback > > could be nice. > > 0003: Looks good to me. > > Added some comments in the

Re: Use streaming read API in ANALYZE

2024-03-26 Thread Nazir Bilal Yavuz
Hi, On Wed, 28 Feb 2024 at 14:42, Nazir Bilal Yavuz wrote: > > > The new version of the streaming read API [1] is posted. I updated the > streaming read API changes patch (0001), using the streaming read API > in ANALYZE patch (0002) remains the same. This should make it eas

Re: Streaming I/O, vectored I/O (WIP)

2024-03-18 Thread Nazir Bilal Yavuz
Assert(operation->io_buffers_len > 0);? -- Regards, Nazir Bilal Yavuz Microsoft

Re: Building with meson on NixOS/nixpkgs

2024-03-18 Thread Nazir Bilal Yavuz
to master and do pass CI. I have a small feedback: 0001 & 0002: Adding code comments to explain why they have fallback could be nice. 0003: Looks good to me. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-03-15 Thread Nazir Bilal Yavuz
Hi, On Thu, 7 Mar 2024 at 15:26, Cédric Villemain wrote: > > On 07/03/2024 12:19, Nazir Bilal Yavuz wrote: > > > > I did not test read performance but I am planning to do that soon. I did not have the time to check other things you mentioned but I tested the read performan

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-03-07 Thread Nazir Bilal Yavuz
Hi, On Wed, 6 Mar 2024 at 18:23, Cédric Villemain wrote: > > Hi Nazir, > > > thank you for your review. I comment below. > > > On 05/03/2024 12:07, Nazir Bilal Yavuz wrote: > >> 2. The second one does implement smgrprefetch with range and loops by > >>

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-03-05 Thread Nazir Bilal Yavuz
_WILLNEED'); > vm_relation_fadvise > - > > (1 row) > > Time: 24.967 ms > ``` I confirm that there is a time difference between calling pg_prewarm by full relation and block by block, but IMO this is expected. When pg_prewarm is called by full relation, it do

Re: Use streaming read API in ANALYZE

2024-02-28 Thread Nazir Bilal Yavuz
Hi, On Mon, 19 Feb 2024 at 18:13, Nazir Bilal Yavuz wrote: > > I worked on using the currently proposed streaming read API [1] in ANALYZE. > The patch is attached. 0001 is the not yet merged streaming read API code > changes that can be applied to the master, 0002 is the

Re: Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

2024-02-26 Thread Nazir Bilal Yavuz
Hi, On Fri, 23 Feb 2024 at 15:34, Daniel Gustafsson wrote: > > > On 23 Feb 2024, at 13:09, Nazir Bilal Yavuz wrote: > > > Does errmsg_internal() need to be used all the time when turning elogs > > into ereports? errmsg_internal()'s comment says that "This should

Re: Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

2024-02-23 Thread Nazir Bilal Yavuz
Hi, Thanks for the review! On Thu, 22 Feb 2024 at 16:55, Daniel Gustafsson wrote: > > > On 6 Dec 2023, at 14:03, Nazir Bilal Yavuz wrote: > > > There is an ongoing thread [1] for adding missing SQL error codes to > > PANIC and FATAL error reports in xlogrecover

Use streaming read API in ANALYZE

2024-02-19 Thread Nazir Bilal Yavuz
kind of feedback would be appreciated. [1]: https://www.postgresql.org/message-id/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft From 509e55997c084f831fcbcb46cabe79d4f97aa93e Mon Sep 17 00:00:00 2001 From: Thomas Munro Date

Re: Show WAL write and fsync stats in pg_stat_io

2024-02-18 Thread Nazir Bilal Yavuz
Hi, On Thu, 18 Jan 2024 at 04:22, Michael Paquier wrote: > > On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote: > > I agree with your points. While the other I/O related work is > > happening we can discuss what we should do in the variable op_by

Re: Do away with zero-padding assumption before WALRead()

2024-02-15 Thread Nazir Bilal Yavuz
he reason or are there any other benefits of always reading XLOG_BLCKSZ instead of reading the sufficient part? I tried to search in older threads and code comments but I could not find an explanation. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Simplify documentation related to Windows builds

2024-02-08 Thread Nazir Bilal Yavuz
diffutils package. It is mentioned under the Windows requirements page [1] since it does not come preinstalled. However, I agree that it could be good to mention it under the meson page listing the requirements. [1] https://www.postgresql.org/docs/devel/installation-platform-notes.html#WINDOWS-REQUIREMENTS -- Regards, Nazir Bilal Yavuz Microsoft

Re: Streaming I/O, vectored I/O (WIP)

2024-02-07 Thread Nazir Bilal Yavuz
but we do not check the overflow while doubling it. This could be a problem in longer reads. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Checking MINIMUM_VERSION_FOR_WAL_SUMMARIES

2024-02-02 Thread Nazir Bilal Yavuz
Hi, On Fri, 2 Feb 2024 at 12:11, Artur Zakirov wrote: > > On Fri, 2 Feb 2024 at 09:41, Nazir Bilal Yavuz wrote: > > You seem right, nice catch. Also, this change makes the check in > > > > snprintf(summarydir, sizeof(summarydir), "%s/%s/summaries"

Re: Checking MINIMUM_VERSION_FOR_WAL_SUMMARIES

2024-02-02 Thread Nazir Bilal Yavuz
eem right, nice catch. Also, this change makes the check in snprintf(summarydir, sizeof(summarydir), "%s/%s/summaries", basedir, PQserverVersion(conn) < MINIMUM_VERSION_FOR_PG_WAL ? "pg_xlog" : "pg_wal"); redundant. PQserverVersion(conn) will always be higher than MINIMUM_VERSION_FOR_PG_WAL. -- Regards, Nazir Bilal Yavuz Microsoft

Re: 003_extrafiles.pl test fails on Windows with the newer Perl versions

2024-01-31 Thread Nazir Bilal Yavuz
Hi, On Wed, 31 Jan 2024 at 01:18, Andrew Dunstan wrote: > > Pushed to all live branches. Thanks for the patch. Thanks for the push! -- Regards, Nazir Bilal Yavuz Microsoft

Re: compiling postgres on windows arm using meson

2024-01-30 Thread Nazir Bilal Yavuz
the docs [1]: PostgreSQL will only build for the x64 architecture on 64-bit Windows. So, I think that is expected. [1] https://www.postgresql.org/docs/current/install-windows-full.html#INSTALL-WINDOWS-FULL-64-BIT -- Regards, Nazir Bilal Yavuz Microsoft

Re: 003_extrafiles.pl test fails on Windows with the newer Perl versions

2024-01-30 Thread Nazir Bilal Yavuz
Hi, On Tue, 30 Jan 2024 at 14:21, Nazir Bilal Yavuz wrote: > > It looks like File::Find converts backslashes to slashes in the newer > Perl versions. I tried to find the related commit and found this: > https://github.com/Perl/perl5/commit/414f14df98cb1c9a20f92c5c54948b67c09f07

003_extrafiles.pl test fails on Windows with the newer Perl versions

2024-01-30 Thread Nazir Bilal Yavuz
d. I looked at other File::Find appearances in the code but they do not compare the paths. So, I do not think there is any need to fix them. Any kind of feedback would be appreciated. -- Regards, Nazir Bilal Yavuz Microsoft From b28f48fe7d98d3dd7d2fcf652bfa5c8a4cd1c2d6 Mon Sep 17 00:00:00 2001

Re: Simplify documentation related to Windows builds

2024-01-19 Thread Nazir Bilal Yavuz
My initial testing showed that the directory is the '\usr\bin' subdirectory of the msys2 installation directory in my environment. [1] https://github.com/anarazel/pg-vm-images/blob/main/scripts/windows_install_perl.ps1 -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-17 Thread Nazir Bilal Yavuz
Hi, On Mon, 15 Jan 2024 at 09:27, Michael Paquier wrote: > > On Fri, Jan 12, 2024 at 04:23:26PM +0300, Nazir Bilal Yavuz wrote: > > On Thu, 11 Jan 2024 at 17:28, Melanie Plageman > > wrote: > >> Even if we made a separate view for WAL I/O stats, we would still have

Re: Create shorthand for including all extra tests

2024-01-15 Thread Nazir Bilal Yavuz
Hi, On Wed, 10 Jan 2024 at 23:48, Peter Eisentraut wrote: > > On 05.09.23 19:26, Nazir Bilal Yavuz wrote: > > Thanks for the feedback! I updated the patch, 'needs-private-lo' > > option enables kerberos, ldap, load_balance and ssl extra tests now. > > As was discuss

Re: speed up a logical replica setup

2024-01-12 Thread Nazir Bilal Yavuz
ed [1]. I do not think error is related to this. [1] https://cirrus-ci.com/task/4700394639589376 -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-12 Thread Nazir Bilal Yavuz
Hi, On Thu, 11 Jan 2024 at 17:28, Melanie Plageman wrote: > > On Thu, Jan 11, 2024 at 6:19 AM Nazir Bilal Yavuz wrote: > > > > On Thu, 11 Jan 2024 at 08:01, Michael Paquier wrote: > > > > > > On Wed, Jan 10, 2024 at 07:24:50PM -0500, Melanie Plageman wrote:

Re: make pg_ctl more friendly

2024-01-11 Thread Nazir Bilal Yavuz
Hi, On Wed, 10 Jan 2024 at 06:33, Junwang Zhao wrote: > > Hi Nazir, > > On Tue, Jan 9, 2024 at 9:23 PM Nazir Bilal Yavuz wrote: > > > > v2-0001-PITR-shutdown-should-not-report-error-by-pg_ctl.patch: > > > > -

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-11 Thread Nazir Bilal Yavuz
Hi, On Thu, 11 Jan 2024 at 08:01, Michael Paquier wrote: > > On Wed, Jan 10, 2024 at 07:24:50PM -0500, Melanie Plageman wrote: > > I have code review feedback as well, but I've saved that for my next email. > > Ah, cool. > > > On Wed, Jan 3, 2024 at 8:11 AM Nazir Bila

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-10 Thread Nazir Bilal Yavuz
Hi, On Wed, 10 Jan 2024 at 08:25, Michael Paquier wrote: > > On Wed, Jan 03, 2024 at 04:10:58PM +0300, Nazir Bilal Yavuz wrote: > > > > I thought removing op_bytes completely ( as you said "This patch > > extends it with two more operation sizes, and there ar

Re: make pg_ctl more friendly

2024-01-09 Thread Nazir Bilal Yavuz
pg_ctl: could not start server" "Examine the log output.". nitpick: It would be better if the order of the error message cases and enums is the same ( i.e. 'POSTMASTER_RECOVERY_SHUTDOWN' before 'POSTMASTER_FAILED' in enum ) -- Regards, Nazir Bilal Yavuz Microsoft

Re: Confine vacuum skip logic to lazy_scan_skip

2024-01-05 Thread Nazir Bilal Yavuz
unction ) and the X will be determined by Postgres itself. Do you have something different in your mind? -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-03 Thread Nazir Bilal Yavuz
Hi, On Sun, 31 Dec 2023 at 03:58, Michael Paquier wrote: > > On Tue, Dec 26, 2023 at 03:35:52PM +0300, Nazir Bilal Yavuz wrote: > > On Tue, 26 Dec 2023 at 13:10, Michael Paquier wrote: > >> I am not sure while the whole point of the exercise is to have all the > >>

Re: Show WAL write and fsync stats in pg_stat_io

2023-12-26 Thread Nazir Bilal Yavuz
Hi, On Tue, 26 Dec 2023 at 13:10, Michael Paquier wrote: > > On Tue, Dec 26, 2023 at 11:27:16AM +0300, Nazir Bilal Yavuz wrote: > > Maybe it is better to create a pg_stat_io_wal view like you said > > before. We could remove unused columns and add op_bytes for each > > wr

Re: Show WAL write and fsync stats in pg_stat_io

2023-12-26 Thread Nazir Bilal Yavuz
Hi, On Tue, 26 Dec 2023 at 03:06, Michael Paquier wrote: > > On Mon, Dec 25, 2023 at 04:09:34PM +0300, Nazir Bilal Yavuz wrote: > > On Wed, 9 Aug 2023 at 21:52, Melanie Plageman > > wrote: > >> If there is any combination of BackendType and IOContext which will &

Re: Show WAL write and fsync stats in pg_stat_io

2023-12-25 Thread Nazir Bilal Yavuz
mber of reads as an approximation of the number of > bytes read. I don't actually know what makes more sense. I don't think I > would like having a number for bytes that is not accurate. Also, we have a similar problem in XLogPageRead() in xlogrecovery.c. pg_pread() call tries to read XLOG_BLCKSZ

Re: Remove MSVC scripts from the tree

2023-12-19 Thread Nazir Bilal Yavuz
e not-found external program in "command" I think we need to require sed when dtrace or selinux is found, not by looking at the return value of the get_option().enabled(). Second patch looks good to me. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Build the docs if there are changes in docs and don't run other tasks if the changes are only in docs

2023-12-14 Thread Nazir Bilal Yavuz
rth implementing this logic on buildfarm animals. In case we want to implement the same logic on the CI, I added a new version of the patch; it skips CI completely if the changes are only in the README files. -- Regards, Nazir Bilal Yavuz Microsoft From 6c268233d13262a31965baf2dbb42913d83dab1d Mo

Re: Add code indentation check to cirrus-ci (was Re: Add BF member koel-like indentation checks to SanityCheck CI)

2023-12-14 Thread Nazir Bilal Yavuz
Hi, You may want to check out the WIP patch [1] about adding meson targets to run pgindent by Andres. [1] https://www.postgresql.org/message-id/20231019044907.ph6dw637loqg3lqk%40awork3.anarazel.de -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2023-12-12 Thread Nazir Bilal Yavuz
, the two GUCs, the > contexts, the I/O operation type and the objects is the tricky part of > this patch. If the dependency to PendingWalStats is removed and if > the interface of pgstat_prepare_io_time is improved, things are a bit > cleaner, but it feels like we could do more.. Nya

Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

2023-12-06 Thread Nazir Bilal Yavuz
this error comes up in many places. Would it be reasonable to create a new error code specifically for this? Any kind of feedback would be appreciated. [1] https://www.postgresql.org/message-id/CAPMWgZ8g17Myb5ZRE5aTNowUohafk4j48mZ_5_Zn9JnR5p2u0w%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Micr

Re: Show WAL write and fsync stats in pg_stat_io

2023-12-01 Thread Nazir Bilal Yavuz
ause they count the same thing (block vs system calls) but I agree that this doesn't look good. Any kind of feedback would be appreciated. -- Regards, Nazir Bilal Yavuz Microsoft From 4ad85b11d418ae78237ed70eced6e3b46d086ef5 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Fri, 1 Dec 2023 10:0

Re: Show WAL write and fsync stats in pg_stat_io

2023-12-01 Thread Nazir Bilal Yavuz
extend it to cover WAL read from WAL buffer stats. Yes, I am planning to create a patch for that after this patch is done. Thanks for informing! -- Regards, Nazir Bilal Yavuz Microsoft

SSL tests fail on OpenSSL v3.2.0

2023-11-27 Thread Nazir Bilal Yavuz
ent_ext.key -f - -w' at /Users/admin/pgsql/src/test/perl/PostgreSQL/Test/Cluster.pm line 1997. macOS CI run: https://cirrus-ci.com/task/5128008789393408 I couldn't find the cause yet but just wanted to inform you. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2023-11-20 Thread Nazir Bilal Yavuz
Hi, Thanks for the feedback. On Mon, 20 Nov 2023 at 10:47, Michael Paquier wrote: > > On Thu, Nov 09, 2023 at 02:39:26PM +0300, Nazir Bilal Yavuz wrote: > > There are some differences between pg_stat_wal and pg_stat_io while > > collecting WAL stats. For example in the XL

Re: Failure during Building Postgres in Windows with Meson

2023-11-09 Thread Nazir Bilal Yavuz
Hi, On Thu, 9 Nov 2023 at 18:27, Tristan Partin wrote: > > Can you try with Meson v1.2.3? I tried with Meson v1.2.3 and upstream, both failed with the same error. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Failure during Building Postgres in Windows with Meson

2023-11-09 Thread Nazir Bilal Yavuz
stgres gets built successfully. [1] https://www.postgresql.org/message-id/20231018.113148.1275969479525954369.horikyota@gmail.com -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2023-11-09 Thread Nazir Bilal Yavuz
g_stat_wal's or pg_stat_io's behaviour will be changed. Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2023-11-09 Thread Nazir Bilal Yavuz
d changed 'pgstat_report_wal(false)' to 'pgstat_flush_io(false)' in xlogrecovery.c. I will share the new version of the patchset once I address the feedback. Regards, Nazir Bilal Yavuz Microsoft From e5db5cd6d8c47cadde0539f06bbee22368d17a41 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Thu, 26 Oct 2

Re: Remove unnecessary 'always:' from CompilerWarnings task

2023-11-08 Thread Nazir Bilal Yavuz
Hi, Thanks for the review. On Wed, 8 Nov 2023 at 10:31, Peter Eisentraut wrote: > > On 05.09.23 12:25, Nazir Bilal Yavuz wrote: > > There are multiple 'always:' keywords under the CompilerWarnings task. > > Instead of that, we can use one 'always:' and move the instructions

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-11-06 Thread Nazir Bilal Yavuz
That could be misleading, IMO the test should be skipped if Postgres is not compiled with the injection points. Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2023-11-06 Thread Nazir Bilal Yavuz
Hi, On Tue, 31 Oct 2023 at 16:57, Nazir Bilal Yavuz wrote: > On Thu, 26 Oct 2023 at 09:28, Michael Paquier wrote: > > > > And perhaps just putting that everything that calls > > pgstat_count_io_op_time() under track_io_timing is just natural? > > What's the perfo

Re: Show WAL write and fsync stats in pg_stat_io

2023-10-31 Thread Nazir Bilal Yavuz
Hi, Thank you for the feedback! On Thu, 26 Oct 2023 at 09:28, Michael Paquier wrote: > > On Wed, Sep 20, 2023 at 10:57:48AM +0300, Nazir Bilal Yavuz wrote: > > Any kind of feedback would be appreciated. > > This was registered in the CF, so I have given it a look. No

Re: pgBufferUsage.blk_{read|write}_time are zero although there are pgBufferUsage.local_blks_{read|written}

2023-10-27 Thread Nazir Bilal Yavuz
stuff alone. I think switching it to 'shared' makes sense. That shouldn't confuse existing monitoring queries much as the numbers won't change, right? Also, if we keep 'shared/local' there could be similar complaints to this thread in the future; so, at least adding comments can be helpful. Regards, Nazir Bilal Yavuz Microsoft

Re: gcc 12.1.0 warning

2023-10-27 Thread Nazir Bilal Yavuz
ng 'upgrading CI Debian images to bookworm'. I tested the bookworm on REL_15, REL_16 and upstream. REL_16 and upstream finished successfully but the CompilerWarnings task failed on REL_15 with the same error. gcc version: 12.2.0 CI Run: https://cirrus-ci.com/task/6151742664998912 Regards, Nazir Bilal Yavuz Microsoft

Re: pgBufferUsage.blk_{read|write}_time are zero although there are pgBufferUsage.local_blks_{read|written}

2023-10-17 Thread Nazir Bilal Yavuz
Hi, Thanks for the review! On Tue, 17 Oct 2023 at 11:40, Michael Paquier wrote: > > On Mon, Oct 16, 2023 at 01:07:07PM +0300, Nazir Bilal Yavuz wrote: > > Yes, that could be a better solution. Also, having more detailed stats > > for shared and local buffers is helpful.

Re: pgBufferUsage.blk_{read|write}_time are zero although there are pgBufferUsage.local_blks_{read|written}

2023-10-16 Thread Nazir Bilal Yavuz
ution. Also, having more detailed stats for shared and local buffers is helpful. I updated patches in line with that: 0001: Counts extends same way as a write. 0002: Rename blk_{read|write}_time as shared_blk_{read|write}_time. 0003: Add new local_blk_{read|write}_time variables. Re

Re: [RFC] Add jit deform_counter

2023-10-12 Thread Nazir Bilal Yavuz
statements is bumped to 1.11 with this patch but oldextversions test is not updated. So, I attached a patch for updating oldextversions. Regards, Nazir Bilal Yavuz Microsoft From d3c63a5d68ed76257d110db6377bd3ec859d65a4 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Thu, 12 Oct 2023 14

Re: pgBufferUsage.blk_{read|write}_time are zero although there are pgBufferUsage.local_blks_{read|written}

2023-10-05 Thread Nazir Bilal Yavuz
me? In my opinion if something increments {shared|local}_blks_written, then it needs to be counted in blk_write_time too. I am not sure why it is decided like that. Regards, Nazir Bilal Yavuz Microsoft

Re: bgwriter doesn't flush WAL stats

2023-10-03 Thread Nazir Bilal Yavuz
the push! Actual commitfest entry for the second patch is: https://commitfest.postgresql.org/45/4416/. I sent a second patch to this thread just to show how I found this bug. There is no need to review it, this commitfest entry could be closed as committed. Regards, Nazir Bilal Yavuz Microsoft

Re: pgBufferUsage.blk_{read|write}_time are zero although there are pgBufferUsage.local_blks_{read|written}

2023-10-03 Thread Nazir Bilal Yavuz
shared buffers (in FlushBuffer()). I think it makes sense to propose a > bug fix to stable branches counting blk_write_time for local buffers > as well. I attached the PG16+ (after pg_stat_io) and PG15- (before pg_stat_io) versions of the same patch. Regards, Nazir Bilal

  1   2   >