INSERTS waiting with wait_event is "transactionid"

2021-04-08 Thread Nagaraj Raj
Hi, We are trying to load data around 1Bil records into one table with INSERT statements (not able to use COPY command) and they are been waiting for a lock and the wait_event is "transactionid", I didn't find any information in the documents. Queries have been waiting for hours. Table DDL'sCREA

RE: procedure using CURSOR to insert is extremely slow

2021-04-08 Thread Szalontai Zoltán
Hi Milos, I discuss this kind of rethinking with the team. Perhaps I can copy our database on AWS for you, and you can check it. thanks, Zoltán From: Milos Babic Sent: Thursday, April 8, 2021 8:22 PM To: Szalontai Zoltán Cc: Pgsql Performance Subject: Re: procedure using CURSOR

Re: procedure using CURSOR to insert is extremely slow

2021-04-08 Thread Milos Babic
Hi Zoltan, you should try to rethink the logic behind the query. Numerous if/then/else can be transformed into case-when, or a bunch of unions, which, I'm 100% certain will do much better than row-by-row insertion. However, this is a general note. Still doesn't explain why it takes faster to inse

RE: procedure using CURSOR to insert is extremely slow

2021-04-08 Thread Mike Sofen
Hi Zoltan, I haven’t needed to use a cursor in 20 years of sometimes very complex sql coding. Why? Cursors result in RBAR (row by agonizing row) operation which eliminates the power of set-based sql operations. Performance will always suffer – sometimes to extremes. I’m all about fas

RE: procedure using CURSOR to insert is extremely slow

2021-04-08 Thread Szalontai Zoltán
Hi Milos, Inside the loops there are frequently if / else branches value transformations used. We could not solve it without using a cursor. Regards, Zoltán From: Milos Babic Sent: Thursday, April 8, 2021 2:31 PM To: Szalontai Zoltán Cc: Pgsql Performance Subject: Re: procedure u

Re: str_aggr function not wokring

2021-04-08 Thread aditya desai
Sure!! Thanks for the response. Apologies for multiple questions. Faced this during high priority MSSQL to PostgreSQL migration. Did not see any equivalent of XML PATH which would give desired results. Finally was able to resolve the issue by rewriting the Proc using WITH and string_aggr in combina

Re: str_aggr function not wokring

2021-04-08 Thread aditya desai
Thanks Patrick. I used WITH Query and feeded that output to string_aggr which worked. However it is giving performance issues. Will check on that. THanks. On Thu, Apr 8, 2021 at 5:11 PM Patrick FICHE wrote: > *From:* aditya desai > *Sent:* Thursday, April 8, 2021 1:32 PM > *To:* Pgsql Performan

Re: procedure using CURSOR to insert is extremely slow

2021-04-08 Thread Milos Babic
Hi Zoltan, is there any particular reason why you don't do a bulk insert as: insert into target_table select ... from source_table(s) (with joins etc) Regards, Milos On Thu, Apr 8, 2021 at 1:24 PM Szalontai Zoltán < szalontai.zol...@t-online.hu> wrote: > Hi, > > > > We have a Class db.t

RE: procedure using CURSOR to insert is extremely slow

2021-04-08 Thread Szalontai Zoltán
How to check execution plans? We are in the Loop of the Cursor, and we do insert operations in it. From: Hervé Schweitzer (HER) Sent: Thursday, April 8, 2021 1:40 PM To: Szalontai Zoltán ; pgsql-performance@lists.postgresql.org Subject: Re: procedure using CURSOR to insert is extremely slow

RE: str_aggr function not wokring

2021-04-08 Thread Patrick FICHE
From: aditya desai Sent: Thursday, April 8, 2021 1:32 PM To: Pgsql Performance Subject: str_aggr function not wokring Hi, I need to combine results of multiple rows in one row. I get below error. Could you please help. Query: select string_agg((select '**' || P.PhaseName || ' - ' || R.Rec

Re: procedure using CURSOR to insert is extremely slow

2021-04-08 Thread HER
If you do a delete on the first step without any statistics, you request will do a full scan of the table, which will be slower. Did you check the different execution plans ? From: Szalontai Zoltán Sent: Thursday, April 8, 2021 01:24 PM To: pgsql-performance@li

str_aggr function not wokring

2021-04-08 Thread aditya desai
Hi, I need to combine results of multiple rows in one row. I get below error. Could you please help. Query: select string_agg((select '**' || P.PhaseName || ' - ' || R.Recommendation AS "ABC" from tblAssessmentRecommendation R,tblAssessmentPhases P where R.PhaseID = P.PhaseID Order BY P.sor

procedure using CURSOR to insert is extremely slow

2021-04-08 Thread Szalontai Zoltán
Hi, We have a Class db.t2.medium database on AWS. We use a procedure to transfer data records from the Source to the Target Schema. Transfers are identified by the log_id field in the target table. The procedure is: 1 all records are deleted from the Target table with the actual log_id v