Re: Sample data generator for performance testing

2024-01-03 Thread Jeremy Schneider
On 1/2/24 11:23 PM, arun chirappurath wrote: > Do we have any open source tools which can be used to create sample data > at scale from our postgres databases? > Which considers data distribution and randomness I would suggest to use the most common tools whenever possible, because then if you

Re: Corruption or wrong results with 14.10?

2023-11-24 Thread Jeremy Schneider
On Thu, Nov 23, 2023 at 7:34 AM Torsten Förtsch wrote: > On Thu, Nov 23, 2023 at 2:29 PM Daniel Westermann (DWE) < > daniel.westerm...@dbi-services.com> wrote: > >> smrdbprod=# select crart_id, chemin, count(*) from smrr_mgr.formula group >> by crart_id, chemin having count(*) > 1; >>

Re: typical active table count?

2023-06-27 Thread Jeremy Schneider
On 6/27/23 12:08 PM, Ron wrote: > On 6/27/23 13:47, Jeremy Schneider wrote: >> On 6/27/23 9:32 AM, Ben Chobot wrote: >>> We certainly have databases where far more than 100 tables are updated >>> within a 10 second period. Is there a specific concern you have? >

Re: typical active table count?

2023-06-27 Thread Jeremy Schneider
On 6/27/23 9:32 AM, Ben Chobot wrote: > We certainly have databases where far more than 100 tables are updated > within a 10 second period. Is there a specific concern you have? > Thank Ben, not a concern but I'm trying to better understand how common this might be. And I think sharing general

typical active table count?

2023-06-27 Thread Jeremy Schneider
Question for other PostgreSQL users On your moderately busy DB, how many different tables might receive at least one change/DML during a 10 second window? 10? 50? 100? More? Ballpark guess off the top of your head. I'm in a discussion & there's questions about whether it's unusual to have more

typical active table count?

2023-06-27 Thread Jeremy Schneider
Question for other PostgreSQL users On your moderately busy DB, how many different tables might receive at least one change/DML during a 10 second window? 10? 50? 100? More? Ballpark guess off the top of your head. I'm in a discussion & there's questions about whether it's unusual to have more

Re: Native Logical Replication Initial Import Qs

2023-06-07 Thread Jeremy Schneider
On 6/7/23 2:12 PM, Don Seiler wrote: > On the logical replication front, the concern is with the initial data > import that happens when the subscription is created (by default). I > know that you can tell the subscription to not copy data and instead use > pg_dump and a replication slot snapshot

Re: Monitoring multixact members growth

2022-08-19 Thread Jeremy Schneider
On 8/19/22 12:52 AM, Vido Vlahinic wrote: > My goal here is to predict where multixact members are growing the > fastest so I can perform manual VACUUM FREEZE only on those tables > > (typically with multi-billion row count) when system is relatively > idle as opposed to just sit and wait for

Re: Returning timestamp with timezone at specified timezone irrespective of client timezone

2020-09-27 Thread Jeremy Schneider
On 9/27/20 20:13, Jeremy Schneider wrote: > create or replace function to_char( >v_tstz timestamp with time zone > ,v_format text > ,v_tz text > ) returns text language plpgsql > immutable parallel safe > as $$ > begin > perform set_config('timezone',v_tz,true)

Re: Returning timestamp with timezone at specified timezone irrespective of client timezone

2020-09-27 Thread Jeremy Schneider
On 9/27/20 16:42, Jeremy Schneider wrote: > On 9/27/20 16:13, Ron wrote: >> On 9/27/20 4:16 PM, aNullValue (Drew Stemen) wrote: >>> What I need is for the ability to return a timestamp with timezone, >>> using the UTC offset that corresponds to a column-defined

Re: Returning timestamp with timezone at specified timezone irrespective of client timezone

2020-09-27 Thread Jeremy Schneider
On 9/27/20 16:13, Ron wrote: > On 9/27/20 4:16 PM, aNullValue (Drew Stemen) wrote: >> What I need is for the ability to return a timestamp with timezone, >> using the UTC offset that corresponds to a column-defined timezone, >> irrespective of the client/session configured timezone. >> >> I have

Re: Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-20 Thread Jeremy Schneider
On 8/20/20 14:46, Tom Lane wrote: > There might be value in a user-invokable tool that runs in an existing > non-crashed database and looks for orphan files, but I'm not aware that > anyone has written one. (Race conditions against concurrent table > creation would be a problem; but probably that

Re: Could not open file pg_xact/0E97

2020-07-20 Thread Jeremy Schneider
> On Jul 18, 2020, at 14:18, Radoslav Nedyalkov wrote: > >  > Well. the vacuum full failed with > > vacuumdb: vacuuming of table "olap.transactions_and_fees_2020_01" in database > "db" failed: ERROR: found xmin 3916900817 from before relfrozenxid 80319533 Do you have checksums enabled for

Re: Efficiently advancing a sequence without risking it going backwards.

2020-07-09 Thread Jeremy Schneider
>> On Jul 9, 2020, at 14:08, Christopher Browne wrote: >  >> On Thu, 9 Jul 2020 at 12:59, Jeremy Schneider >> wrote: > >> >> > On Jul 6, 2020, at 19:06, Paul McGarry wrote: >> > >> > I don't think I can use setval(), because it ri

Re: Efficiently advancing a sequence without risking it going backwards.

2020-07-09 Thread Jeremy Schneider
> On Jul 6, 2020, at 19:06, Paul McGarry wrote: > > I don't think I can use setval(), because it risks making sequences go > backwards, eg: > > 1) Check values > DB1sequence: 1234 > DB2sequence: 1233 (1 behind) > 2) setval('DB2sequence',1234); > > but if between (1) and (2) there are 2

Re: Real application clustering in postgres.

2020-03-06 Thread Jeremy Schneider
On 3/6/20 01:25, Ron wrote: > On 3/5/20 6:07 AM, Laurenz Albe wrote: >> On Thu, 2020-03-05 at 07:45 +, Daulat Ram wrote: >>> Is there any possibility/options to setup a real application clustering in >>> Postgres as in Oracle we have a RAC feature. >> No, and as far as I know nobody feels

Re: PL/pgSQL question about EXCEPTION clause & corrupt records

2020-02-17 Thread Jeremy Schneider
On 2/17/20 08:23, Jeremy Schneider wrote: > FWIW, Bertrand blogged an even faster way to do this about a month ago - > using pageinspect and processing blocks instead of rows > > https://bdrouvot.wordpress.com/2020/01/18/retrieve-postgresql-variable-length-storage-inform

Re: PL/pgSQL question about EXCEPTION clause & corrupt records

2020-02-17 Thread Jeremy Schneider
FWIW, Bertrand blogged an even faster way to do this about a month ago - using pageinspect and processing blocks instead of rows https://bdrouvot.wordpress.com/2020/01/18/retrieve-postgresql-variable-length-storage-information-thanks-to-pageinspect/ -J Sent from my TI-83 > On Feb 17, 2020, at

Re: PL/pgSQL question about EXCEPTION clause & corrupt records

2020-02-14 Thread Jeremy Schneider
> On Feb 14, 2020, at 04:39, Nick Renders wrote: > > I get the following message: > > ERROR: missing chunk number 0 for toast value 8289525 in pg_toast_5572299 What version of PostgreSQL are you running? I’ve seen this a number of times the past couple years; curious if the lurking bug

Re: logical replication on 9.6: replica configuration

2019-05-12 Thread Jeremy Schneider
Because you’re on 9.6, you can’t use the native create subscription/publication sql commands. I’d check out pglogical from 2nd quadrant; this should be supported on RDS 9.6 iirc. https://www.2ndquadrant.com/en/resources/pglogical/pglogical-docs/ That said, I’m a bit curious what security

Re: Hot Standby Conflict on pg_attribute

2019-05-11 Thread Jeremy Schneider
Just a quick footnote: If autovac truncations are frequently causing replica lag, and if this is a problem for you, IIUC one way you can stop autovac from doing the truncations even on older versions is setting old_snapshot_threshold to any value at all besides zero. (On 12+ you can directly

Re: PostgreSQL on Amazon RDS

2019-05-08 Thread Jeremy Schneider
On 5/6/19 23:27, Rashmi V Bharadwaj wrote: > Is there a SQL query or a database parameter setting that I can use from > an external application to determine if the PostgreSQL database is on > cloud (like on Amazon RDS or IBM cloud) or on a non-cloud on-prem > environment? Here's my psqlrc file

Re: Amazon Linux Support?

2019-05-07 Thread Jeremy Schneider
> On Apr 30, 2019, at 07:16, Adrian Klaver wrote: > >> On 4/30/19 2:02 AM, Lewis Shobbrook wrote: >> Hi Guys, >> With the repo changes associated with the April 17 changes, >> https://pgstef.github.io/2019/04/17/one_rpm_to_rule_them_all.html >> It is evident that support for amazon linux has

Re: Transaction Id Space, Freezing and Wraparound

2018-12-08 Thread Jeremy Schneider
On 11/20/18 13:00, Tom Lane wrote: > If the stored XIDs were 64 bits wide, we'd not have to bother with all > of this mess ... but adding another 64 bits to tuple headers would be > a painful space cost, not to mention the storage compatibility issues. People keep saying that. But didn't someone

Re: postgis after pg_upgrade

2018-12-04 Thread Jeremy Schneider
On 11/30/18 05:11, Slavcho Trnkovski wrote: > I have postgresql 9.4 with postgis extension installed (latest version, > 2.4.5). I upgraded postgresql from 9.4 to 9.6.  > After upgrading to 9.6 I get the following result > from PostGIS_full_version() >  select PostGIS_full_version(); > >  ...

Re: Is it possible to sort strings in EBCDIC order in PostgreSQL server?

2018-01-23 Thread Jeremy Schneider
On 12/12/17 10:21, Peter Geoghegan wrote: > ICU supports creating custom collations that reorder upper and lower > case, or digits with scripts (e.g. Latin alphabet characters). See the > documentation -- "23.2.2.3.2. ICU collations". Advanced customization > is possible. I just gave this a try