Re: Portworx snapshots

2018-10-28 Thread Stephen Frost
Greetings,

* Ghislain ROUVIGNAC (g...@sylob.com) wrote:
> Portworx says that on a running PostgreSQL it can:
> 
>- replicate volumes for failover
>- take snapshots of volumes
>- backup volumes

The downside with any snapshot-style approach is that it means that when
you have a failure, you have to go through and replay all the WAL since
the last checkpoint, which is single-threaded and can take a large
amount of time.

This is why PostgreSQL has streaming replication, where we are
constantly sending WAL to the replica and replaying it immediately, and
that also allows us to have synchronous replication that is quorum based
and works with PostgreSQL, unlike what a snapshot level system would
provide.

When doing your testing, I'd strongly recommend that you have a large
max_wal_size, run a large pgbench which writes a lot of data, and see
how long a failover takes with this system.

> Does someone use them in production ?
> How reliable are these features ?
> Are there performance impacts of snapshots ?

I don't know anything about the actual utilization of this in production
or if this implementation is reliable, just to be clear.  My comments
specifically are about the performance of using a snapshot-based
approach (which could be this solution or various other ones).

Thanks!

Stephen


signature.asc
Description: PGP signature


Re: Question about partition table

2018-10-28 Thread Thomas Boussekey
Hello Yuxia,

As time partitioning is a natively available, I recommend you to use
time-partitioning, as it is.

You can find below an example I used to subpartition my transaction table
by quaterly partition.
I have a repository table named miniexport.tby_part_followup to pilot
partition creation.

FOR live_part_table IN
  SELECT id_business_unit,
date_trunc ('quarter', tpf_dt_min) as min_timestamp,
date_trunc ('quarter', tpf_dt_max + interval '3 months') as
max_timestamp
FROM miniexport.tby_part_followup
WHERE tpf_bool_part_BU_Quarter_ok = false
ORDER BY id_business_unit
LIMIT bi_max_bu LOOP


--RAISE NOTICE 'Creating partitions into schema posdata2 for BU %
...', live_part_table.id_business_unit::varchar;
EXECUTE 'CREATE TABLE posdata2.transaction_p' ||
live_part_table.id_business_unit::varchar || ' PARTITION OF
posdata2.transaction FOR VALUES in (' ||
live_part_table.id_business_unit::varchar || ') PARTITION BY range
(transaction_date);';

-- Create quarter partitions
dt_curr_timestamp := live_part_table.min_timestamp;
WHILE dt_curr_timestamp <= live_part_table.max_timestamp LOOP
  -- Set running variables
  SELECT EXTRACT(YEAR FROM dt_curr_timestamp)  INTO int_curr_year;
  SELECT EXTRACT(QUARTER FROM dt_curr_timestamp) INTO
int_curr_quarter;

  --RAISE NOTICE 'Creating SubPartition for BU %, QUARTER %',
live_part_table.id_business_unit::varchar, int_curr_year::varchar || 'Q' ||
int_curr_quarter::varchar;
  EXECUTE 'CREATE TABLE posdata2.transaction_p' ||
live_part_table.id_business_unit::varchar || '_' || int_curr_year::varchar
|| 'Q' || int_curr_quarter::varchar || ' PARTITION OF
posdata2.transaction_p' || live_part_table.id_business_unit::varchar || '
FOR VALUES FROM (''' || dt_curr_timestamp || ''') TO (''' ||
dt_curr_timestamp + interval '3 months' || ''')';
  EXECUTE 'CREATE UNIQUE INDEX transaction_p' ||
live_part_table.id_business_unit::varchar || '_' || int_curr_year::varchar
|| 'Q' || int_curr_quarter::varchar || '_idx01 ON posdata2.transaction_p'
|| live_part_table.id_business_unit::varchar || '_' ||
int_curr_year::varchar || 'Q' || int_curr_quarter::varchar || '
(id_transaction, id_business_unit);';

  -- Increment dt_curr_timestamp value
  dt_curr_timestamp := dt_curr_timestamp + interval '3 months';
END LOOP;

EXECUTE 'UPDATE miniexport.tby_part_followup SET
tpf_bool_part_BU_Quarter_ok = true where id_business_unit = ' ||
live_part_table.id_business_unit::varchar || ';';

a_count = a_count + 1;
--GET DIAGNOSTICS a_count = ROW_COUNT;
END LOOP;


Moreover, Sébastien Lardière wrote a tooling kit to manage time partitions:
https://github.com/slardiere/PartMgr

I hope this helps!
Regards,
Thomas

Le sam. 27 oct. 2018 à 01:05, Yuxia Qiu  a écrit :

> HI Dear PostgreSQL team,
>
>I have created a partition table as bellow:
>*CREATE TABLE* measurement_year_month (
>  logdate date not null,
>  peaktemp int,
>  unitsales int
> ) PARTITION BY RANGE (EXTRACT(YEAR FROM logdate), EXTRACT(MONTH FROM
>  logdate));
>
> so the content for this  column *partexprs* for this table  in
> pg_partitioned_table will be:
>
> ({FUNCEXPR :funcid 1384 :funcresulttype 701 :funcretset false
> :funcvariadic false :funcformat 0 :funccollid 0 :inputcollid 100 :args
> ({CONST :consttype 25 :consttypmod -1 :constcollid 100 :constlen -1
> :constbyval false :constisnull false :location 122 :constvalue 8 [ 32 0 0 0
> 121 101 97 114 ]} {VAR :varno 1 :varattno 1 :vartype 1082 :vartypmod -1
> :varcollid 0 :varlevelsup 0 :varnoold 1 :varoattno 1 :location 132})
> :location 114} {FUNCEXPR :funcid 1384 :funcresulttype 701 :funcretset false
> :funcvariadic false :funcformat 0 :funccollid 0 :inputcollid 100 :args
> ({CONST :consttype 25 :consttypmod -1 :constcollid 100 :constlen -1
> :constbyval false :constisnull false :location 150 :constvalue 9 [ 36 0 0 0
> 109 111 110 116 104 ]} {VAR :varno 1 :varattno 1 :vartype 1082 :vartypmod
> -1 :varcollid 0 :varlevelsup 0 :varnoold 1 :varoattno 1 :location 162})
> :location 142})
>
> My question is: Can I get string value from this column? and how?
> In the end I want to have the bellow result:
>
>
> *Table name* *Partition
> information*
> measurement_year_month(EXTRACT(YEAR FROM logdate),
> EXTRACT(MONTH FROM  logdate))
>
>
> Your help is highly appreciated.
>
> Thanks,
> Yuxia
>
>
>


Re: Enabling bdr in multiple databases on the same postgresql instance/cluster

2018-10-28 Thread Martín Marqués
El 23/10/18 a las 10:24, Daniel Fink (PDF) escribió:
> 
> I already have a running cluster of BDR nodes.
> 
> Now we want to add an additional database on the same hosts.
> 
> Can I just create a new database and then create/join nodes as in this
> description:
> 
> http://bdr-project.org/docs/1.0.3/quickstart-enabling.html

If you refer to the following architecture:

Node1_DB1 <--> Node2_DB1

And you want to create a new database DB2 on Node1 and have it
replicating to DB2 on Node2 with BDR, then yes, you have to follow the
same process used to create the first BDR group.

Keep in mind you'll need more slots and replication connections, so
check the values of max_replication_slots, max_worker_processes and
max_wal_senders.

Regards,

-- 
Martín Marquéshttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services



Re: Ltree: set of allowed charcters is limited to [A-Za-z0-9_]. Could the dash "-" be included?

2018-10-28 Thread joernbs
Hello David,

I think this is a misunderstanding.
The seperator in ltree is the dot (.) , of cause I did not asked to change
that.
I asked about to expand allowed characters in the ltree-string [A-Za-z0-9_]
to [a-zA-Z0-9_/- ] including dash(-), slash(/) and whitespace( ), common
charcaters in wording or real path-names to be transformed into and from
ltree.

Jörn

Am Sa., 27. Okt. 2018 um 18:14 Uhr schrieb David G. Johnston <
david.g.johns...@gmail.com>:

> On Saturday, October 27, 2018, joernbs  wrote:
>
>> Dear friends,
>>
>> I would like to use ltree for search paths in a warehouse application,
>> something like "Material-Entry-01.Main-Aisle.Shelf-Aisle-R07/R08.R07-12-03"
>> Unfortunately I can not use common separators like dash (-) or slash(/)
>>
>> Documentation states only thes characters [A-Za-z0-9_] are allowed.
>> https://www.postgresql.org/docs/10/static/ltree.html
>>
>
> I don’t see how this would be possible to do with the existing type - too
> much potential breakage of existing data.  Your example itself shows why
> using dash as a separator is a bad idea.
>
> David J.
>
>