Re: Invalid byte sequence when importing Oracle BLOB

2021-04-26 Thread Peter J. Holzer
On 2021-04-26 07:45:26 -0500, Ron wrote: > On 4/26/21 7:32 AM, Peter J. Holzer wrote: > > On 2021-04-26 06:49:18 -0500, Ron wrote: > > > The destination is an (RDS) Postgresql 12.5 with encoding UTF8, and is > > > being > > > loaded throu

Re: Invalid byte sequence when importing Oracle BLOB

2021-04-26 Thread Peter J. Holzer
ql bytea column. > > Seven times out of about 60M rows, I get this error: > Psql:909242: ERROR:  invalid byte sequence for encoding "UTF8": 0xed 0xaf 0xbf Decoding UTF8 doesn't make sense for a bytea column. How does that data look like in the file generated by ora2pg?

Re: BRIN index on timestamptz

2021-04-26 Thread Peter J. Holzer
tains say records from the last year and records are normally only updated after one or two days after being created that would probably still work quite well. If there is a substantial number of records which is still updated after a year, it probably won't work at all. hp -- _

Re: archive_commnad parameter question

2021-04-19 Thread Peter J. Holzer
uot; in particular tests whether the argument exists and is a regular file) and the "!" inverts the result. So the whole line checks that the target *doesn't* already exist before attempting to copy over it. hp -- _ | Peter J. Holzer| Story must make more sense

Re: Primary keys and composite unique keys(basic question)

2021-04-08 Thread Peter J. Holzer
I've stopped doing this since the SERIAL type makes it much more convenient to have a separate sequence per table. But of course that means that almost any table will have a row with id 10785 and one with 10875. hp -- _ | Peter J. Holzer| Story must make more sense than realit

Re: PostgreSQL log query's result size

2021-04-07 Thread Peter J. Holzer
finding queries which returned 0 rows than those that returned many rows. And for "size of the result data" I think the number of rows would generally be more useful than the size in bytes. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) |

Re: Using indexes through a plpgsql procedure

2021-04-07 Thread Peter J. Holzer
y renaming the old and new table: begin; create table ep_new(...); -- populate ep_new here drop table if exists ep_old; alter table ep rename to ep_old; alter table ep_new rename to ep; commit; Partitioning should also work but that feels like a hack. hp -- _ | P

Re: Upgrading from 11 to 13

2021-04-03 Thread Peter J. Holzer
On 2021-04-01 21:56:17 -0400, Bruce Momjian wrote: > On Thu, Apr 1, 2021 at 09:55:28PM -0400, Bruce Momjian wrote: > > Here it is with descriptions: > > Sorry, please ignore. Too late. Now we all know the code names for previous PostgreSQL releases. hp -- _ |

Re: = t1 - t0 but t0 + i <> t1 when t1 and t2 timestamptz values and i is an interval value

2021-04-03 Thread Peter J. Holzer
re for practical purposes one and the same, otherwise we would not > have leap seconds as a method of syncing the two. I disagree. We have leap seconds exactly because they are not the same. Atomic clock time just counts at at a constant rate - it doesn't care about the Earth's rotation. People ho

Re: Postgres Analog of Oracle APPEND hint

2021-03-05 Thread Peter J. Holzer
. If you are doing "complicated joins on source tables" that's probably where the bottleneck will be, so you shouldn't worry about the insert speed unless (or until) you notice that the bottleneck is writing the data, not reading it. hp -- _ | Peter J. Holzer| Story must m

Re: Streaming replication between different OS

2021-02-22 Thread Peter J. Holzer
poken in Germany ("DE"). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Streaming replication between different OS

2021-02-22 Thread Peter J. Holzer
cially if they are known to cause trouble. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Please help to improve SELECT JOIN performance, EXPLAIN ANALYZE attached

2021-02-21 Thread Peter J. Holzer
On 2021-02-21 10:14:04 -0700, Michael Lewis wrote: > No issues for us. We have used a low sample rate of 1% or so and gotten some > very useful data. Oh, somehow I never noticed the auto_explain.sample_rate parameter in the docs. Good to know. hp -- _ | Peter J. Holzer|

Re: Please help to improve SELECT JOIN performance, EXPLAIN ANALYZE attached

2021-02-20 Thread Peter J. Holzer
ence with trying this in a real-world workload? (I was never brave enough) hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.

Re: Please help to improve SELECT JOIN performance, EXPLAIN ANALYZE attached

2021-02-20 Thread Peter J. Holzer
o, ...) try to cache that in the application. That probably doesn't change very often and doesn't have to be retrieved from the database every time. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Ch

Re: Please help to improve SELECT JOIN performance, EXPLAIN ANALYZE attached

2021-02-20 Thread Peter J. Holzer
efault so you might want to increase it. The usual recommendation is to start with 25% of the memory (that would be 16 GB in your case) and then see if it gets better if decrease or increase it. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) |

Re: Order by not working

2021-02-19 Thread Peter J. Holzer
e able to use it. In reality that doesn't help non-programmers much (it's still a formal language with precise semantics and the computer will do what you say, not what you mean), but makes it harder for programmers. hp -- _ | Peter J. Holzer| Story must

Re: PostgreSQL Replication

2021-02-19 Thread Peter J. Holzer
ut that's a function of load in general, not the number of applications. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challe

Re: checkpointer and other server processes crashing

2021-02-19 Thread Peter J. Holzer
retty much guarantuees the existence of a connection pool). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!"

Re: checkpointer and other server processes crashing

2021-02-19 Thread Peter J. Holzer
t did not exist before. Or just more of them. I could imagine that switching from Python/Gunicorn to Go increased the number of queries that could be in-flight at the same time. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | |

Re: Slow while inserting and retrieval (compared to SQL Server)

2021-02-19 Thread Peter J. Holzer
this much > time. How much time is "this much time"? Are we talking a few milliseconds here? Less? More? Much more? It's hard to give advice if you don't tell us more than "slower than SQL server". Please be specific. Use actual numbers. hp -- _ | Peter J.

Re: Slow index creation

2021-02-19 Thread Peter J. Holzer
nly takes a few milliseconds. So definitely try that if you need to know where your functions spend their time. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Which partition scheme makes sense for my time based IoT-datagrams?

2021-02-11 Thread Peter J. Holzer
ample (if devices report at random times) or empty (if they all report at midnight and it isn't just after midnight). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "

Re: SV: Insertion time is very high for inserting data in postgres

2021-02-11 Thread Peter J. Holzer
stigate what went wrong than to blindly make some changes to the code. As a first measure I would at least turn on statement logging and/or pg_stat_statements to see which statements are slow, and then investigate the slow statements further. auto_explain might also be useful.

Re: vacuumdb not letting me connect to db

2021-02-07 Thread Peter J. Holzer
finished. FInally among those where the performance was acceptable choose the value which was fastest. (Note: If you do this on the same database, subsequent runs will benefit from work already done, so the take the results with a grain of salt). hp -- _ | Peter J. Holzer|

Re: ransomware

2021-02-02 Thread Peter J. Holzer
ickly on a busy database. The question is: Does that help you? At that point the data is already gone (at least partially), and you can only restore it from backup. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | |

Re: Do we need a way to moderate mailing lists?

2021-01-18 Thread Peter J. Holzer
an image to show a problem, it can be put on > some server and the link could be posted, like this one showing a PANIC > of a system http://www.unixarea.de/fbsd-panic-20210110.jpg That has the disadvantage of not being archived. hp -- _ | Peter J. Holzer| Story must make more sens

Re: Do we need a way to moderate mailing lists?

2021-01-18 Thread Peter J. Holzer
fer to not get an extra copy directly. (but I can live with that). Of course the mailing list server can't filter mails it never sees. Mutt adds a header to indicate the preferences of the sender, but I think that is only recognized by mutt, so it's not a general solution. hp -- _

Re: Do we need a way to moderate mailing lists?

2021-01-18 Thread Peter J. Holzer
t it should be short enough to read as part of the message. Of course this is very subjective and may even depend on my mood (Sometimes I find a 20 line SQL query too long, sometimes I'm happy to dig through 200 lines of Perl code ...). It also depends very much on coding style: If code is badly for

Re: Do we need a way to moderate mailing lists?

2021-01-18 Thread Peter J. Holzer
y because most MUAs displayed only one message at a time. The first MUA I've seen that displayed an entire thread at once was Gmail. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross

Re: System (and DB) upgrade problem., "part 2"

2021-01-18 Thread Peter J. Holzer
ackage contains a script pg_upgradecluster which knows about the distribution-specific directory layout. You would normally use that script instead pg_upgrade directly. Maybe the Fedora package has something similar? hp -- _ | Peter J. Holzer| Story must make more sense th

Re: Set COLLATE on a session level

2020-12-11 Thread Peter J. Holzer
It is language-specific and therefore user-specific if you have international users. (I acknowledge the potential performance problems, but they are the same with an explicit collation clause). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: pg_dump - how to force to show timestamps in client log

2020-11-21 Thread Peter J. Holzer
"$name.$$" $db 2>&1 | ts > log/"$name".$(isodate).log ts is available here: https://github.com/hjp/simple/tree/master/ts hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at

Re: database aliasing options ?

2020-11-11 Thread Peter J. Holzer
ol called vip-manager. Compared to DNS this has the advantage that latency is usually shorter. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Another user error? [RESOLVING]

2020-11-01 Thread Peter J. Holzer
find all errors. If it finds an error, it reports it and aborts the query. So if your statement contains more than one error (which is quite likely in a statement over 2000 lines long), fixing one error will just show the next. hp -- _ | Peter J. Holzer| St

Re: What's the best way to translate MS generated translations of user input to what the user actually typed prior to insert or update into PG backend table ?

2020-10-30 Thread Peter J. Holzer
select replace('a … string …', '…', '...'); or use the chr() function: select replace('a … string …', chr(8230), '...'); I would prefer the former as it is easier to read (as long as the characters are printable), but the latter may be easier to type. hp -- _ | Peter J. Holze

Re: Issue executing query from container

2020-10-30 Thread Peter J. Holzer
the production system as the manual states that auto_explain.log_analyze "can have an extremely negative impact on performance". hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross,

Re: Attaching database

2020-10-25 Thread Peter J. Holzer
ically, the foreign data wrapper) which opens that connection. To the client it looks like it's just accessing a normal table within the same database. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Ch

Re: print formated special characteres

2020-10-17 Thread Peter J. Holzer
isingly hard. I'm not sure what the C standard says about that. But these days I would expect any programming language to get it right at least for the simple cases. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: What's your experience with using Postgres in IoT-contexts?

2020-10-14 Thread Peter J. Holzer
orage also means backup and recovery and I don't think you > have that planned for your IOT. That depends on how valuable those data are. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-

Re: Column aliases in GROUP BY and HAVING

2020-10-12 Thread Peter J. Holzer
On 2020-10-12 10:40:03 -0400, Tom Lane wrote: > "Peter J. Holzer" writes: > > In the GROUP BY clause I can use the alias year which was defined > > earlier in SELECT. > > This is a pretty unfortunate legacy thing that we support because > backwards compatibili

Column aliases in GROUP BY and HAVING

2020-10-12 Thread Peter J. Holzer
appropriateness of using an SSN as an id. This is a completely made-up example. -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: How to migrate column type from uuid to serial

2020-10-10 Thread Peter J. Holzer
On 2020-10-10 11:31:23 +0200, Peter J. Holzer wrote: > On 2020-10-07 20:10:34 +0530, Hemil Ruparel wrote: > > Sorry if this is silly but if it is a 128 bit number, why do we need 32 > > characters to represent it? Isn't 8 bits one byte? > > Yes, 8 bits are 1 byte. But that's

Re: What's your experience with using Postgres in IoT-contexts?

2020-10-10 Thread Peter J. Holzer
On 2020-10-10 11:22:42 +0200, Thorsten Schöning wrote: > Guten Tag Peter J. Holzer, > am Samstag, 10. Oktober 2020 um 10:56 schrieben Sie: > > > Do you plan to move some of that reporting to the IoT devices? (Maybe > > equip them with a display with a dashboa

Re: How to migrate column type from uuid to serial

2020-10-10 Thread Peter J. Holzer
= 256). They could also have used 3 decimal digits (000 - 255) for each byte, but that would have wasted even more space, or they could have used base 32 or 64 for the whole number, but that would make conversion harder. hp -- _ | Peter J. Holzer| Story must make more sense than r

Re: What's your experience with using Postgres in IoT-contexts?

2020-10-10 Thread Peter J. Holzer
eems overkill. I'd simply write them to files. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Writing WAL files

2020-10-10 Thread Peter J. Holzer
ntage of providing an end to end check (do I really get the correct value?), not the database's idea of whether replication is working. (The check is written in Go and buried in a svn repo at work, but I could publish it if there is interest) hp -- _ | Peter

Re: Restoring a database problem

2020-10-03 Thread Peter J. Holzer
database guy second. hp [1] Yes, I know that this doesn't affect connections through Unix sockets. -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: pg_upgrade Python version issue on openSUSE

2020-10-03 Thread Peter J. Holzer
0 years) maintenance periods. So in practical terms, Python 2 isn't dead, it just smells funny. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Can I get some PostgreSQL developer feedback on these five general issues I have with PostgreSQL and its ecosystem?

2020-10-03 Thread Peter J. Holzer
d adjust parameters (this is something a tool could do, and maybe better than a human, but this is getting into AI territory). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, &qu

Re: PostgreSQL processes use large amount of private memory on Windows

2020-09-17 Thread Peter J. Holzer
the size of all postgres processes. Send an alert if one of them is "too large". This should give you a good idea what the processes were doing at the time they allocated that memory, so that you can reproduce the problem. hp -- _ | Peter J. Holzer| Story must make more

Re: ODBC Driver dsplay name

2020-09-05 Thread Peter J. Holzer
The character encoding and therefore the set of characters you can use. Always use PostgreSQL Unicode, unless you have (very old and arguably broken) software which can't handle it. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) |

Re: Most effective and fast way to load few Tbyte of data from flat files into postgresql

2020-08-25 Thread Peter J. Holzer
largest table (or I/O bandwidth). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: UUID or auto-increment

2020-08-10 Thread Peter J. Holzer
9 seconds. I think that as far as index locality is concerned, this is essentially random for most applications. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Cr

Re: n_distinct off by a factor of 1000

2020-06-28 Thread Peter J. Holzer
On 2020-06-24 16:27:35 -0600, Michael Lewis wrote: > On Wed, Jun 24, 2020, 2:35 PM Peter J. Holzer wrote: > > Yes, estimating the number of distinct values from a relatively small > sample is hard when you don't know the underlying distribution. It might > be poss

Re: n_distinct off by a factor of 1000

2020-06-24 Thread Peter J. Holzer
uld probably have to resort to monte carlo simulation or soemthing like that. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Persistent Connections

2020-06-24 Thread Peter J. Holzer
On 2020-06-24 13:55:00 -0400, Bee.Lists wrote: > On Jun 24, 2020, at 6:47 AM, Peter J. Holzer wrote: > > The default is 100. What was your reason for reducing it to such a low > > value? > > “PostgreSQL 9 High Availability” recommended core count * 3. I suspected somethin

Re: Persistent Connections

2020-06-24 Thread Peter J. Holzer
bout a few idle connections. What you shouldn't learn from this is that a pooler will make your problems magically go away. Because it won't. jp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Persistent Connections

2020-06-24 Thread Peter J. Holzer
ries through SSH on the LAN. And maybe some more connections. I can see that this could easily reach 12 connections. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, &qu

Re: n_distinct off by a factor of 1000

2020-06-24 Thread Peter J. Holzer
it can have encountered at most that many different values, which means that it must have encountered each value about 12 or 13 times on average. My guess is that there are relatively few (less than 12) distinct values which make up the bulk (over 90 %) of these tables and a lot (33 million)

Re: ERROR: invalid memory alloc request size 18446744073709551613

2020-06-22 Thread Peter J. Holzer
[2-1] db=bxs,user=postgres COMANDO:  COPY > public.cham_chamada Does this always happen in conjunction with a COPY command or sometimes with other commands, too? If the former, are you copying into the database or out of it? hp -- _ | Peter J. Holzer

Re: The backup API and general purpose backup software

2020-06-22 Thread Peter J. Holzer
On 2020-06-21 17:35:41 -0500, Ron wrote: > On 6/21/20 10:45 AM, Peter J. Holzer wrote: > > On 2020-06-21 10:32:16 -0500, Ron wrote: > > > On 6/21/20 8:28 AM, Peter J. Holzer wrote: > > > > To make a full backup with the "new" (non-exclusive) API,

Re: Netapp SnapCenter

2020-06-22 Thread Peter J. Holzer
e (that assumes of course that you are archiving WALs continuously, but if you don't, you can't do PITR in general, so if you have that requirement you are doing it). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h..

Re: The backup API and general purpose backup software

2020-06-21 Thread Peter J. Holzer
On 2020-06-21 10:32:16 -0500, Ron wrote: > On 6/21/20 8:28 AM, Peter J. Holzer wrote: > > To make a full backup with the "new" (non-exclusive) API, a software > > must do the following > > > > 1. open a connection to the database > > > >

The backup API and general purpose backup software

2020-06-21 Thread Peter J. Holzer
start the database hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Definition of REPEATABLE READ

2020-06-21 Thread Peter J. Holzer
much stronger that warranted by the wording in the standard)? hp [1] http://pmg.csail.mit.edu/papers/icde00.pdf [2] https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-95-51.pdf -- _ | Peter J. Holzer| Story must

Re: i am getting issue as FATAL: password authentication failed for user ' postgres' tried

2020-06-20 Thread Peter J. Holzer
; this problem. > > Should have added to previous post: > > Are you sure that you are using the correct password or that the 'postgres' > user has a password? And that the OP is indeed using the 'postgres' user and not the ' postgres' user (as she wrote in the subject).

Re: Binary encoding of TIMESTAMP WITH TIME ZONE

2020-06-05 Thread Peter J. Holzer
different epoch). That also doesn't include a timezone, so conversion should be straightforward and not require any timezone to be involved. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, &quo

Re: Move configuration files with pg_upgrade

2020-06-04 Thread Peter J. Holzer
ot a general problem - did you get any error messages or warnings during the upgrade? hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ |

Re: Oracle vs. PostgreSQL - a comment

2020-06-01 Thread Peter J. Holzer
, every time I have to deal with one of our legacy Oracle databases I notice quite a few things that PostgreSQL has and Oracle doesn't. But of course that's also not fair. Over the last 6 years I've become quite familiar with PostgreSQL and have forgotten much about Oracle. And those databases are ol

Re: Audit Role Connections

2020-05-29 Thread Peter J. Holzer
11-1] user=m***,db=wds,pid=13918 LOG: disconnection: session time: 0:00:00.117 user=m*** database=wds host=143.130.**.** port=54037 (user names and IP addresses censored for privacy reasons) hp -- _ | Peter J. Holzer| Story must make more sense

Re: Linux Update Experience

2020-05-29 Thread Peter J. Holzer
pened here), so being on relevant announce-lists of having the URL of the repo website handy helps. Sometimes you can force installation (althought that will often cause problems later). In some cases I built my own packages. hp -- _ | Peter J. Holzer| Story must make more sen

Re: Password reset

2020-05-17 Thread Peter J. Holzer
tabase ...). On Linux systems PostgreSQL is usually set up so that the user "postgres" can locally connect without a password. So you would ssh into the server as postgres and then invoke psql and change any passwords. hp -- _ | Peter J. Holzer| Story must make more se

Re: Inherited an 18TB DB & need to backup

2020-05-16 Thread Peter J. Holzer
weeks because you are still evaluating the fancier alternatives. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Column reset all values

2020-05-16 Thread Peter J. Holzer
can skip the table entirely. You can do that with a partial index (WHERE col IS NOT NULL) or maybe even a constraint. So I would drop the full index, update the table and then create a partial index. hp -- _ | Peter J. Holzer| Story must make more sense tha

Re: Enforcing uniqueness on [real estate/postal] addresses

2020-05-16 Thread Peter J. Holzer
ke shed on main street, There might be more than one, so PostgreSQL is correct not to enforce the unique constraint. In the last one there is no street name - it's not unknown, we know that there is none because this is a small village which doesn't have street names, just house numbers. hp --

Re: Column reset all values

2020-05-13 Thread Peter J. Holzer
to drop the index before doing this. You obviously won't need the index afterwards and the database may be able to use HOT updates if there is no index on the column (but that depends on the amount of unused space in each block). hp -- _ | Peter J. Holzer| Story mus

Re: Enforcing uniqueness on [real estate/postal] addresses

2020-05-12 Thread Peter J. Holzer
but different descriptions? What does that mean? To different properties which happen to be at the same place or two descriptions for the same property? (What is an address_identifier_general, btw?) I agree with the rest of posting. hp -- _ | Peter J. Holzer| Story must

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Peter J. Holzer
hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Thoughts on how to avoid a massive integer update.

2020-05-05 Thread Peter J. Holzer
the tables shouldn't bloat much. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: How to move a 11.4 cluster to another Linux host, but empty?

2020-05-03 Thread Peter J. Holzer
want to preserve that a simple initdb doesn't recreate? Configuration? Users and passwords? Other stuff? If you can answer this question, the solution will probably be simple. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | |

Re: How do work tercile, percentile & funcion percentile_cont() ?

2020-04-22 Thread Peter J. Holzer
29*(1/3) = 9.667 and 29*(2/3) = 19.333. These are obviously 10.667 and 20.333 respectively. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www

Re: how to slow down parts of Pg

2020-04-22 Thread Peter J. Holzer
needs to run on a router as close to the bottleneck as possible - typically that means either the border router or the firewall. So it is something the customer's network guy should set up. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) |

Re: pg_restore: could not close data file: Success

2020-04-15 Thread Peter J. Holzer
On 2020-04-15 12:01:46 +0200, Peter J. Holzer wrote: > I'm trying to restore a backup on a different machine and it terminates > with the not really helpful messages: > > pg_restore: [directory archiver] could not close data file: Success > pg_restore: [parallel archiver] a work

pg_restore: could not close data file: Success

2020-04-15 Thread Peter J. Holzer
e application). Does that sound plausible or should I look somewhere else? A web search returned nothing relevant. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Crea

Re: Why there is 30000 rows is sample

2020-04-04 Thread Peter J. Holzer
mpiling, of course - but why would you want to?). The value 100 can be controlled either by changing default_statistics_target or by changing the statistics target of a specific column of a specific table (alter table ... alter column ... set statistics ...) hp -- _ | Peter J. Holzer

Re: Ident authentication failed

2020-03-29 Thread Peter J. Holzer
possible) "md5". Actually, for local connections I prefer "peer". I'm already authenticated by the OS, no need for a (second) password. I should add that you shouldn't use "trust" unless * no connection from other hosts is allowed, and * all users on this host should have fu

Idle sessions keep deleted files open

2020-03-29 Thread Peter J. Holzer
tables, keep an eye on idle sessions - they may keep deleted files around for quite some time. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ |

Re: core. files inside base directory

2020-03-26 Thread Peter J. Holzer
(SYSV), SVR4-style, from 'sleep 120', real uid: ... execfn: '/bin/sleep' ... for each file. (Of course the program won't be "sleep" in your case. To analyze the coredumps further you would have to use a debugger (e.g. gdb). hp -- _ | Peter J. Holzer| Story must make m

Re: Loading 500m json files to database

2020-03-24 Thread Peter J. Holzer
itting the work int batches and executing several batches in parallel probably helps. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature

Re: Could postgres12 support millions of sequences? (like 10 million)

2020-03-22 Thread Peter J. Holzer
On 2020-03-21 13:45:54 -0700, pabloa98 wrote: > On Sat, Mar 21, 2020 at 12:08 PM Peter J. Holzer wrote: > And I think that "care about gaps -> sequence doesn't work" is a > knee-jerk reaction. It's similar to "can't parse HTML with regexps". > True

Re: Could postgres12 support millions of sequences? (like 10 million)

2020-03-22 Thread Peter J. Holzer
On 2020-03-21 14:51:35 -0600, Rob Sargent wrote: > > On Mar 21, 2020, at 1:13 PM, Peter J. Holzer wrote: > > > > On 2020-03-21 12:55:33 -0600, Rob Sargent wrote: > >> To me the description of the ID smacks of database-in-the-name folly. I > >> recognize that

Re: Could postgres12 support millions of sequences? (like 10 million)

2020-03-21 Thread Peter J. Holzer
, the social security number contains the birth date. Invoice numbers, project numbers or court case numbers often contain the year. That's because they are used by *people*, and people like their identifiers to make some kind of sense. The computer doesn't care. hp -- _ | Pet

Re: Could postgres12 support millions of sequences? (like 10 million)

2020-03-21 Thread Peter J. Holzer
On 2020-03-20 17:53:11 -0700, Adrian Klaver wrote: > On 3/20/20 4:29 PM, Peter J. Holzer wrote: > > On 2020-03-20 17:11:42 -0600, Rob Sargent wrote: > > > On Mar 20, 2020, at 4:59 PM, Peter J. Holzer wrote: > > > > On 2020-03-19 16:48:19 -0700, David G. Johnston wro

Re: Could postgres12 support millions of sequences? (like 10 million)

2020-03-20 Thread Peter J. Holzer
On 2020-03-20 17:11:42 -0600, Rob Sargent wrote: > On Mar 20, 2020, at 4:59 PM, Peter J. Holzer wrote: > > On 2020-03-19 16:48:19 -0700, David G. Johnston wrote: > >> First, it sounds like you care about there being no gaps in the records > >> you end > >

Re: Could postgres12 support millions of sequences? (like 10 million)

2020-03-20 Thread Peter J. Holzer
ould work. But that effectively serializes your transactions and may cause some to be aborted to prevent deadlocks. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writin

Formatting output (was: Order by and timestamp)

2020-03-16 Thread Peter J. Holzer
o save the mail to a file and manually undo the line breaks to read it. I rarely bother to do that. * ASCII graphics which only line up in a certain proportional font * text/plain messages with very long lines which really should be paragraphs. hp -- _ | Peter J. Holzer

Re: PostgreSQL10.x client Vs. PostgreSQL 11.x server

2020-03-13 Thread Peter J. Holzer
r-side enhancements. Also, client programs (e.g. psql) may also have some enhancements. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http:/

Re: strange locks on PG 11 with Golang programs

2020-03-11 Thread Peter J. Holzer
unless there is a specific need for a | continuous single database connection. All tutorials I've seen follow this recommendation, so a Go programmer might not even be aware that connections exist. hp -- _ | Peter J. Holzer| Story must make more sense than r

Re: Real application clustering in postgres.

2020-03-10 Thread Peter J. Holzer
On 2020-03-09 09:57:37 +0100, Laurenz Albe wrote: > On Sun, 2020-03-08 at 21:13 +0100, Peter J. Holzer wrote: > > But to be fair, a master/slave setup a la patroni isn't immune against > > "writing junk" either: Not on the hardware level (either of the nodes > > ma

<    1   2   3   4   5   6   >