Re: [gentoo-user] Postgresql upgrade

[Andreas K. Huettel]

Am Sonntag, 30. Juli 2017, 01:44:27 CEST schrieb Peter Humphrey:
>
> They are trying though. Daniel Vratil wrote this on the KDE-PIM list
> 
> yesterday:
> > If you use KMail (or Kontact), please help us, the KDE PIM developers, to
> > get a better picture of how you use it so that we know which parts of the
> > software we should focus on, and how we should evolve it in the future. We
> > will use the results of the survey to make the experience of using KMail
> > as best as possible for everyone.
> > 
> > You can fill the survey here: https://survey.kde.org/index.php/852475. It
> > won't take you more than 5 minutes.

If that's the survey I've already filled out then it is a joke. The most 
important points, fault tolerance and stability, are not even mentioned.

-- 
Andreas K. Hüttel
dilfri...@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Postgresql upgrade

[Mick]

On Sunday 30 Jul 2017 00:44:27 Peter Humphrey wrote:
> They are trying though. Daniel Vratil wrote this on the KDE-PIM list 
> 
> yesterday:
> > If you use KMail (or Kontact), please help us, the KDE PIM developers, to
> > get a better picture of how you use it so that we know which parts of the 
> > software we should focus on, and how we should evolve it in the future.
> > We 
> > will use the results of the survey to make the experience of using KMail
> > as best as possible for everyone.
> > 
> > You can fill the survey here: https://survey.kde.org/index.php/852475. It 
> > won't take you more than 5 minutes.
> 
> -- 
> Regards
> Peter

Thank you Peter, I posted my feedback in the hope they may listen to it and 
not make KMail/KDEPIM worse ...  :-)
-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Postgresql upgrade

[Peter Humphrey]

On Saturday 29 Jul 2017 13:03:45 Alan McKinnon wrote:

> Don't use kmail
> 
> Seriously, why are putting up with the pain that POS is causing you?
> You've been posting about serious kmail and akonadi issues for about 4
> years now if memory serves, and it has never gotten better. It probably
> never will :-)

They are trying though. Daniel Vratil wrote this on the KDE-PIM list 
yesterday:

> If you use KMail (or Kontact), please help us, the KDE PIM developers, to
> get a better picture of how you use it so that we know which parts of the 
> software we should focus on, and how we should evolve it in the future. We 
> will use the results of the survey to make the experience of using KMail
> as best as possible for everyone.

> You can fill the survey here: https://survey.kde.org/index.php/852475. It 
> won't take you more than 5 minutes.

-- 
Regards
Peter

Re: [gentoo-user] Postgresql upgrade

[Alan McKinnon]

On 29/07/2017 14:27, Mick wrote:
> 
> On 29 July 2017 at 12:19, Mick  > wrote:
> 
> 
> 
> On 29 July 2017 at 12:03, Alan McKinnon  > wrote:
> 
> 
> Don't use kmail
> 
> Seriously, why are putting up with the pain that POS is causing you?
> You've been posting about serious kmail and akonadi issues for
> about 4
> years now if memory serves, and it has never gotten better. It
> probably
> never will :-)
> 
> It must be well more than 4 years.  What can I say, you are right, I
> must be glutton for punishment. LOL!
> 
> TBH, it has been serving me fine for quite a few years now, although
> the migration to akonadi was a painful affair by all accounts.  This
> was originally caused by me using POP3 and a tonne of filters.  I
> moved to IMAP4 and few filters and it all worked relatively
> painlessly since.
> 
> This however must be a postgresql related error, as it worked fine
> before I removed version 9.5.  Any idea what this "Invalid database
> object during initial database connection" might be and how to
> recover from it?
> 
> 
> Hmm, even more symlinks have been broken,  This time I spotted
> /usr/bin/psql pointing to a non-existent
> ../lib64/postgresql-9.5/bin/psql.  I came across it when I tried to
> connect to the database manually and couldn't run psql, but could run
> psql96.  I'm still not sure if I did something wrong this time during
> the upgrade and migration and what it might have been, of if something
> else is amiss and merits a bug report.
> 
> akonadi is still uncooperative.  :-(
> 
> It must be related to whatever it runs to start postgres.

Ah wait. Ignore my last mail from 1 minute ago. I mis-read what you were
saying. Sorry for the noise



-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Postgresql upgrade

[Alan McKinnon]

On 29/07/2017 14:27, Mick wrote:
> 
> On 29 July 2017 at 12:19, Mick  > wrote:
> 
> 
> 
> On 29 July 2017 at 12:03, Alan McKinnon  > wrote:
> 
> 
> Don't use kmail
> 
> Seriously, why are putting up with the pain that POS is causing you?
> You've been posting about serious kmail and akonadi issues for
> about 4
> years now if memory serves, and it has never gotten better. It
> probably
> never will :-)
> 
> It must be well more than 4 years.  What can I say, you are right, I
> must be glutton for punishment. LOL!
> 
> TBH, it has been serving me fine for quite a few years now, although
> the migration to akonadi was a painful affair by all accounts.  This
> was originally caused by me using POP3 and a tonne of filters.  I
> moved to IMAP4 and few filters and it all worked relatively
> painlessly since.
> 
> This however must be a postgresql related error, as it worked fine
> before I removed version 9.5.  Any idea what this "Invalid database
> object during initial database connection" might be and how to
> recover from it?


That's the fancy, decorated, code equivalent of "something went wrong".
It's semantically meaningless, but it does say that something went wrong
at the beginning steps somewhere...


> Hmm, even more symlinks have been broken,  This time I spotted
> /usr/bin/psql pointing to a non-existent
> ../lib64/postgresql-9.5/bin/psql.  I came across it when I tried to
> connect to the database manually and couldn't run psql, but could run
> psql96.  I'm still not sure if I did something wrong this time during
> the upgrade and migration and what it might have been, of if something
> else is amiss and merits a bug report.
> 
> akonadi is still uncooperative.  :-(
> 
> It must be related to whatever it runs to start postgres.

Backup the postgres configs and database files, emerge -C all postgres
versions, make sure there are no files left with postgres in the name,
and emerge the version back that you want. Restore your backed up
configs just in case the ebuild wreaked them. Start postgres, the db
should be unaffected as all you did was replace binary code files.


-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Postgresql upgrade

[Mick]

On 29 July 2017 at 12:19, Mick  wrote:

>
>
> On 29 July 2017 at 12:03, Alan McKinnon  wrote:
>
>>
>> Don't use kmail
>>
>> Seriously, why are putting up with the pain that POS is causing you?
>> You've been posting about serious kmail and akonadi issues for about 4
>> years now if memory serves, and it has never gotten better. It probably
>> never will :-)
>>
>> It must be well more than 4 years.  What can I say, you are right, I must
> be glutton for punishment. LOL!
>
> TBH, it has been serving me fine for quite a few years now, although the
> migration to akonadi was a painful affair by all accounts.  This was
> originally caused by me using POP3 and a tonne of filters.  I moved to
> IMAP4 and few filters and it all worked relatively painlessly since.
>
> This however must be a postgresql related error, as it worked fine before
> I removed version 9.5.  Any idea what this "Invalid database object during
> initial database connection" might be and how to recover from it?
>

Hmm, even more symlinks have been broken,  This time I spotted
/usr/bin/psql pointing to a non-existent ../lib64/postgresql-9.5/bin/psql.
I came across it when I tried to connect to the database manually and
couldn't run psql, but could run psql96.  I'm still not sure if I did
something wrong this time during the upgrade and migration and what it
might have been, of if something else is amiss and merits a bug report.

akonadi is still uncooperative.  :-(

It must be related to whatever it runs to start postgres.
-- 
Regards,
Mick

Re: [gentoo-user] Postgresql upgrade

[Mick]

On 29 July 2017 at 12:03, Alan McKinnon  wrote:

>
> Don't use kmail
>
> Seriously, why are putting up with the pain that POS is causing you?
> You've been posting about serious kmail and akonadi issues for about 4
> years now if memory serves, and it has never gotten better. It probably
> never will :-)
>
> It must be well more than 4 years.  What can I say, you are right, I must
be glutton for punishment. LOL!

TBH, it has been serving me fine for quite a few years now, although the
migration to akonadi was a painful affair by all accounts.  This was
originally caused by me using POP3 and a tonne of filters.  I moved to
IMAP4 and few filters and it all worked relatively painlessly since.

This however must be a postgresql related error, as it worked fine before I
removed version 9.5.  Any idea what this "Invalid database object during
initial database connection" might be and how to recover from it?
-- 
Regards,
Mick

Re: [gentoo-user] Postgresql upgrade

[Alan McKinnon]

On 29/07/2017 13:01, Mick wrote:
> Right, I would report it, except I can't recall if I ran eselect after
> isntalling 9.6. ... I am *almost* sure I did.
>  
> 
> Meanwhile fix the symlinks to what they should be using "ln -sfn" and
> all will be good in the world.
> 
> --
> Alan McKinnon
> alan.mckin...@gmail.com 
> 
> Thanks Alan, I was hoping this would be the case.  However, the blasted
> akonadi, which is why I am running a database on a laptop in the first
> place, refuses to start:

Don't use kmail

Seriously, why are putting up with the pain that POS is causing you?
You've been posting about serious kmail and akonadi issues for about 4
years now if memory serves, and it has never gotten better. It probably
never will :-)


-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Postgresql upgrade

[Mick]

On 29 July 2017 at 11:12, Alan McKinnon  wrote:

> On 29/07/2017 11:59, Mick wrote:
> > It seems this is one of these things I keep forgetting how to perform
> > correctly, despite taking notes and reading the documentation.  I
> thought I
> > had upgraded postgresql from 9.5.7 to 9.6.3-r1 a couple of weeks ago.
> >
> > Today depclean asked me to remove 9.5.7 and after a moment's hesitation
> I went
> > along with it.  To my surprise I got this at the end of it:
> >
> > [snip...]
> > <<<  dir /usr/include/postgresql-9.5/server/catalog
> > <<<  dir /usr/include/postgresql-9.5/server/bootstrap
> > <<<  dir /usr/include/postgresql-9.5/server/access
> > <<<  dir /usr/include/postgresql-9.5/server
> > <<<  dir /usr/include/postgresql-9.5/libpq
> > <<<  dir /usr/include/postgresql-9.5/internal/libpq
> > <<<  dir /usr/include/postgresql-9.5/internal
> > <<<  dir /usr/include/postgresql-9.5/informix/esql
> > <<<  dir /usr/include/postgresql-9.5/informix
> > <<<  dir /usr/include/postgresql-9.5
> > --- !empty   dir /usr/include
> > --- !empty   dir /usr/bin
> > --- !empty   dir /usr
> > --- !empty   dir /etc/postgresql-9.5
> > --- !empty   dir /etc/pam.d
> > --- !empty   dir /etc/init.d
> > --- !empty   dir /etc/conf.d
> > --- !empty   dir /etc
> > Unsetting 9.5 as default...done.
> > Setting 9.6 as the default...ln: failed to create symbolic link
> > '/usr/include/libpq-fe.h': File exists
> > !!! Error: Unable to create link! postgresql-9.6/libpq-fe.h ->
> > /usr/include/libpq-fe.h
> > exiting
>  Regenerating /etc/ld.so.cache...
> > Packages installed:   1321
> > Packages in world:216
> > Packages in system:   44
> > Required packages:1321
> > Number removed:   1
> >
> >
> > Looking at /usr/include/ I see this:
> >
> > lrwxrwxrwx   1 root root 29 Jul 16 13:56 postgres_ext.h ->
> > postgresql-9.5/postgres_ext.h
> > lrwxrwxrwx   1 root root 14 Jul 29 10:34 postgresql -> postgresql-9.6
> > drwxr-xr-x   6 root root   4096 Jul 16 13:55 postgresql-9.6
> >
> > Although the old /usr/include/postgresql-9.5 directory and file
> postgres_ext.h
> > have been removed, the symlink is still pointint to the old file, rather
> than
> > having been replaced with a symlink to
> > /usr/include/postgresql-9.6/postgres_ext.h:
> >
> > $ ls -la /usr/include/postgresql-9.6/postgres_ext.h
> > -rw-r--r-- 1 root root 2151 Jul 16 13:54
> > /usr/include/postgresql-9.6/postgres_ext.h
> >
> >
> > I'm trying to understand why this might have happened.  Which
> process/action
> > is responsible for setting this symlink?
> >
> > Also, the entry run by depclean above also confused me:
> >
> > Unsetting 9.5 as default...done.
> > Setting 9.6 as the default...
> >
> > How is the default version of postgresql being set in a system?  What
> specific
> > actions do these two entries entail?  I thought with openrc at least it
> is a
> > matter of setting up the latest postgresql version to start up in
> rc-update.
> >
> > I've replaced the symlink with the live postgresql manually for now - I
> hope I
> > haven't borked the database ...
> >
>
>
> postgresql is slotted, so various headers files and such need symlinks
> installed so postgresql uses the correct versioned file. You have 2
> versions, which one is correct? - the one the symlink points to.
>
> Yes, but what confused me is that this symlink was correct:

 postgresql -> postgresql-9.6

while all others weren't.


> Installing 9.6 on a host with 9.5 present does only that - installs it.
> It doesn't run it or set it as default, it only installs it. To run it
> or set it as default is an extra step that you decide yourself when you
> want to do it.
>
> depcleaning 9.5 removes the default version, so the obvious thing for
> the code to do is set 9.6 as the new default. Maybe the ebuild does it,
> maybe it's eselect. Doesn't matter, because something should have
> removed those stale symlinks and didn't. This is a reportable bug
>
> Right, I would report it, except I can't recall if I ran eselect after
isntalling 9.6. ... I am *almost* sure I did.


> Meanwhile fix the symlinks to what they should be using "ln -sfn" and
> all will be good in the world.
>
> --
> Alan McKinnon
> alan.mckin...@gmail.com
>
> Thanks Alan, I was hoping this would be the case.  However, the blasted
akonadi, which is why I am running a database on a laptop in the first
place, refuses to start:

[11:34:27] ~ $ akonadictl start
Starting Akonadi Server...
   done.
Connecting to deprecated signal
QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
[11:34:38] ~ $ QSqlDatabase: QPSQL driver not loaded
QSqlDatabase: available drivers: QSQLITE QPSQL7 QPSQL
Invalid database object during initial database connection
"[
0: akonadiserver(_Z11akBacktracev+0x4a) [0x463ada]
1: akonadiserver() [0x463dab]
2: /lib64/libc.so.6(+0x33170) [0x7f454d39d170]
3: /lib64/libc.so.6(gsignal+0x38) 

Re: [gentoo-user] Postgresql upgrade

[Alan McKinnon]

On 29/07/2017 11:59, Mick wrote:
> It seems this is one of these things I keep forgetting how to perform 
> correctly, despite taking notes and reading the documentation.  I thought I 
> had upgraded postgresql from 9.5.7 to 9.6.3-r1 a couple of weeks ago.
> 
> Today depclean asked me to remove 9.5.7 and after a moment's hesitation I 
> went 
> along with it.  To my surprise I got this at the end of it:
> 
> [snip...]
> <<<  dir /usr/include/postgresql-9.5/server/catalog
> <<<  dir /usr/include/postgresql-9.5/server/bootstrap
> <<<  dir /usr/include/postgresql-9.5/server/access
> <<<  dir /usr/include/postgresql-9.5/server
> <<<  dir /usr/include/postgresql-9.5/libpq
> <<<  dir /usr/include/postgresql-9.5/internal/libpq
> <<<  dir /usr/include/postgresql-9.5/internal
> <<<  dir /usr/include/postgresql-9.5/informix/esql
> <<<  dir /usr/include/postgresql-9.5/informix
> <<<  dir /usr/include/postgresql-9.5
> --- !empty   dir /usr/include
> --- !empty   dir /usr/bin
> --- !empty   dir /usr
> --- !empty   dir /etc/postgresql-9.5
> --- !empty   dir /etc/pam.d
> --- !empty   dir /etc/init.d
> --- !empty   dir /etc/conf.d
> --- !empty   dir /etc
> Unsetting 9.5 as default...done.
> Setting 9.6 as the default...ln: failed to create symbolic link 
> '/usr/include/libpq-fe.h': File exists
> !!! Error: Unable to create link! postgresql-9.6/libpq-fe.h -> 
> /usr/include/libpq-fe.h
> exiting
 Regenerating /etc/ld.so.cache...
> Packages installed:   1321
> Packages in world:216
> Packages in system:   44
> Required packages:1321
> Number removed:   1
> 
> 
> Looking at /usr/include/ I see this:
> 
> lrwxrwxrwx   1 root root 29 Jul 16 13:56 postgres_ext.h -> 
> postgresql-9.5/postgres_ext.h
> lrwxrwxrwx   1 root root 14 Jul 29 10:34 postgresql -> postgresql-9.6
> drwxr-xr-x   6 root root   4096 Jul 16 13:55 postgresql-9.6
> 
> Although the old /usr/include/postgresql-9.5 directory and file 
> postgres_ext.h 
> have been removed, the symlink is still pointint to the old file, rather than 
> having been replaced with a symlink to 
> /usr/include/postgresql-9.6/postgres_ext.h:
> 
> $ ls -la /usr/include/postgresql-9.6/postgres_ext.h 
> -rw-r--r-- 1 root root 2151 Jul 16 13:54 
> /usr/include/postgresql-9.6/postgres_ext.h
> 
> 
> I'm trying to understand why this might have happened.  Which process/action 
> is responsible for setting this symlink?  
> 
> Also, the entry run by depclean above also confused me:
> 
> Unsetting 9.5 as default...done.
> Setting 9.6 as the default...
> 
> How is the default version of postgresql being set in a system?  What 
> specific 
> actions do these two entries entail?  I thought with openrc at least it is a 
> matter of setting up the latest postgresql version to start up in rc-update. 
> 
> I've replaced the symlink with the live postgresql manually for now - I hope 
> I 
> haven't borked the database ...
> 



postgresql is slotted, so various headers files and such need symlinks
installed so postgresql uses the correct versioned file. You have 2
versions, which one is correct? - the one the symlink points to.

Installing 9.6 on a host with 9.5 present does only that - installs it.
It doesn't run it or set it as default, it only installs it. To run it
or set it as default is an extra step that you decide yourself when you
want to do it.

depcleaning 9.5 removes the default version, so the obvious thing for
the code to do is set 9.6 as the new default. Maybe the ebuild does it,
maybe it's eselect. Doesn't matter, because something should have
removed those stale symlinks and didn't. This is a reportable bug

Meanwhile fix the symlinks to what they should be using "ln -sfn" and
all will be good in the world.

-- 
Alan McKinnon
alan.mckin...@gmail.com

[gentoo-user] Postgresql upgrade

[Mick]

It seems this is one of these things I keep forgetting how to perform 
correctly, despite taking notes and reading the documentation.  I thought I 
had upgraded postgresql from 9.5.7 to 9.6.3-r1 a couple of weeks ago.

Today depclean asked me to remove 9.5.7 and after a moment's hesitation I went 
along with it.  To my surprise I got this at the end of it:

[snip...]
<<<  dir /usr/include/postgresql-9.5/server/catalog
<<<  dir /usr/include/postgresql-9.5/server/bootstrap
<<<  dir /usr/include/postgresql-9.5/server/access
<<<  dir /usr/include/postgresql-9.5/server
<<<  dir /usr/include/postgresql-9.5/libpq
<<<  dir /usr/include/postgresql-9.5/internal/libpq
<<<  dir /usr/include/postgresql-9.5/internal
<<<  dir /usr/include/postgresql-9.5/informix/esql
<<<  dir /usr/include/postgresql-9.5/informix
<<<  dir /usr/include/postgresql-9.5
--- !empty   dir /usr/include
--- !empty   dir /usr/bin
--- !empty   dir /usr
--- !empty   dir /etc/postgresql-9.5
--- !empty   dir /etc/pam.d
--- !empty   dir /etc/init.d
--- !empty   dir /etc/conf.d
--- !empty   dir /etc
Unsetting 9.5 as default...done.
Setting 9.6 as the default...ln: failed to create symbolic link 
'/usr/include/libpq-fe.h': File exists
!!! Error: Unable to create link! postgresql-9.6/libpq-fe.h -> 
/usr/include/libpq-fe.h
exiting
>>> Regenerating /etc/ld.so.cache...
Packages installed:   1321
Packages in world:216
Packages in system:   44
Required packages:1321
Number removed:   1


Looking at /usr/include/ I see this:

lrwxrwxrwx   1 root root 29 Jul 16 13:56 postgres_ext.h -> 
postgresql-9.5/postgres_ext.h
lrwxrwxrwx   1 root root 14 Jul 29 10:34 postgresql -> postgresql-9.6
drwxr-xr-x   6 root root   4096 Jul 16 13:55 postgresql-9.6

Although the old /usr/include/postgresql-9.5 directory and file postgres_ext.h 
have been removed, the symlink is still pointint to the old file, rather than 
having been replaced with a symlink to 
/usr/include/postgresql-9.6/postgres_ext.h:

$ ls -la /usr/include/postgresql-9.6/postgres_ext.h 
-rw-r--r-- 1 root root 2151 Jul 16 13:54 
/usr/include/postgresql-9.6/postgres_ext.h


I'm trying to understand why this might have happened.  Which process/action 
is responsible for setting this symlink?  

Also, the entry run by depclean above also confused me:

Unsetting 9.5 as default...done.
Setting 9.6 as the default...

How is the default version of postgresql being set in a system?  What specific 
actions do these two entries entail?  I thought with openrc at least it is a 
matter of setting up the latest postgresql version to start up in rc-update. 

I've replaced the symlink with the live postgresql manually for now - I hope I 
haven't borked the database ...
-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[james]

On 08/12/2016 09:15 AM, R0b0t1 wrote:

On Fri, Aug 12, 2016 at 9:13 AM, R0b0t1  wrote:

On Fri, Aug 12, 2016 at 8:00 AM, james  wrote:

Mathematics ==>Electro-Mechanical Engineering ==>Electronics Engineering
==>DataBase Weenies ==>(accounting)Codes.


The study of anything is really the study of war.


Readers will find it amusing that Machiavelli's writings included
convenient descriptions of pike-and-shot formations as "ASCII" art.




Plausible, but consider some perspective on Mac::

A medical professor once asked her class to submit a one paragraph 
thesis on how Machiavelli works affected modern medicine. When the 
youngest member of the class (quite young actually) Espoused that most 
acknowledge that  Mac was very ill, later in life, His Thesis was that 
that (catastrophic) illness actually had consumed Mac much earlier in 
life and therefore, the study and reading of MAC, was more attributable 
to a manifestation of 'societal sickness', rather than a learned pursuit 
of that which is worthy of pursuit.



He receive a low mark in that class, but truth is truth, especially in 
the eyes of the author, a brilliant truth most often.


There is an ironic posting in Hacker Mews about the lack of credibility 
amongst their customers, when focused on modern psychiatry, you just 
might find in interesting. All other forms of modern medicine receive 
quite high marks, from their customers.



caveat emptor,
James

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[R0b0t1]

On Fri, Aug 12, 2016 at 9:13 AM, R0b0t1  wrote:
> On Fri, Aug 12, 2016 at 8:00 AM, james  wrote:
>> Mathematics ==>Electro-Mechanical Engineering ==>Electronics Engineering
>> ==>DataBase Weenies ==>(accounting)Codes.
>
> The study of anything is really the study of war.

Readers will find it amusing that Machiavelli's writings included
convenient descriptions of pike-and-shot formations as "ASCII" art.

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[R0b0t1]

On Fri, Aug 12, 2016 at 8:00 AM, james  wrote:
> Mathematics ==>Electro-Mechanical Engineering ==>Electronics Engineering
> ==>DataBase Weenies ==>(accounting)Codes.

The study of anything is really the study of war.

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[james]

On 08/11/2016 07:48 AM, Douglas J Hunley wrote:


On Tue, Aug 2, 2016 at 1:51 AM, james > wrote:

Douglas did manage to pull his own bacon from the fire, in the end
of his article, but it wreaks of vendor hyperbole, imho.


Again, not the author


--
{
  "name": "douglas j hunley",
  "email": "doug.hun...@gmail.com ",
  "social": [
{
"blog": "https://hunleyd.github.io/;,
"twitter": "@hunleyd"
}
]
}



IFF I made a logical sequence attachment error {a boo_boo}::
1K apologies

IFF I bruised your ego::
1M apologies

IFF I insulted your pride::
1G apologies

IFelse

My goal was to clear up common ignorance of where the ACID properties
came from::

Mathematics ==>Electro-Mechanical Engineering ==>Electronics Engineering
==>DataBase Weenies ==>(accounting)Codes.

OK? That's my thesis and conclusion:: sprinkle with apologies as 
necessary. I do knowledge that DataBase (weeny) vendors are the 
Microsoft of Robustness and Reliability, espoused by the current state 
of affairs in transaction processes, which is now a staple of modern 
computations, much like MicroSoft made computers so idiots can 
participate too. No arguments therein.


BUT, I take the time to 'educate' folks for a very important reason::
Distributed and parallel processing, now entering it's 
fourth/fifth/sixth/ rendition, offers up fundamental 
mathematically based constructs, that can be realized in 
(electronic)hardware or Software

or both, to build 'systems' that far exceed the robustness of ACID
properties currently found in a current database scheme. Furthermore, 
whores like Oracle, need to be retired from the computational landscape, 
as they are the robber barrons of yore and we just do not need them any 
more. I.E. learn the basics and implement new constructs

in distributed and parallel schemes (aka  the cluster).


Fundamental and sound and proven principals of mathematics and EE 
provide solutions for many 'degrees of freedom' for more robust 
solutions than the Vendor hyperbole of Database vendors. And yes, your 
favorite University, and Wiki*, have failed to accurately document this; 
nothing I can do about that but share, as I am doing here.


OK? So, the interested can do their own research, and others can trudge 
along their merry way. (The apologies are sincere, but, I am a bit 
crass:: no apologies on that note).


hth,
James

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Douglas J Hunley]

On Tue, Aug 2, 2016 at 1:51 AM, james  wrote:

> Douglas did manage to pull his own bacon from the fire, in the end of his
> article, but it wreaks of vendor hyperbole, imho.
>

Again, not the author


-- 
{
  "name": "douglas j hunley",
  "email": "doug.hun...@gmail.com",
  "social": [
{
"blog": "https://hunleyd.github.io/;,
"twitter": "@hunleyd"
}
]
}

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Douglas J Hunley]

On Mon, Aug 1, 2016 at 9:43 AM, james  wrote:

> I guess my point is 'Douglas' is full of stuffing, OR that is what folks
> are doing when they 'role their own solution specifically customized to
> their specific needs' as he alludes to near the end of his commentary? (I'd
> like your opinion of this and maybe some links to current schemes how to
> have ACID/99.999% accurate transactions on clusters of various
> architectures.)  Douglas, like yourself, writes of these things in a very
> lucid fashion, so that is why I'm asking you for your thoughts.


Douglas didn't write the damn thing, merely added it to the discussion
here. Thank you very much


-- 
{
  "name": "douglas j hunley",
  "email": "doug.hun...@gmail.com",
  "social": [
{
"blog": "https://hunleyd.github.io/;,
"twitter": "@hunleyd"
}
]
}

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[R0b0t1]

On Thu, Aug 4, 2016 at 12:08 PM, james  wrote:
>
> Atomicity; Consistency; Isolation, Durability == ACID (so we are all on the
> same page).
>
> Not my thesis. My thesis, inspired by these threads, is that all of these
> (4) properties of ACID, originated in the telephone networks, as separate
> issues.

https://en.wikipedia.org/wiki/Two_Generals'_Problem
http://tvtropes.org/pmwiki/pmwiki.php/Main/OlderThanDirt

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[james]

On 08/04/2016 05:09 AM, J. Roeleveld wrote:

On Tuesday, August 02, 2016 12:16:32 AM james wrote:

On 08/01/2016 11:49 AM, J. Roeleveld wrote:

On Monday, August 01, 2016 08:43:49 AM james wrote:





Way back, when the earth was cooling and we all had dinosaurs for pets,
some of us hacked on AT "3B2" unix systems. They were know for their
'roll back and recovery', triplicated (or more) transaction processes
and 'voters' system to ferret out if a transaction was complete and
correct. There was no ACID, the current 'gold standard' if you believe
what Douglas and other write about concerning databases.



Comparing results of codes run on 3 different processors or separate
machines for agreement withing tolerances, is quite different.  The very
essence of using voting where there a result less that 1.0 (that is
n-1/n or n-x/n  was requisite on identical (replicated) processes all
returning the same result ( expecting either a 0 or 1) returned. Results
being logical or within rounding error of acceptance. Surely we need not
split hairs. I was merely pointing out that the basis telecom systems
formed the early and of widespread transaction processing industries and
is the grand daddy of the ACID model/norms/constructs of modern
transaction processing.


Hmm... I am having difficulty following how ACID and ensuring results are
correct by double or triple checking are related.


Atomicity; Consistency; Isolation, Durability == ACID (so we are all on 
the same page).


Not my thesis. My thesis, inspired by these threads, is that all of 
these (4) properties of ACID, originated in the telephone networks, as 
separate issues. When telephonic switching moved from electro-mechanical 
systems to computers, each of these properties where develop by the 
telephonic software and equipment providers. Banks followed the 
switching systems and these (4) ACID properties were realized to be 
universally useful and instituted  and rebranded as 'transactions'


Database systems, developed by IBM and other quickly realized the value 
of ACID properties in all sorts of forms of data movement and 
modification (ie the transaction).  Database developers and vendors
did not invent ACID properties. Indeed and in fact those properties were 
 first used collectively in the legacy  telephonic systems, best 
desribed by SS(7). Earlier version are a case study in redundancy and 
reliability of those early telecom systems. Granted latency was a big 
problem, that moving from electric circuits to digital circuits was 
fixed; yet still there was the five-nines of quality (99.999%) wonderful.




For massively parallel needs,
distributed processing rules, but it is not trivial


Agreed.







Another point, there are single big GPUs that can be run as thousands of
different processors on either FPGA or GPU, granted using SIMD/MIMD
style processors and thing like 'systolic algorithms' but that sort of
this is out of scope here. (Vulcan might change that, in an open source
kind of way, maybe). Furthermore, GPU resources combined with DDR-5 can
blur the line and may actually be more cost effective for many forms of
transaction processing, but clusters, in their current forms are very
much general purpose machines.


I don't really agree here. For most software, having a really fast CPU helps.
Having a lot of mediocre CPUs means the vast majority isn't doing anything
useful.
Software running on clusters needs to be written with massive parallel
processing in mind. Most developers don't understand this part.


Where did you get the idea that folks builing clusters, are not as 
interested in using the fastest processors possible; dude, that's just 
failed (non-sequitur)logic.


Well this premise of yours is a corollary to my thesis; and the early 
telecom systems developers were historically 'bad ass' and highly 
intelligent. It has taken the software development world decades to 
catch up to key systems attributes of hardware design (redundancy and 
roll-back and recovery). Now that things are digital, you can run codes 
on a variety of different hardware to abstract the properties of ACID

and supercede ACID, with yet more properties of robust hardware design.
(Sadly, even most EE professors are severly lacking in this knowledge). 
Modern EE experts have most of their magic attributed to European 
Mathmeticians, but that's another issue, too complex for the average 
java* coder. Curiously, you can read all about, Hilbert, should you need 
to scratch that itch





My point:: Douglas is dead wrong about ACID being dominated by Databases,
for technical reasons, particularly for advanced teams of experts.


Wikipedia actually disagrees with you:
https://en.wikipedia.org/wiki/ACID
"In computer science, ACID (Atomicity, Consistency, Isolation, Durability) is
a set of properties of database transactions."


Exactly. Database vendors got the ideas and components (literals and 
abstractions
from the telephonics industries to get a leg up on moving 

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[J. Roeleveld]

On Tuesday, August 02, 2016 12:16:32 AM james wrote:
> On 08/01/2016 11:49 AM, J. Roeleveld wrote:
> > On Monday, August 01, 2016 08:43:49 AM james wrote:



> >> Way back, when the earth was cooling and we all had dinosaurs for pets,
> >> some of us hacked on AT "3B2" unix systems. They were know for their
> >> 'roll back and recovery', triplicated (or more) transaction processes
> >> and 'voters' system to ferret out if a transaction was complete and
> >> correct. There was no ACID, the current 'gold standard' if you believe
> >> what Douglas and other write about concerning databases.
> >> 
> >> In essence, (from crusted up memories) a basic (SS7) transaction related
> >> to the local telephone switch, was ran  on 3 machines. The results were
> >> compared. If they matched, the transaction went forward as valid. If 2/3
> >> matched,
> > 
> > And what in the likely case when only 1 was correct?
> 
> 1/3 was a failure, in fact X<1 could be defined (parameter setting) as a
> failure depending on the need.

I actually meant:
system A says true
system B and C say false
And "true" was correct.
(Being devil's advocate here)

> > Have you seen the movie "minority report"?
> > If yes, think back to why Tom Cruise was found 'guilty' when he wasn't and
> > how often this actually occured.
> 
> Apples to Oranges. The (3) "pre-cons" were  not equal, ableit the voted,
> most of the time all three in agreement, but the dominant pre-con was
> always on the correct side of the issue. But that is make-believe.

Ofcourse, but it was the first example that I could come up with.

> Comparing results of codes run on 3 different processors or separate
> machines for agreement withing tolerances, is quite different.  The very
> essence of using voting where there a result less that 1.0 (that is
> n-1/n or n-x/n  was requisite on identical (replicated) processes all
> returning the same result ( expecting either a 0 or 1) returned. Results
> being logical or within rounding error of acceptance. Surely we need not
> split hairs. I was merely pointing out that the basis telecom systems
> formed the early and of widespread transaction processing industries and
> is the grand daddy of the ACID model/norms/constructs of modern
> transaction processing.

Hmm... I am having difficulty following how ACID and ensuring results are 
correct by double or triple checking are related.

> And Douglas is

Which Douglas are you referring to? The one in this thread didn't actually 
write the article he linked to. (Unless he has 2 different identities)

> dead wrong that those sorts of (ACID) transactions cannot be made to fly
> on clusters versus a single machine.

It depends on how you define a cluster. I tend to view a cluster as a single 
system that just happens to be spread over multiple physical boxes.

> For massively parallel needs,
> distributed processing rules, but it is not trivial

Agreed.

> and hence Uber, with
> mostly a bunch of kids, seems to be struggling and have made bad
> decisions.

Lets ignore if the decisions are good or bad. Only thing we can be certain of, 
without seeing their code and environment, is that it doesn't scale the way 
they need it to.

> Prolly, there mid managers and software architects are the
> weak link, or they did get expert guidance that was not inhouse, or poor
> decisions to get some code running quickly etc etc. I do not really care
> about UBER.

Neither do I. And decisions are usually made by a single architect or 
developer who starts the project. His/her manager usually just accepts his/her 
word on this and all future decisions. Up until the moment the manager gets 
replaced. Then it depends on how much the manager trusts the original 
developer.
Other developers (internal or external) usually have a hard time pointing out 
potential issues if the first developer doesn't agree and/or understand.

> My singular issue is Douglas was completely dead wrong
> (which nicely promoted himself as a postgress expert and his business
> credentals, and just barely saved his credibility by stating what UBER
> is now doing that is superior to a grade ACID, dB solution.

I didn't see that in the article. Must have missed that part.

> Another point, there are single big GPUs that can be run as thousands of
> different processors on either FPGA or GPU, granted using SIMD/MIMD
> style processors and thing like 'systolic algorithms' but that sort of
> this is out of scope here. (Vulcan might change that, in an open source
> kind of way, maybe). Furthermore, GPU resources combined with DDR-5 can
> blur the line and may actually be more cost effective for many forms of
> transaction processing, but clusters, in their current forms are very
> much general purpose machines.

I don't really agree here. For most software, having a really fast CPU helps. 
Having a lot of mediocre CPUs means the vast majority isn't doing anything 
useful.
Software running on clusters needs to be written with massive parallel 

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Rich Freeman]

On Fri, Jul 29, 2016 at 4:58 PM, Mick  wrote:
> Interesting article explaining why Uber are moving away from PostgreSQL.  I am
> running both DBs on different desktop PCs for akonadi and I'm also running
> MySQL on a number of websites.  Let's which one goes sideways first.  :p
>
>  https://eng.uber.com/mysql-migration/
>

There is a thread on this on the Postgres lists as well (unsurprisingly):

https://www.postgresql.org/message-id/flat/579795DF.10502%40commandprompt.com#579795df.10...@commandprompt.com

I'm only halfway through it but the Postgres devs strike me as being
very levelheaded and competent.  They seem to acknowledge the genuine
issues and point of some of the tradeoffs that Uber is making without
pointing them out.

One thing I really did like about the Uber post was that even if it
isn't a complete/fair comparison/etc it is really informative as an
introduction into how some of the architecture works.  The same
applies to much of the Postgres thread.  I found it really useful for
understanding how both indexing/replication solutions work under the
hood.

-- 
Rich

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[J. Roeleveld]

On Monday, August 01, 2016 09:07:05 PM Rich Freeman wrote:
> On Mon, Aug 1, 2016 at 1:31 PM, J. Roeleveld  wrote:
> > On Monday, August 01, 2016 11:01:28 AM Rich Freeman wrote:
> >> Neither my employer nor the big software provider
> >> in question is likely to attract top-notch DB talent (indeed, mine has
> >> steadily gotten rid of anybody who knows how to do anything in Oracle
> >> beyond creating schemas it seems,
> > 
> > Actively? Or by simply letting the good ones go while replacing them with
> > someone less clued up?
> 
> A bit of both.  A big part of it was probably sacking anybody doing
> anything other than creating tables (since you can't keep operating
> without that), and outsourcing to 3rd parties and wanting
> bottom-dollar prices.

Yes, one of the more common decisions. Often because the person hired to 
handle the department comes from an outsourcing company or because they happen 
to meet at the golf course.

> There are accidentally some reasonably competent people in IT at my
> company, but I don't think it is because we really are good at
> targeting world-class talent.

I wonder which companies are actually good at that?

> > The problem is that the likes of Informatica (one
> > of the leading ETL software vendors) don't actually support PostgreSQL.
> 
> Please tell me that it actually does support xml in a sane way, and it
> is only our incompetent developers who seem to be hand-generating xml
> files by printing strings?


There are actually 2 supported methods (not counting randomly sticking strings 
together):

1) The default XML handling (source/target and transformation). This sort-of 
works for "simple" XML files. The definition for "simple" is in the sales-
contract: No more then ?? levels deep, XSD less then ???MB and XML file less 
than ???MB. I don't remember the actual numbers, but check with whoever has 
the actual contract in your company. It should be listed there or call 
Informatica support.

2) B2B / UDO. The UDO stands for Unstructured Data Option. Bit strange, but 
that's where it lives. It's a proper XML handling engine that should be able 
to handle any XML you care to throw at it. Also documents with a standardised 
layout. It's the preferred method of handling XML files with Informatica. (Do 
use at least 9.6.1 for this. 9.5 has a very annoying feature...



> I have an integration that involves Informatica, and another solution
> that just synchronizes files from an smb share to a foreign FTP site.
> Of course I don't have access to the share that lies in-between, so
> when the interface breaks I get to play with two different groups to
> try to figure out where the process died.  Informatica appears to be
> running on Unix and I get helpful questions from the maintainers about
> what path the files are on, as if I'd have any idea where some SMB
> share (whose path I am not told) is mounted on some Unix server I have
> no access to.

Check the session-log (from Informatica), that should contain the actual path 
Informatica uses to write the file to.

> Gotta love division of labor.  Heaven forbid anybody have visibility
> to the full picture so that the right group can be engaged on the
> first try...

I see this all too often. They usually claim it's because of security. Not 
understanding that by obscuring all the details, the first person to get the 
full picture is the one going to cause havoc and the people that are then 
tasked to fix it, don't know enough to do it right in a reasonable time-frame.

--
Joost

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[james]

On 08/01/2016 01:03 PM, Rich Freeman wrote:

On Mon, Aug 1, 2016 at 12:49 PM, J. Roeleveld  wrote:

On Monday, August 01, 2016 08:43:49 AM james wrote:


Sure this part is only related to
transaction processing as there was much more to the "five 9s" legacy,
but imho, that is the heart of what was the precursor to ACID property's
now so greatly espoused in SQL codes that Douglas refers to.

Do folks concur or disagree at this point?


ACID is about data integrity. The "best 2 out of 3" voting was, in my opinion,
a work-around for unreliable hardware. It is based on a clever idea, but when
2 computers having the same data and logic come up with 2 different answers, I
wouldn't trust either of them.


I agree, this was a solution for hardware issues.  However, hardware
issues can STILL happen today, so there is an argument for it.  There
are really two ways to get to robustness: clever hardware, and clever
software.  The old way was to do it in hardware, the newer way is to
do it in software (see Google with their racks of cheap motherboards).
I suspect software will always be the better way, but you can't just
write a check to get better software the way you can with hardware.
Doing it right with software means hiring really good people, which is
something a LOT of companies don't want to do (well, they think
they're doing it, but they're not).

Basically I believe the concept with the mainframe was that you could
probably open the thing up, break one random board with a hammer, and
the application would still keep running just fine.  IBM would then
magically show up the next day and replace the board without anybody
doing anything.  All the hardware had redundancy, so you can run your
application for a decade or two without fear of a hardware failure.


Not with todays clusters and cheap hardware. As you pointed out 
expertise (and common sense) are the quintessential qualities for staff 
and managers.




However, you pay a small fortune for all of this.


Not today, that was then those absorbant prices. Sequoia made so much 
money, I pretty sure that how they ultimately became a VC firm?





The other trend as
I understand it in mainframes is renting your own hardware to you.


Yes, find a CPA that spent 10 years or so inside the IRS and you get 
even more aggressive  profitibility vectors. Some accouants move 
hardware, assest and corporations around and about the world in a shell 
game and never pay taxes, just recycling assets among billionares. It's 
pretty sickening, if you really learn the details of what goes on.



That is, you buy a box, and you can just pay to turn on extra
CPUs/etc.  You can imagine what the margins are like for that to be
practical, but for non-trendy businesses that don't want to offer free
ice cream and pay Silicon Valley wages I guess it is an alternative to
building good software.


Investment credits, sell/rent hardware to overseas divison, then move 
them to another country that pays you re-locate and bring a few jobs. 
Heck, event the US stats play that stupid game with recruiting 
corporations. Get and IRA career agent drunk some time and pull a few 
stories out of them.



You have seen how "democracies" work, right? :)
The more voters involved, the longer it takes for all the votes to be counted.
With a small number, it might actually still scale, but when you pass a magic
number (no clue what this would be), the counting time starts to exceed any
time you might have gained by adding more voters.

Also, this, to me, seems to counteract the whole reason for using clusters:
Have different nodes handle a different part of the problem.


I agree.  The old mainframe way of doing things isn't going to make
anything faster.  I don't think it will necessarily make things much
slower as long as all the hardware is in the same box.  However, if
you want to start doing this at a cluster scale with offsite replicas
I imagine the latencies would kill just about anything.  That was one
of the arguments against the Postgres vacuum approach where replicas
could end up having in-use records deleted.  The solutions are to
delay the replicas (not great), or synchronize back to the master
(also not great).  The MySQL approach apparently lets all the replicas
do their own vacuuming, which does neatly solve that particular
problem (presumably at the cost of more work for the replicas, and of
course they're no longer binary replicas).


Why Rich, using common sense? What's wrong with you? I thought you were 
a good corporate lacky?  Bob from accounting has already presented to 
the BOD and got approval. Rich, can you be a team player (silent idiot) 
just once for the team?







The way Uber created the cluster is useful when having 1 node handle all the
updates and multiple nodes providing read-only access while also providing
failover functionality.


I agree.  I do remember listening to a Postgres talk by one of the
devs and while everybody's holy grail 

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[james]

On 08/01/2016 11:49 AM, J. Roeleveld wrote:

On Monday, August 01, 2016 08:43:49 AM james wrote:

On 08/01/2016 02:16 AM, J. Roeleveld wrote:

On Saturday, July 30, 2016 06:38:01 AM Rich Freeman wrote:

On Sat, Jul 30, 2016 at 6:24 AM, Alan McKinnon 


wrote:

On 29/07/2016 22:58, Mick wrote:

Interesting article explaining why Uber are moving away from
PostgreSQL.
I am
running both DBs on different desktop PCs for akonadi and I'm also
running
MySQL on a number of websites.  Let's which one goes sideways first.
:p

 https://eng.uber.com/mysql-migration/


I don't think your akonadi and some web sites compares in any way to
Uber
and what they do.

FWIW, my Dev colleagues support and entire large corporate ISP's
operational and customer data on PostgreSQL-9.3. With clustering. With
no
db-related issues :-)


Agree, you'd need to be fairly large-scale to have their issues,


And also have to design your database by people who think MySQL actually
follows common SQL standards.


but I
think the article was something anybody interested in databases should
read.  If nothing else it is a really easy to follow explanation of
the underlying architectures.


Check the link posted by Douglas.
Ubers article has some misunderstandings about the architecture with
conclusions drawn that are, at least also, caused by their database design
and usage.


I'll probably post this to my LUG mailing list.  I think one of the
Postgres devs lurks there so I'm curious to his impressions.

I was a bit surprised to hear about the data corruption bug.  I've
always considered Postgres to have a better reputation for data
integrity.


They do.


And of course almost any FOSS project could have a bug.  I
don't know if either project does the kind of regression testing to
reliably detect this sort of issue.


Not sure either, I do think PostgreSQL does a lot with regression tests.


I'd think that it is more likely
that the likes of Oracle would (for their flagship DB (not for MySQL),


Never worked with Oracle (or other big software vendors), have you? :)


and they'd probably be more likely to send out an engineer to beg
forgiveness while they fix your database).


Only if you're a big (as in, spend a lot of money with them) customer.


Of course, if you're Uber
the hit you'd take from downtime/etc isn't made up for entirely by
having somebody take a few days to get everything fixed.


--
Joost


I certainly respect your skills and posts on Databases, Joost, as
everything you have posted, in the past is 'spot on'.


Comes with a keen interest and long-term (think decades) of working with
different databases.


Granted, I'm no database expert, far from it.


Not many people are, nor do they need to be.


But I want to share a few thing with you,
and hope you  (and others) will 'chime in' on these comments.

Way back, when the earth was cooling and we all had dinosaurs for pets,
some of us hacked on AT "3B2" unix systems. They were know for their
'roll back and recovery', triplicated (or more) transaction processes
and 'voters' system to ferret out if a transaction was complete and
correct. There was no ACID, the current 'gold standard' if you believe
what Douglas and other write about concerning databases.

In essence, (from crusted up memories) a basic (SS7) transaction related
to the local telephone switch, was ran  on 3 machines. The results were
compared. If they matched, the transaction went forward as valid. If 2/3
matched,


And what in the likely case when only 1 was correct?


1/3 was a failure, in fact X<1 could be defined (parameter setting) as a 
failure depending on the need.



Have you seen the movie "minority report"?
If yes, think back to why Tom Cruise was found 'guilty' when he wasn't and how
often this actually occured.


Apples to Oranges. The (3) "pre-cons" were  not equal, ableit the voted, 
most of the time all three in agreement, but the dominant pre-con was 
always on the correct side of the issue. But that is make-believe. 
Comparing results of codes run on 3 different processors or separate 
machines for agreement withing tolerances, is quite different.  The very 
essence of using voting where there a result less that 1.0 (that is 
n-1/n or n-x/n  was requisite on identical (replicated) processes all 
returning the same result ( expecting either a 0 or 1) returned. Results 
being logical or within rounding error of acceptance. Surely we need not 
split hairs. I was merely pointing out that the basis telecom systems 
formed the early and of widespread transaction processing industries and 
is the grand daddy of the ACID model/norms/constructs of modern 
transaction processing. And Douglas is
dead wrong that those sorts of (ACID) transactions cannot be made to fly 
on clusters versus a single machine. For massively parallel needs,
distributed processing rules, but it is not trivial and hence Uber, with 
mostly a bunch of kids, seems to be struggling and have made bad 
decisions. 

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Rich Freeman]

On Mon, Aug 1, 2016 at 1:31 PM, J. Roeleveld  wrote:
> On Monday, August 01, 2016 11:01:28 AM Rich Freeman wrote:
>> Neither my employer nor the big software provider
>> in question is likely to attract top-notch DB talent (indeed, mine has
>> steadily gotten rid of anybody who knows how to do anything in Oracle
>> beyond creating schemas it seems,
>
> Actively? Or by simply letting the good ones go while replacing them with
> someone less clued up?

A bit of both.  A big part of it was probably sacking anybody doing
anything other than creating tables (since you can't keep operating
without that), and outsourcing to 3rd parties and wanting
bottom-dollar prices.

There are accidentally some reasonably competent people in IT at my
company, but I don't think it is because we really are good at
targeting world-class talent.

>
> The problem is that the likes of Informatica (one
> of the leading ETL software vendors) don't actually support PostgreSQL.

Please tell me that it actually does support xml in a sane way, and it
is only our incompetent developers who seem to be hand-generating xml
files by printing strings?

I have an integration that involves Informatica, and another solution
that just synchronizes files from an smb share to a foreign FTP site.
Of course I don't have access to the share that lies in-between, so
when the interface breaks I get to play with two different groups to
try to figure out where the process died.  Informatica appears to be
running on Unix and I get helpful questions from the maintainers about
what path the files are on, as if I'd have any idea where some SMB
share (whose path I am not told) is mounted on some Unix server I have
no access to.

Gotta love division of labor.  Heaven forbid anybody have visibility
to the full picture so that the right group can be engaged on the
first try...

-- 
Rich

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Rich Freeman]

On Mon, Aug 1, 2016 at 7:18 PM, Alan McKinnon  wrote:
>
> So the original article very much seems to have been written with a skewed
> bias and wrong focus. That's bias as in "shifted to one side as used in
> math" not bias as in "opinionated asshat beating some special drum"
>

Well, I wouldn't say "wrong focus" so much as "particular focus."  The
original article doesn't really purport to be a holistic comparison of
the two systems, just an explanation of why they're migrating.  I
think people are reading a bit too much into it.

However, the original article would probably benefit from a few
caveats thrown in.

-- 
Rich

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Alan McKinnon]

On 01/08/2016 17:01, Rich Freeman wrote:

On Mon, Aug 1, 2016 at 3:16 AM, J. Roeleveld  wrote:

>
> Check the link posted by Douglas.
> Ubers article has some misunderstandings about the architecture with
> conclusions drawn that are, at least also, caused by their database design and
> usage.

I've read it.  I don't think it actually alleges any misunderstandings
about the Postgres architecture, but rather that it doesn't perform as
well in Uber's design.  I don't think it actually alleges that Uber's
design is a bad one in any way.



He does also make the stinger at the end:

On 2013 Uber migrated FROM mysql TO postgres, and now in 2016 they 
migrated FROM postgres TO Schemaless (with just happens to have InnoDB 
as backend).


So the original article very much seems to have been written with a 
skewed bias and wrong focus. That's bias as in "shifted to one side as 
used in math" not bias as in "opinionated asshat beating some special drum"

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Rich Freeman]

On Mon, Aug 1, 2016 at 12:49 PM, J. Roeleveld  wrote:
> On Monday, August 01, 2016 08:43:49 AM james wrote:
>
>> Sure this part is only related to
>> transaction processing as there was much more to the "five 9s" legacy,
>> but imho, that is the heart of what was the precursor to ACID property's
>> now so greatly espoused in SQL codes that Douglas refers to.
>>
>> Do folks concur or disagree at this point?
>
> ACID is about data integrity. The "best 2 out of 3" voting was, in my opinion,
> a work-around for unreliable hardware. It is based on a clever idea, but when
> 2 computers having the same data and logic come up with 2 different answers, I
> wouldn't trust either of them.

I agree, this was a solution for hardware issues.  However, hardware
issues can STILL happen today, so there is an argument for it.  There
are really two ways to get to robustness: clever hardware, and clever
software.  The old way was to do it in hardware, the newer way is to
do it in software (see Google with their racks of cheap motherboards).
I suspect software will always be the better way, but you can't just
write a check to get better software the way you can with hardware.
Doing it right with software means hiring really good people, which is
something a LOT of companies don't want to do (well, they think
they're doing it, but they're not).

Basically I believe the concept with the mainframe was that you could
probably open the thing up, break one random board with a hammer, and
the application would still keep running just fine.  IBM would then
magically show up the next day and replace the board without anybody
doing anything.  All the hardware had redundancy, so you can run your
application for a decade or two without fear of a hardware failure.

However, you pay a small fortune for all of this.  The other trend as
I understand it in mainframes is renting your own hardware to you.
That is, you buy a box, and you can just pay to turn on extra
CPUs/etc.  You can imagine what the margins are like for that to be
practical, but for non-trendy businesses that don't want to offer free
ice cream and pay Silicon Valley wages I guess it is an alternative to
building good software.

>
> You have seen how "democracies" work, right? :)
> The more voters involved, the longer it takes for all the votes to be counted.
> With a small number, it might actually still scale, but when you pass a magic
> number (no clue what this would be), the counting time starts to exceed any
> time you might have gained by adding more voters.
>
> Also, this, to me, seems to counteract the whole reason for using clusters:
> Have different nodes handle a different part of the problem.

I agree.  The old mainframe way of doing things isn't going to make
anything faster.  I don't think it will necessarily make things much
slower as long as all the hardware is in the same box.  However, if
you want to start doing this at a cluster scale with offsite replicas
I imagine the latencies would kill just about anything.  That was one
of the arguments against the Postgres vacuum approach where replicas
could end up having in-use records deleted.  The solutions are to
delay the replicas (not great), or synchronize back to the master
(also not great).  The MySQL approach apparently lets all the replicas
do their own vacuuming, which does neatly solve that particular
problem (presumably at the cost of more work for the replicas, and of
course they're no longer binary replicas).

>
> The way Uber created the cluster is useful when having 1 node handle all the
> updates and multiple nodes providing read-only access while also providing
> failover functionality.

I agree.  I do remember listening to a Postgres talk by one of the
devs and while everybody's holy grail is the magical replica where you
just have a bunch of replicas and you do any operation on any replica
and everything is up to date, in reality that is almost impossible to
achieve with any solution.

-- 
Rich

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[J. Roeleveld]

On Monday, August 01, 2016 11:01:28 AM Rich Freeman wrote:
> On Mon, Aug 1, 2016 at 3:16 AM, J. Roeleveld  wrote:
> > Check the link posted by Douglas.
> > Ubers article has some misunderstandings about the architecture with
> > conclusions drawn that are, at least also, caused by their database design
> > and usage.
> 
> I've read it.  I don't think it actually alleges any misunderstandings
> about the Postgres architecture, but rather that it doesn't perform as
> well in Uber's design.  I don't think it actually alleges that Uber's
> design is a bad one in any way.

It was written quite diplomatic. Seeing the create table for the sample tables 
already make me wonder how they designed their database schema. Especially 
from a performance point of view. But that is a seperate discussion :)

> But, I'm certainly interested in anything else that develops here...

Same here, and I am hoping some others will also come up with some interesting 
bits.

> >> And of course almost any FOSS project could have a bug.  I
> >> don't know if either project does the kind of regression testing to
> >> reliably detect this sort of issue.
> > 
> > Not sure either, I do think PostgreSQL does a lot with regression tests.
> 
> Obviously they missed that bug.  Of course, so did Uber in their
> internal testing.  I've seen a DB bug in production (granted, only one
> so far) and they aren't pretty.  A big issue for Uber is that their
> transaction rate and DB size is such that they really don't have a
> practical option of restoring backups.

>From the slides on their migration from MySQL to PostgreSQL in 2013, I see it 
took them 45 minutes to migrate 50GB of data.
To me, that seems like a very bad transfer-rate for, what I would consider, a 
dev environment. It's only about 20MB/s.
I've seen "bad performing" ETL processes reading from 300GB of XML files and 
loading that into 3 DB-tables within 1.5 hours. That's about 57MB/s.
With the XML-engine using up nearly 98% of the total CPU-load.

If the data would have been supplied in CSV-files, it would have been roughly 
100GB of data. This could be easily loaded within 20 minutes. Equalling to 
85MB/s. (Filling up the network bandwidth)

I think their database design and infrastructure isn't optimized for their 
specific work-load. Which is, unfortunately, quite common.

> Obviously they'd do that in a
> complete disaster, but short of that they can't really afford to do
> so.  By the time a backup is recorded it would be incredibly out of
> date.  They have the same issue with the lack of online upgrades
> (which the responding article doesn't really talk about).  They really
> need it to just work all the time.

When I migrate a Postgresql to a new major version, I migrate 1 database at a 
time to minimize downtime. This is done by piping the output of the backup-
process straight into a restore-proces connected to the new server.

If it were even more time-critical, I would develop a migration proces that 
would:
1) copy all the current (as in, needed today) to the new database
2) disable the application
3) copy all the latest changes for today to the new database
4) reenable the application (pointing to new database)
5) copy all the historical data I might need

I would add a note on the website and send out an email first informing the 
customers that the data is being migrated and historical data might be 
incomplete during this proces.

> >> I'd think that it is more likely
> >> that the likes of Oracle would (for their flagship DB (not for MySQL),
> > 
> > Never worked with Oracle (or other big software vendors), have you? :)
> 
> Actually, I almost exclusively work with them.  Some are better than
> others.  I don't work directly with Oracle, but I can say that the two
> times I've worked with an Oracle consultant they've been worth their
> weight in gold, and cost about as much.

They do have some good ones...

> The one was fixing some kind
> of RDB data corruption on a VAX that was easily a decade out of date
> at the time; I was shocked that they could find somebody who knew how
> to fix it.  interestingly, it looks like they only abandoned RDB
> recently.

Probably one of the few people in the world. And he/she might have been hired 
in by Oracle for this particular issue.

> They do tend to be a solution that involves throwing money at
> problems.  My employer was having issues with a database from another
> big software vendor which I'm sure was the result of bad application
> design, but throwing Exadata at it did solve the problem, at an
> astonishing price.

I was at Collaborate last year and spoke to some of the guys from Oracle. (Not 
going into specifics to protect their jobs). When asked if one of my customers 
should be using Oracle RAC or Exadata, the answer came down to: "If you think 
RAC might be sufficient, it usually is"

Exadata, however, is a really nice design. But throwing faster machines at a 
problem should only be part of 

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[J. Roeleveld]

On Monday, August 01, 2016 08:43:49 AM james wrote:
> On 08/01/2016 02:16 AM, J. Roeleveld wrote:
> > On Saturday, July 30, 2016 06:38:01 AM Rich Freeman wrote:
> >> On Sat, Jul 30, 2016 at 6:24 AM, Alan McKinnon 
> > 
> > wrote:
> >>> On 29/07/2016 22:58, Mick wrote:
>  Interesting article explaining why Uber are moving away from
>  PostgreSQL.
>  I am
>  running both DBs on different desktop PCs for akonadi and I'm also
>  running
>  MySQL on a number of websites.  Let's which one goes sideways first. 
>  :p
>  
>   https://eng.uber.com/mysql-migration/
> >>> 
> >>> I don't think your akonadi and some web sites compares in any way to
> >>> Uber
> >>> and what they do.
> >>> 
> >>> FWIW, my Dev colleagues support and entire large corporate ISP's
> >>> operational and customer data on PostgreSQL-9.3. With clustering. With
> >>> no
> >>> db-related issues :-)
> >> 
> >> Agree, you'd need to be fairly large-scale to have their issues,
> > 
> > And also have to design your database by people who think MySQL actually
> > follows common SQL standards.
> > 
> >> but I
> >> think the article was something anybody interested in databases should
> >> read.  If nothing else it is a really easy to follow explanation of
> >> the underlying architectures.
> > 
> > Check the link posted by Douglas.
> > Ubers article has some misunderstandings about the architecture with
> > conclusions drawn that are, at least also, caused by their database design
> > and usage.
> > 
> >> I'll probably post this to my LUG mailing list.  I think one of the
> >> Postgres devs lurks there so I'm curious to his impressions.
> >> 
> >> I was a bit surprised to hear about the data corruption bug.  I've
> >> always considered Postgres to have a better reputation for data
> >> integrity.
> > 
> > They do.
> > 
> >> And of course almost any FOSS project could have a bug.  I
> >> don't know if either project does the kind of regression testing to
> >> reliably detect this sort of issue.
> > 
> > Not sure either, I do think PostgreSQL does a lot with regression tests.
> > 
> >> I'd think that it is more likely
> >> that the likes of Oracle would (for their flagship DB (not for MySQL),
> > 
> > Never worked with Oracle (or other big software vendors), have you? :)
> > 
> >> and they'd probably be more likely to send out an engineer to beg
> >> forgiveness while they fix your database).
> > 
> > Only if you're a big (as in, spend a lot of money with them) customer.
> > 
> >> Of course, if you're Uber
> >> the hit you'd take from downtime/etc isn't made up for entirely by
> >> having somebody take a few days to get everything fixed.
> > 
> > --
> > Joost
> 
> I certainly respect your skills and posts on Databases, Joost, as
> everything you have posted, in the past is 'spot on'.

Comes with a keen interest and long-term (think decades) of working with 
different databases.

> Granted, I'm no database expert, far from it.

Not many people are, nor do they need to be.

> But I want to share a few thing with you,
> and hope you  (and others) will 'chime in' on these comments.
> 
> Way back, when the earth was cooling and we all had dinosaurs for pets,
> some of us hacked on AT "3B2" unix systems. They were know for their
> 'roll back and recovery', triplicated (or more) transaction processes
> and 'voters' system to ferret out if a transaction was complete and
> correct. There was no ACID, the current 'gold standard' if you believe
> what Douglas and other write about concerning databases.
> 
> In essence, (from crusted up memories) a basic (SS7) transaction related
> to the local telephone switch, was ran  on 3 machines. The results were
> compared. If they matched, the transaction went forward as valid. If 2/3
> matched,

And what in the likely case when only 1 was correct?
Have you seen the movie "minority report"?
If yes, think back to why Tom Cruise was found 'guilty' when he wasn't and how 
often this actually occured.

> and the switch was was configured, then the code would
> essentially 'vote' and majority ruled. This is what led to phone calls
> (switched phone calls) having variable delays, often in the order of
> seconds, mis-connections and other problems we all encountered during
> periods of excessive demand.

Not sure if that was the cause in the past, but these days it can also still 
take a few seconds before the other end rings. This is due to the phone-system 
(all PBXs in the path) needing to setup the routing between both end-points 
prior to the ring-tone actually starting.
When the system is busy, these lookups will take time and can even time-out. 
(Try wishing everyone you know a happy new year using a wired phone and you'll 
see what I mean. Mobile phones have a seperate problem at that time)

> That scenario was at the heart of how old, crappy AT unix (SVR?) could
> perform so well and therefore established the gold standard for RT
> transaction processing, aka the 

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Rich Freeman]

On Mon, Aug 1, 2016 at 3:16 AM, J. Roeleveld  wrote:
>
> Check the link posted by Douglas.
> Ubers article has some misunderstandings about the architecture with
> conclusions drawn that are, at least also, caused by their database design and
> usage.

I've read it.  I don't think it actually alleges any misunderstandings
about the Postgres architecture, but rather that it doesn't perform as
well in Uber's design.  I don't think it actually alleges that Uber's
design is a bad one in any way.

But, I'm certainly interested in anything else that develops here...

>
>> And of course almost any FOSS project could have a bug.  I
>> don't know if either project does the kind of regression testing to
>> reliably detect this sort of issue.
>
> Not sure either, I do think PostgreSQL does a lot with regression tests.
>

Obviously they missed that bug.  Of course, so did Uber in their
internal testing.  I've seen a DB bug in production (granted, only one
so far) and they aren't pretty.  A big issue for Uber is that their
transaction rate and DB size is such that they really don't have a
practical option of restoring backups.  Obviously they'd do that in a
complete disaster, but short of that they can't really afford to do
so.  By the time a backup is recorded it would be incredibly out of
date.  They have the same issue with the lack of online upgrades
(which the responding article doesn't really talk about).  They really
need it to just work all the time.

>> I'd think that it is more likely
>> that the likes of Oracle would (for their flagship DB (not for MySQL),
>
> Never worked with Oracle (or other big software vendors), have you? :)

Actually, I almost exclusively work with them.  Some are better than
others.  I don't work directly with Oracle, but I can say that the two
times I've worked with an Oracle consultant they've been worth their
weight in gold, and cost about as much.  The one was fixing some kind
of RDB data corruption on a VAX that was easily a decade out of date
at the time; I was shocked that they could find somebody who knew how
to fix it.  interestingly, it looks like they only abandoned RDB
recently.

They do tend to be a solution that involves throwing money at
problems.  My employer was having issues with a database from another
big software vendor which I'm sure was the result of bad application
design, but throwing Exadata at it did solve the problem, at an
astonishing price.  Neither my employer nor the big software provider
in question is likely to attract top-notch DB talent (indeed, mine has
steadily gotten rid of anybody who knows how to do anything in Oracle
beyond creating schemas it seems, though I can only imagine how much
they pay annually in their license fees; and yes, I'm sure 99.9% of
what they use Oracle (or SQL Server) for would work just fine in
Postgres).

>
> Only if you're a big (as in, spend a lot of money with them) customer.
>

So, we are that (and I think a few of our IT execs used to be Oracle
employees, which I'm sure isn't hurting their business).  I'll admit
that Uber might not get the same attention.  Seems like Oracle is the
solution at work from everything to software that runs the entire
company to software that hosts one table for 10 employees (well, when
somebody notices and gets it out of Access).  Well, unless it involves
an MS-oriented dev or Sharepoint, in which case somebody inevitably
wants it on SQL Server.  I did mention that we're not a world-class IT
shop, didn't I?

-- 
Rich

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[james]

On 08/01/2016 02:16 AM, J. Roeleveld wrote:

On Saturday, July 30, 2016 06:38:01 AM Rich Freeman wrote:

On Sat, Jul 30, 2016 at 6:24 AM, Alan McKinnon 

wrote:

On 29/07/2016 22:58, Mick wrote:

Interesting article explaining why Uber are moving away from PostgreSQL.
I am
running both DBs on different desktop PCs for akonadi and I'm also
running
MySQL on a number of websites.  Let's which one goes sideways first.  :p

 https://eng.uber.com/mysql-migration/


I don't think your akonadi and some web sites compares in any way to Uber
and what they do.

FWIW, my Dev colleagues support and entire large corporate ISP's
operational and customer data on PostgreSQL-9.3. With clustering. With no
db-related issues :-)


Agree, you'd need to be fairly large-scale to have their issues,


And also have to design your database by people who think MySQL actually
follows common SQL standards.


but I
think the article was something anybody interested in databases should
read.  If nothing else it is a really easy to follow explanation of
the underlying architectures.


Check the link posted by Douglas.
Ubers article has some misunderstandings about the architecture with
conclusions drawn that are, at least also, caused by their database design and
usage.


I'll probably post this to my LUG mailing list.  I think one of the
Postgres devs lurks there so I'm curious to his impressions.

I was a bit surprised to hear about the data corruption bug.  I've
always considered Postgres to have a better reputation for data
integrity.


They do.


And of course almost any FOSS project could have a bug.  I
don't know if either project does the kind of regression testing to
reliably detect this sort of issue.


Not sure either, I do think PostgreSQL does a lot with regression tests.


I'd think that it is more likely
that the likes of Oracle would (for their flagship DB (not for MySQL),


Never worked with Oracle (or other big software vendors), have you? :)


and they'd probably be more likely to send out an engineer to beg
forgiveness while they fix your database).


Only if you're a big (as in, spend a lot of money with them) customer.


Of course, if you're Uber
the hit you'd take from downtime/etc isn't made up for entirely by
having somebody take a few days to get everything fixed.


--
Joost




I certainly respect your skills and posts on Databases, Joost, as 
everything you have posted, in the past is 'spot on'. Granted, I'm no 
database expert, far from it. But I want to share a few thing with you, 
and hope you  (and others) will 'chime in' on these comments.


Way back, when the earth was cooling and we all had dinosaurs for pets,
some of us hacked on AT "3B2" unix systems. They were know for their
'roll back and recovery', triplicated (or more) transaction processes 
and 'voters' system to ferret out if a transaction was complete and 
correct. There was no ACID, the current 'gold standard' if you believe 
what Douglas and other write about concerning databases.


In essence, (from crusted up memories) a basic (SS7) transaction related 
to the local telephone switch, was ran  on 3 machines. The results were 
compared. If they matched, the transaction went forward as valid. If 2/3 
matched, and the switch was was configured, then the code would 
essentially 'vote' and majority ruled. This is what led to phone calls 
(switched phone calls) having variable delays, often in the order of 
seconds, mis-connections and other problems we all encountered during 
periods of excessive demand.


That scenario was at the heart of how old, crappy AT unix (SVR?) could 
perform so well and therefore established the gold standard for RT 
transaction processing, aka the "five  9s" 99.999% of up-time (about 5 
minutes per year of downtime). Sure this part is only related to 
transaction processing as there was much more to the "five 9s" legacy, 
but imho, that is the heart of what was the precursor to ACID property's 
now so greatly espoused in SQL codes that Douglas refers to.


Do folks concur or disagree at this point?


The reason this is important to me (and others?), is that, if this idea 
(granted there is much more detail to it) is still valid, then it can 
form  the basis for building up superior-ACID processes, that meet or 
exceed, the properties of an expensive (think Oracle) transaction 
process on distributed (parallel) or clustered systems, to a degree of 
accuracy only limited by the limit of the number of odd numbered voter 
codes involve in the distributed and replicated parts of the 
transaction. I even added some code where replicated routines were 
written in different languages, and the results compared to add an 
additional layer of verification before the voter step. (gotta love 
assembler?).


I guess my point is 'Douglas' is full of stuffing, OR that is what folks 
are doing when they 'role their own solution specifically customized to 
their specific needs' as he alludes to near the end of 

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[J. Roeleveld]

On Saturday, July 30, 2016 06:38:01 AM Rich Freeman wrote:
> On Sat, Jul 30, 2016 at 6:24 AM, Alan McKinnon  
wrote:
> > On 29/07/2016 22:58, Mick wrote:
> >> Interesting article explaining why Uber are moving away from PostgreSQL.
> >> I am
> >> running both DBs on different desktop PCs for akonadi and I'm also
> >> running
> >> MySQL on a number of websites.  Let's which one goes sideways first.  :p
> >> 
> >>  https://eng.uber.com/mysql-migration/
> > 
> > I don't think your akonadi and some web sites compares in any way to Uber
> > and what they do.
> > 
> > FWIW, my Dev colleagues support and entire large corporate ISP's
> > operational and customer data on PostgreSQL-9.3. With clustering. With no
> > db-related issues :-)
> 
> Agree, you'd need to be fairly large-scale to have their issues,

And also have to design your database by people who think MySQL actually 
follows common SQL standards.

> but I
> think the article was something anybody interested in databases should
> read.  If nothing else it is a really easy to follow explanation of
> the underlying architectures.

Check the link posted by Douglas.
Ubers article has some misunderstandings about the architecture with 
conclusions drawn that are, at least also, caused by their database design and 
usage.

> I'll probably post this to my LUG mailing list.  I think one of the
> Postgres devs lurks there so I'm curious to his impressions.
> 
> I was a bit surprised to hear about the data corruption bug.  I've
> always considered Postgres to have a better reputation for data
> integrity.

They do.

> And of course almost any FOSS project could have a bug.  I
> don't know if either project does the kind of regression testing to
> reliably detect this sort of issue.

Not sure either, I do think PostgreSQL does a lot with regression tests.

> I'd think that it is more likely
> that the likes of Oracle would (for their flagship DB (not for MySQL),

Never worked with Oracle (or other big software vendors), have you? :)

> and they'd probably be more likely to send out an engineer to beg
> forgiveness while they fix your database).

Only if you're a big (as in, spend a lot of money with them) customer.

> Of course, if you're Uber
> the hit you'd take from downtime/etc isn't made up for entirely by
> having somebody take a few days to get everything fixed.

--
Joost

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Douglas J Hunley]

On Fri, Jul 29, 2016 at 7:01 PM, Mick  wrote:

> Yes, same here, I would be interested to hear what the Postgres dev says,
> should he respond to it.
>

One PostgreSQL dev's response - https://t.co/LfPlIPWulc


-- 
{
  "name": "douglas j hunley",
  "email": "doug.hun...@gmail.com",
  "social": [
{
"blog": "https://hunleyd.github.io/;,
"twitter": "@hunleyd"
}
]
}

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Mick]

On Saturday 30 Jul 2016 06:38:01 Rich Freeman wrote:
> On Sat, Jul 30, 2016 at 6:24 AM, Alan McKinnon  
wrote:
> > On 29/07/2016 22:58, Mick wrote:
> >> Interesting article explaining why Uber are moving away from PostgreSQL.
> >> I am
> >> running both DBs on different desktop PCs for akonadi and I'm also
> >> running
> >> MySQL on a number of websites.  Let's which one goes sideways first.  :p
> >> 
> >>  https://eng.uber.com/mysql-migration/
> > 
> > I don't think your akonadi and some web sites compares in any way to Uber
> > and what they do.
> > 
> > FWIW, my Dev colleagues support and entire large corporate ISP's
> > operational and customer data on PostgreSQL-9.3. With clustering. With no
> > db-related issues :-)
> 
> Agree, you'd need to be fairly large-scale to have their issues, but I
> think the article was something anybody interested in databases should
> read.  If nothing else it is a really easy to follow explanation of
> the underlying architectures.
> 
> I'll probably post this to my LUG mailing list.  I think one of the
> Postgres devs lurks there so I'm curious to his impressions.
> 
> I was a bit surprised to hear about the data corruption bug.  I've
> always considered Postgres to have a better reputation for data
> integrity.  

Yes, same here, I would be interested to hear what the Postgres dev says, 
should he respond to it.

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Rich Freeman]

On Sat, Jul 30, 2016 at 6:24 AM, Alan McKinnon  wrote:
> On 29/07/2016 22:58, Mick wrote:
>>
>> Interesting article explaining why Uber are moving away from PostgreSQL.
>> I am
>> running both DBs on different desktop PCs for akonadi and I'm also running
>> MySQL on a number of websites.  Let's which one goes sideways first.  :p
>>
>>  https://eng.uber.com/mysql-migration/
>>
>
>
> I don't think your akonadi and some web sites compares in any way to Uber
> and what they do.
>
> FWIW, my Dev colleagues support and entire large corporate ISP's operational
> and customer data on PostgreSQL-9.3. With clustering. With no db-related
> issues :-)
>

Agree, you'd need to be fairly large-scale to have their issues, but I
think the article was something anybody interested in databases should
read.  If nothing else it is a really easy to follow explanation of
the underlying architectures.

I'll probably post this to my LUG mailing list.  I think one of the
Postgres devs lurks there so I'm curious to his impressions.

I was a bit surprised to hear about the data corruption bug.  I've
always considered Postgres to have a better reputation for data
integrity.  And of course almost any FOSS project could have a bug.  I
don't know if either project does the kind of regression testing to
reliably detect this sort of issue.  I'd think that it is more likely
that the likes of Oracle would (for their flagship DB (not for MySQL),
and they'd probably be more likely to send out an engineer to beg
forgiveness while they fix your database).  Of course, if you're Uber
the hit you'd take from downtime/etc isn't made up for entirely by
having somebody take a few days to get everything fixed.

-- 
Rich

Re: [gentoo-user] PostgreSQL Vs MySQL @Uber

[Alan McKinnon]

On 29/07/2016 22:58, Mick wrote:

Interesting article explaining why Uber are moving away from PostgreSQL.  I am
running both DBs on different desktop PCs for akonadi and I'm also running
MySQL on a number of websites.  Let's which one goes sideways first.  :p

 https://eng.uber.com/mysql-migration/




I don't think your akonadi and some web sites compares in any way to 
Uber and what they do.


FWIW, my Dev colleagues support and entire large corporate ISP's 
operational and customer data on PostgreSQL-9.3. With clustering. With 
no db-related issues :-)

[gentoo-user] PostgreSQL Vs MySQL @Uber

[Mick]

Interesting article explaining why Uber are moving away from PostgreSQL.  I am 
running both DBs on different desktop PCs for akonadi and I'm also running 
MySQL on a number of websites.  Let's which one goes sideways first.  :p

 https://eng.uber.com/mysql-migration/

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[J. Roeleveld]

Longer answer (as promised)

On Saturday, May 21, 2016 04:56:18 PM waltd...@waltdnes.org wrote:
> On Sat, May 21, 2016 at 08:55:39AM +0200, J. Roeleveld wrote

> > Did you run
> > emerge --config dev-db/postgresql:9.5
> 
>   Yes.
> 
> > succesfully?
> 
>   Obviously not.

:)

>   I think I've finally figured it out.  I edited
> /etc/conf.d/postgresql-9.5 to PG_INITDB_OPTS="--encoding=UTF8" and ran
> "emerge --config dev-db/postgresql:9.5".  I got an error message about
> the data directory not be empty (probably from my first attempt).  I ran
> 
> rm /var/lib/postgresql/9.5/data/*
> emerge --config dev-db/postgresql:9.5
> 
> and got a bit further.  I did get error messages as follows
> 
> 
> ###
> The database cluster will be initialized with locale "en_US.iso88591".
> initdb: encoding mismatch
> The encoding you selected (UTF8) and the encoding that the
> selected locale uses (LATIN1) do not match.  This would lead to
> misbehavior in various character string processing functions.
> Rerun initdb and either do not specify an encoding explicitly,
> or choose a matching combination.
> mv: cannot stat '/var/lib/postgresql/9.5/data/pg_hba.conf': No such file or
> directory mv: cannot stat '/var/lib/postgresql/9.5/data/pg_ident.conf': No
> such file or directory mv: cannot stat
> '/var/lib/postgresql/9.5/data/postgresql.conf': No such file or directory
> ###

If there is a mismatch, the config-script refuses to work correctly.

>   I fixed that.  In /etc/conf.d/postgresql-9.5 I set
> PG_INITDB_OPTS="--encoding=iso88591"  and ran
> 
> rm /var/lib/postgresql/9.5/data/*
> emerge --config dev-db/postgresql:9.5
> 
> and got the following.  Does it look OK?  Do I understand correctly...
> 
> config files are located in /etc/postgresql-9.5/
> the actual databases are located in /var/lib/postgresql/9.5/data
> 
> ###



> 
> WARNING: enabling "trust" authentication for local connections
> You can change this by editing pg_hba.conf or using the option -A, or
> --auth-local and --auth-host, the next time you run initdb.

This is nothing to worry about, as long as the database is not accessible from 
outside.
The "trust" part means it ignores passwords. By default, this is only for 
"localhost"

> Success. You can now start the database server using:
> 
> /usr/lib64/postgresql-9.5/bin/pg_ctl -D /var/lib/postgresql/9.5/data -l
> logfile start

Ignore this line.

>  * The autovacuum function, which was in contrib, has been moved to the main
> * PostgreSQL functions starting with 8.1, and starting with 8.4 is now
> enabled * by default. You can disable it in the cluster's:
>  * /etc/postgresql-9.5/postgresql.conf
>  *
>  * The PostgreSQL server, by default, will log events to:
>  * /var/lib/postgresql/9.5/data/postmaster.log
>  *
>  * You should use the '/etc/init.d/postgresql-9.5' script to run PostgreSQL
>  * instead of 'pg_ctl'.

Listen to this line :)
The init-script uses "pg_ctl" to start/stop postgresql.

> ###
> 
>   There's still one apparent internal contradiction in the output...
> 
> ###
> Success. You can now start the database server using:
> 
> /usr/lib64/postgresql-9.5/bin/pg_ctl -D /var/lib/postgresql/9.5/data -l
> logfile start
> ###
> 
> ...but it also says...
> 
> ###
>  * You should use the '/etc/init.d/postgresql-9.5' script to run PostgreSQL
>  * instead of 'pg_ctl'.
> ###

See above, use the init-script and you're fine.

If you change the configuration, you can tell Postgresql to reload the new 
config by issuing " /etc/init.d/postgresql-9.5 reload "

Most common config-changes to a running system: adding/removing users/access to 
/etc/postgresql-9.5/pg_hba.conf.

One additional piece of info, by default, it logs to
/var/lib/postgresql/9.5/data/postmaster.log

If you want it to log to a different location or to syslog, you can edit this 
in the file:
/etc/postgresql-9.5/postgresql.conf

Look for the section:
#--
# ERROR REPORTING AND LOGGING
#--

The comments in the file tell you which settings require a restart (of 
postgresql). The ones that don't should come into effect with just the 
"reload" command I mentioned above.

--
Joost

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[J. Roeleveld]

On Sunday, May 22, 2016 01:30:32 AM waltd...@waltdnes.org wrote:
> On Sun, May 22, 2016 at 04:41:44AM +, J. Roeleveld wrote
> 
> > Quick reply (longer one later)
> > 2nd run looks better.
> > 
> > The output is from the supplied scripts. For Gentoo, use the
> > /etc/init.d script. That will call the other one where necessary.
> > 
> > About the codepage. What does "eselect locale list" show?
> > In other words, which locale do you actually use?
> 
>   en_US.iso88591
> 
> [i3][waltdnes][~] eselect locale list
> Available targets for the LANG variable:
>   [1]   C
>   [2]   POSIX
>   [3]   en_US
>   [4]   en_US.iso88591 *
>   [5]   en_US.utf8
>   [ ]   (free form)
> 
>   I expect to be storing a lot of numeric data in postgresql, but not
> much text data, let alone, non-English text data.  I live in Toronto,
> Canada.  ISO8859-1 can handle ASCII (English), and accented Latin-1
> characters that exist in Canadian French.

In order for the configuration to actually be able to configure the database to 
handle UTF8, you need to run at least UTF8 or a superset.
I always run my servers in UTF8.

If you will not require non-ISO8859-1 characters, you should be ok with your 
current setup.

--
Joost

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[waltdnes]

On Sun, May 22, 2016 at 04:41:44AM +, J. Roeleveld wrote
> 
> Quick reply (longer one later)
> 2nd run looks better. 
> 
> The output is from the supplied scripts. For Gentoo, use the
> /etc/init.d script. That will call the other one where necessary.
> 
> About the codepage. What does "eselect locale list" show?
> In other words, which locale do you actually use?

  en_US.iso88591

[i3][waltdnes][~] eselect locale list
Available targets for the LANG variable:
  [1]   C
  [2]   POSIX
  [3]   en_US
  [4]   en_US.iso88591 *
  [5]   en_US.utf8
  [ ]   (free form)

  I expect to be storing a lot of numeric data in postgresql, but not
much text data, let alone, non-English text data.  I live in Toronto,
Canada.  ISO8859-1 can handle ASCII (English), and accented Latin-1
characters that exist in Canadian French.

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[J. Roeleveld]

On May 21, 2016 10:56:18 PM GMT+02:00, waltd...@waltdnes.org wrote:
>On Sat, May 21, 2016 at 08:55:39AM +0200, J. Roeleveld wrote
>> Longer answer:
>> 
>> On Friday, May 20, 2016 10:36:41 PM waltd...@waltdnes.org wrote:
>> >   Yes, I did RTFM at
>https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart
>> > and that's part of my problem.   I figured it would be a simple
>> > search and replace "9.3" ==> "9.5" in the wiki, but...
>> 
>> A quick scan should indicate that.
>> However:
>> PG_INITDB_OPTS="--locale=en_US.UTF-8 --lc-messages=sv_SE.UTF-8"
>> is wrong. See below.
>> 
>> > 1) The wiki recommends...
>> > PG_INITDB_OPTS="--locale=en_US.UTF-8"
>> 
>> Where did you configure this?
>> 
>> I did the following:
>> # cat /etc/conf.d/postgresql-9.5 | grep -i utf
>> PG_INITDB_OPTS="--encoding=UTF8"
>> 
>> 
>> Did you run 
>> emerge --config dev-db/postgresql:9.5
>
>  Yes.
>
>> succesfully?
>
>  Obviously not.  I think I've finally figured it out.  I edited
>/etc/conf.d/postgresql-9.5 to PG_INITDB_OPTS="--encoding=UTF8" and ran
>"emerge --config dev-db/postgresql:9.5".  I got an error message about
>the data directory not be empty (probably from my first attempt).  I
>ran
>
>rm /var/lib/postgresql/9.5/data/*
>emerge --config dev-db/postgresql:9.5
>
>and got a bit further.  I did get error messages as follows
>
>
>###
>The database cluster will be initialized with locale "en_US.iso88591".
>initdb: encoding mismatch
>The encoding you selected (UTF8) and the encoding that the
>selected locale uses (LATIN1) do not match.  This would lead to
>misbehavior in various character string processing functions.
>Rerun initdb and either do not specify an encoding explicitly,
>or choose a matching combination.
>mv: cannot stat '/var/lib/postgresql/9.5/data/pg_hba.conf': No such
>file or directory
>mv: cannot stat '/var/lib/postgresql/9.5/data/pg_ident.conf': No such
>file or directory
>mv: cannot stat '/var/lib/postgresql/9.5/data/postgresql.conf': No such
>file or directory
>###
>
>  I fixed that.  In /etc/conf.d/postgresql-9.5 I set
>PG_INITDB_OPTS="--encoding=iso88591"  and ran
>
>rm /var/lib/postgresql/9.5/data/*
>emerge --config dev-db/postgresql:9.5
>
>and got the following.  Does it look OK?  Do I understand correctly...
>
>config files are located in /etc/postgresql-9.5/
>the actual databases are located in /var/lib/postgresql/9.5/data
>
>###
>[i3][root][~] emerge --config dev-db/postgresql:9.5
>
>Configuring pkg...
>
> * You can modify the paths and options passed to initdb by editing:
> * /etc/conf.d/postgresql-9.5
> * 
> * Information on options that can be passed to initdb are found at:
> * http://www.postgresql.org/docs/9.5/static/creating-cluster.html
> * http://www.postgresql.org/docs/9.5/static/app-initdb.html
> * 
> * PG_INITDB_OPTS is currently set to:
> * --encoding=UTF8
> * 
> * Configuration files will be installed to:
> * /etc/postgresql-9.5/
> * 
> * The database cluster will be created in:
> * /var/lib/postgresql/9.5/data
> * 
> * Are you ready to continue? (y/n)
>y
> * Creating the data directory ...
> * Initializing the database ...
>The files belonging to this database system will be owned by user
>"postgres".
>This user must also own the server process.
>
>The database cluster will be initialized with locale "en_US.iso88591".
>The default text search configuration will be set to "english".
>
>Data page checksums are disabled.
>
>fixing permissions on existing directory /var/lib/postgresql/9.5/data
>... ok
>creating subdirectories ... ok
>selecting default max_connections ... 100
>selecting default shared_buffers ... 128MB
>selecting dynamic shared memory implementation ... posix
>creating configuration files ... ok
>creating template1 database in /var/lib/postgresql/9.5/data/base/1 ...
>ok
>initializing pg_authid ... ok
>initializing dependencies ... ok
>creating system views ... ok
>loading system objects' descriptions ... ok
>creating collations ... ok
>creating conversions ... ok
>creating dictionaries ... ok
>setting privileges on built-in objects ... ok
>creating information schema ... ok
>loading PL/pgSQL server-side language ... ok
>vacuuming database template1 ... ok
>copying template1 to template0 ... ok
>copying template1 to postgres ... ok
>syncing data to disk ... ok
>
>WARNING: enabling "trust" authentication for local connections
>You can change this by editing pg_hba.conf or using the option -A, or
>--auth-local and --auth-host, the next time you run initdb.
>
>Success. You can now start the database server using:
>
>/usr/lib64/postgresql-9.5/bin/pg_ctl -D /var/lib/postgresql/9.5/data -l
>logfile start
>
>* The autovacuum function, which was in contrib, has been moved to the
>main
>* PostgreSQL functions starting with 8.1, and starting with 8.4 is now
>enabled
> * 

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[waltdnes]

On Sat, May 21, 2016 at 08:55:39AM +0200, J. Roeleveld wrote
> Longer answer:
> 
> On Friday, May 20, 2016 10:36:41 PM waltd...@waltdnes.org wrote:
> >   Yes, I did RTFM at https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart
> > and that's part of my problem.   I figured it would be a simple
> > search and replace "9.3" ==> "9.5" in the wiki, but...
> 
> A quick scan should indicate that.
> However:
> PG_INITDB_OPTS="--locale=en_US.UTF-8 --lc-messages=sv_SE.UTF-8"
> is wrong. See below.
> 
> > 1) The wiki recommends...
> > PG_INITDB_OPTS="--locale=en_US.UTF-8"
> 
> Where did you configure this?
> 
> I did the following:
> # cat /etc/conf.d/postgresql-9.5 | grep -i utf
> PG_INITDB_OPTS="--encoding=UTF8"
> 
> 
> Did you run 
> emerge --config dev-db/postgresql:9.5

  Yes.

> succesfully?

  Obviously not.  I think I've finally figured it out.  I edited
/etc/conf.d/postgresql-9.5 to PG_INITDB_OPTS="--encoding=UTF8" and ran
"emerge --config dev-db/postgresql:9.5".  I got an error message about
the data directory not be empty (probably from my first attempt).  I ran

rm /var/lib/postgresql/9.5/data/*
emerge --config dev-db/postgresql:9.5

and got a bit further.  I did get error messages as follows


###
The database cluster will be initialized with locale "en_US.iso88591".
initdb: encoding mismatch
The encoding you selected (UTF8) and the encoding that the
selected locale uses (LATIN1) do not match.  This would lead to
misbehavior in various character string processing functions.
Rerun initdb and either do not specify an encoding explicitly,
or choose a matching combination.
mv: cannot stat '/var/lib/postgresql/9.5/data/pg_hba.conf': No such file or 
directory
mv: cannot stat '/var/lib/postgresql/9.5/data/pg_ident.conf': No such file or 
directory
mv: cannot stat '/var/lib/postgresql/9.5/data/postgresql.conf': No such file or 
directory
###

  I fixed that.  In /etc/conf.d/postgresql-9.5 I set
PG_INITDB_OPTS="--encoding=iso88591"  and ran

rm /var/lib/postgresql/9.5/data/*
emerge --config dev-db/postgresql:9.5

and got the following.  Does it look OK?  Do I understand correctly...

config files are located in /etc/postgresql-9.5/
the actual databases are located in /var/lib/postgresql/9.5/data

###
[i3][root][~] emerge --config dev-db/postgresql:9.5

Configuring pkg...

 * You can modify the paths and options passed to initdb by editing:
 * /etc/conf.d/postgresql-9.5
 * 
 * Information on options that can be passed to initdb are found at:
 * http://www.postgresql.org/docs/9.5/static/creating-cluster.html
 * http://www.postgresql.org/docs/9.5/static/app-initdb.html
 * 
 * PG_INITDB_OPTS is currently set to:
 * --encoding=UTF8
 * 
 * Configuration files will be installed to:
 * /etc/postgresql-9.5/
 * 
 * The database cluster will be created in:
 * /var/lib/postgresql/9.5/data
 * 
 * Are you ready to continue? (y/n)
y
 * Creating the data directory ...
 * Initializing the database ...
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.iso88591".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/9.5/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /var/lib/postgresql/9.5/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

/usr/lib64/postgresql-9.5/bin/pg_ctl -D /var/lib/postgresql/9.5/data -l 
logfile start

 * The autovacuum function, which was in contrib, has been moved to the main
 * PostgreSQL functions starting with 8.1, and starting with 8.4 is now enabled
 * by default. You can disable it in the cluster's:
 * /etc/postgresql-9.5/postgresql.conf
 * 
 * The PostgreSQL server, by default, will log events to:
 * 

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[Alec Ten Harmsel]

On 2016-05-21 07:32, J. Roeleveld wrote:

On Saturday, May 21, 2016 06:51:46 AM Alec Ten Harmsel wrote:


`equery use gnumeric' gives the `libgda' flag, which should pull in
database support. I've never used it, so I don't know whether or not it
works/how well it works. What is in this spreadsheet? If it is financial
stuff, you can use Gnucash, which supports using a database as a backend.

Does this finally work?
Last time I tried this, half the functionality didn't work at all and the
other half was buggy. (This was years ago)


I have no idea, but I'm going to test in a VM because I'm a little 
curious now.



My main problem is that columns of several thousand rows are functions

based on other columns of several thousand rows.  For the time-being,
I've split up the spreadsheet into a few pieces, but a database is the
best solution.  If I could run the calculations in the database, and
pull in the final results as static numbers for graphing, that would
greatly reduce the strain on the spreadsheet.  Or is it possible to
graph directly from postgresql?

Here are my recommendations, in order of "least code" to "most code" (I
don't think postgresql supports graphing):

1. Write some sql scripts that compute the data you need and output CSV,
then import to Gnumeric and do the plots.

For script examples:
http://stackoverflow.com/questions/1517635/save-pl-pgsql-output-from-postgresql-to-a-csv-file


2. Write python script(s) that run SQL commands and plot the data with
matplotlib.
3. Write a webapp so you don't have to run scripts by hand - the plots
are generated by opening a web page.

4. Write it all in C++ :)


Qt and QCustomPlot are nice, but I'm not sure I have quite that much time.


Depending on how much automation you want vs. how much time you want to
spend writing/debugging code, hopefully one of those helps. I help
researchers use a HPC cluster; some are very savvy programmers, some are
not. For working on "big data" projects, some will throw raw data into a
Hadoop cluster and happily do all their work using Hadoop, while some
will put in raw data, clean it up, and then pull it out and use MATLAB,
stata, R, etc., so you just need to find the workflow that works best
for you. I personally would choose option 3, as it involves the least
amount of running scripts over and over, but to each his own.

I have actual free time now (done with school, finally), so I might be
able to help prototype if you would like as well.

Something I could use (and others):
A simple PHP page which I can feed:
- connection parameters to a database
- select-query
- which result-field to use for the horizontal axis
and then plots the remaining fields for the vertical axis.

I haven't checked with google yet, so if there is a decent example, I'd be
interested :)

--
Joost



Google gave me nothing. I have not written PHP for a long time, and I'm 
so unfamiliar with deploying/running PHP that it would take me a long 
time to write this.


Another option is R - I just did some searching, and it supports pulling 
data from a database. R's basic plotting functions are real nice. I 
imagine a script could do some basic queries + plotting in 20-30 lines.


Alec

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[J. Roeleveld]

On Saturday, May 21, 2016 06:51:46 AM Alec Ten Harmsel wrote:
> Joost knows far more about databases than I do, so I mostly commented on
> the workflow part.
>
> On 2016-05-20 22:36, waltd...@waltdnes.org wrote:



> I have never run postgresql on gentoo (hopefully soon :D), but on
> Debian-derived distros and RPM-based distros, PGDATA is always somewhere
> in /var. /etc seems wrong.

There are symlinks from the /var location to /etc for the configuration files.
The data itself, eg. PGDATA, sits, by default, in /var/.


> `equery use gnumeric' gives the `libgda' flag, which should pull in
> database support. I've never used it, so I don't know whether or not it
> works/how well it works. What is in this spreadsheet? If it is financial
> stuff, you can use Gnucash, which supports using a database as a backend.

Does this finally work?
Last time I tried this, half the functionality didn't work at all and the 
other half was buggy. (This was years ago)

> >My main problem is that columns of several thousand rows are functions
> > 
> > based on other columns of several thousand rows.  For the time-being,
> > I've split up the spreadsheet into a few pieces, but a database is the
> > best solution.  If I could run the calculations in the database, and
> > pull in the final results as static numbers for graphing, that would
> > greatly reduce the strain on the spreadsheet.  Or is it possible to
> > graph directly from postgresql?
> 
> Here are my recommendations, in order of "least code" to "most code" (I
> don't think postgresql supports graphing):
> 
> 1. Write some sql scripts that compute the data you need and output CSV,
> then import to Gnumeric and do the plots.

For script examples:
http://stackoverflow.com/questions/1517635/save-pl-pgsql-output-from-postgresql-to-a-csv-file

> 2. Write python script(s) that run SQL commands and plot the data with
> matplotlib.
> 3. Write a webapp so you don't have to run scripts by hand - the plots
> are generated by opening a web page.
4. Write it all in C++ :)

> Depending on how much automation you want vs. how much time you want to
> spend writing/debugging code, hopefully one of those helps. I help
> researchers use a HPC cluster; some are very savvy programmers, some are
> not. For working on "big data" projects, some will throw raw data into a
> Hadoop cluster and happily do all their work using Hadoop, while some
> will put in raw data, clean it up, and then pull it out and use MATLAB,
> stata, R, etc., so you just need to find the workflow that works best
> for you. I personally would choose option 3, as it involves the least
> amount of running scripts over and over, but to each his own.
> 
> I have actual free time now (done with school, finally), so I might be
> able to help prototype if you would like as well.

Something I could use (and others):
A simple PHP page which I can feed:
- connection parameters to a database
- select-query
- which result-field to use for the horizontal axis
and then plots the remaining fields for the vertical axis.

I haven't checked with google yet, so if there is a decent example, I'd be 
interested :)

--
Joost

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[Alec Ten Harmsel]

Joost knows far more about databases than I do, so I mostly commented on 
the workflow part.


On 2016-05-20 22:36, waltd...@waltdnes.org wrote:

   Yes, I did RTFM at https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart
and that's part of my problem.   I figured it would be a simple
search and replace "9.3" ==> "9.5" in the wiki, but...

1) The wiki recommends...
PG_INITDB_OPTS="--locale=en_US.UTF-8"

...but I get...


The database cluster will be initialized with locale "en_US.iso88591".
initdb: "en_US.UTF8" is not a valid server encoding name

"locale -a" returns...
C
POSIX
en_US
en_US.iso88591
en_US.utf8

2) The wiki says...

This time the focus is upon the files in the PGDATA directory,
/etc/postgresql-9.3 , instead with primary focus on the
postgresql.conf and pg_hba.conf files.

"ls /etc/postgresql-9.5/" returns...
postgresql.conf  psqlrc

but postgresql seems to want them in /var/lib instead...


mv: cannot stat '/var/lib/postgresql/9.5/data/pg_hba.conf': No such
file or directory
mv: cannot stat '/var/lib/postgresql/9.5/data/pg_ident.conf': No
such file or directory
mv: cannot stat '/var/lib/postgresql/9.5/data/postgresql.conf':
No such file or directory

   Can somebody please confirm the correct way to go?


I have never run postgresql on gentoo (hopefully soon :D), but on 
Debian-derived distros and RPM-based distros, PGDATA is always somewhere 
in /var. /etc seems wrong.




   Why I want postgresql... I've been keeping a bunch of data in a
spreadsheet, and it's gotten too large.  The spreadsheet locks up my
system when I try to update it.  I've used "top" and watched as
gnumeric's memory consumption grows to eat all available ram.  It locks
up the system so I can't even ssh in.  This is on an X86_64 with 8 gigs
of RAM!  Fortunately, "magic-sysrq" allows a relatively clean shutdown.
While we're at it, is there a way for gnumeric to pull in data directly
from postgresql?  ODBC?  I'm aware of copying from postgresql to a CSV
file and importing that, but it's rather clunky.


`equery use gnumeric' gives the `libgda' flag, which should pull in 
database support. I've never used it, so I don't know whether or not it 
works/how well it works. What is in this spreadsheet? If it is financial 
stuff, you can use Gnucash, which supports using a database as a backend.




   My main problem is that columns of several thousand rows are functions
based on other columns of several thousand rows.  For the time-being,
I've split up the spreadsheet into a few pieces, but a database is the
best solution.  If I could run the calculations in the database, and
pull in the final results as static numbers for graphing, that would
greatly reduce the strain on the spreadsheet.  Or is it possible to
graph directly from postgresql?


Here are my recommendations, in order of "least code" to "most code" (I 
don't think postgresql supports graphing):


1. Write some sql scripts that compute the data you need and output CSV, 
then import to Gnumeric and do the plots.
2. Write python script(s) that run SQL commands and plot the data with 
matplotlib.
3. Write a webapp so you don't have to run scripts by hand - the plots 
are generated by opening a web page.


Depending on how much automation you want vs. how much time you want to 
spend writing/debugging code, hopefully one of those helps. I help 
researchers use a HPC cluster; some are very savvy programmers, some are 
not. For working on "big data" projects, some will throw raw data into a 
Hadoop cluster and happily do all their work using Hadoop, while some 
will put in raw data, clean it up, and then pull it out and use MATLAB, 
stata, R, etc., so you just need to find the workflow that works best 
for you. I personally would choose option 3, as it involves the least 
amount of running scripts over and over, but to each his own.


I have actual free time now (done with school, finally), so I might be 
able to help prototype if you would like as well.


Alec

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[J. Roeleveld]

Longer answer:

On Friday, May 20, 2016 10:36:41 PM waltd...@waltdnes.org wrote:
>   Yes, I did RTFM at https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart
> and that's part of my problem.   I figured it would be a simple
> search and replace "9.3" ==> "9.5" in the wiki, but...

A quick scan should indicate that.
However:
PG_INITDB_OPTS="--locale=en_US.UTF-8 --lc-messages=sv_SE.UTF-8"
is wrong. See below.

> 1) The wiki recommends...
> PG_INITDB_OPTS="--locale=en_US.UTF-8"

Where did you configure this?

I did the following:
# cat /etc/conf.d/postgresql-9.5 | grep -i utf
PG_INITDB_OPTS="--encoding=UTF8"


> ...but I get...
> 
> > The database cluster will be initialized with locale "en_US.iso88591".
> > initdb: "en_US.UTF8" is not a valid server encoding name
> 
> "locale -a" returns...
> C
> POSIX
> en_US
> en_US.iso88591
> en_US.utf8

Postgresql only uses the codepage, not the localisation ("en_US") part.

> 2) The wiki says...
> 
> > This time the focus is upon the files in the PGDATA directory,
> > /etc/postgresql-9.3 , instead with primary focus on the
> > postgresql.conf and pg_hba.conf files.
> 
> "ls /etc/postgresql-9.5/" returns...
> postgresql.conf  psqlrc
> 
> but postgresql seems to want them in /var/lib instead...
> 
> > mv: cannot stat '/var/lib/postgresql/9.5/data/pg_hba.conf': No such
> > file or directory
> > mv: cannot stat '/var/lib/postgresql/9.5/data/pg_ident.conf': No
> > such file or directory
> > mv: cannot stat '/var/lib/postgresql/9.5/data/postgresql.conf':
> > No such file or directory
> 
>   Can somebody please confirm the correct way to go?

Did you run 
emerge --config dev-db/postgresql:9.5
succesfully?

>   Why I want postgresql... I've been keeping a bunch of data in a
> spreadsheet, and it's gotten too large.  The spreadsheet locks up my
> system when I try to update it.  I've used "top" and watched as
> gnumeric's memory consumption grows to eat all available ram.  It locks
> up the system so I can't even ssh in.  This is on an X86_64 with 8 gigs
> of RAM!  Fortunately, "magic-sysrq" allows a relatively clean shutdown.
> While we're at it, is there a way for gnumeric to pull in data directly
> from postgresql?  ODBC?  I'm aware of copying from postgresql to a CSV
> file and importing that, but it's rather clunky.

There are ODBC and native drivers. You need to check which have support 
directly. Look for "postgres" USE-flags in spreadsheet applications.

>   My main problem is that columns of several thousand rows are functions
> based on other columns of several thousand rows.  For the time-being,
> I've split up the spreadsheet into a few pieces, but a database is the
> best solution.  If I could run the calculations in the database, and
> pull in the final results as static numbers for graphing, that would
> greatly reduce the strain on the spreadsheet.  Or is it possible to
> graph directly from postgresql?

Not to my knowledge, I tend to use spreadsheets or graphics libraries in C++ 
GUI applications. (Still playing with the latter, so not the best resource for 
that)

>   I used to work with Oracle and PL/SQL before I retired, so I think I
> know what I'm getting into as far as the database stuff is concerned.
> Once I get past the Gentoo-specific install problems, I'll subscribe to
> a postgresql mailing list, and ask postgresql-specific questions there.

Postgresql has it's own procedural language, might be nice to look into that 
in that case.

I would suggest the USER-mailing list.
The development one (HACKERS) deals with the actual internals, not something 
most users would be interested in.

--
Joost

Re: [gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[J. Roeleveld]

On May 21, 2016 4:36:41 AM GMT+02:00, waltd...@waltdnes.org wrote:
>  Yes, I did RTFM at https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart
>and that's part of my problem.   I figured it would be a simple
>search and replace "9.3" ==> "9.5" in the wiki, but...
>
>1) The wiki recommends...
>PG_INITDB_OPTS="--locale=en_US.UTF-8"
>
>...but I get...
>
>> The database cluster will be initialized with locale
>"en_US.iso88591".
>> initdb: "en_US.UTF8" is not a valid server encoding name
>
>"locale -a" returns...
>C
>POSIX
>en_US
>en_US.iso88591
>en_US.utf8
>
>2) The wiki says...
>> This time the focus is upon the files in the PGDATA directory,
>> /etc/postgresql-9.3 , instead with primary focus on the
>> postgresql.conf and pg_hba.conf files.
>
>"ls /etc/postgresql-9.5/" returns...
>postgresql.conf  psqlrc
>
>but postgresql seems to want them in /var/lib instead...
>
>> mv: cannot stat '/var/lib/postgresql/9.5/data/pg_hba.conf': No such
>> file or directory
>> mv: cannot stat '/var/lib/postgresql/9.5/data/pg_ident.conf': No
>> such file or directory
>> mv: cannot stat '/var/lib/postgresql/9.5/data/postgresql.conf':
>> No such file or directory
>
>  Can somebody please confirm the correct way to go?
>
>  Why I want postgresql... I've been keeping a bunch of data in a
>spreadsheet, and it's gotten too large.  The spreadsheet locks up my
>system when I try to update it.  I've used "top" and watched as
>gnumeric's memory consumption grows to eat all available ram.  It locks
>up the system so I can't even ssh in.  This is on an X86_64 with 8 gigs
>of RAM!  Fortunately, "magic-sysrq" allows a relatively clean shutdown.
>While we're at it, is there a way for gnumeric to pull in data directly
>from postgresql?  ODBC?  I'm aware of copying from postgresql to a CSV
>file and importing that, but it's rather clunky.
>
> My main problem is that columns of several thousand rows are functions
>based on other columns of several thousand rows.  For the time-being,
>I've split up the spreadsheet into a few pieces, but a database is the
>best solution.  If I could run the calculations in the database, and
>pull in the final results as static numbers for graphing, that would
>greatly reduce the strain on the spreadsheet.  Or is it possible to
>graph directly from postgresql?
>
>  I used to work with Oracle and PL/SQL before I retired, so I think I
>know what I'm getting into as far as the database stuff is concerned.
>Once I get past the Gentoo-specific install problems, I'll subscribe to
>a postgresql mailing list, and ask postgresql-specific questions there.

Quick response. Longer one later when I have access to the database server.

I don't use the wiki, instead I follow the post-emerge messages.

UTF8 is possible. I only use that.
Not sure about gnumeric, bit libreoffice can read directly from postgresql.
ODBC is not as simple on Linux as it is on Windows. But when configured, it 
should work as well. 
Exporting data to CSV is mentioned in several howtos and in the manpage of psql 
(commandline tool)

--
Joost 
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[gentoo-user] postgresql 9.5.2 versus Gentoo wiki install instructions?

[waltdnes]

  Yes, I did RTFM at https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart
and that's part of my problem.   I figured it would be a simple
search and replace "9.3" ==> "9.5" in the wiki, but...

1) The wiki recommends...
PG_INITDB_OPTS="--locale=en_US.UTF-8"

...but I get...

> The database cluster will be initialized with locale "en_US.iso88591".
> initdb: "en_US.UTF8" is not a valid server encoding name

"locale -a" returns...
C
POSIX
en_US
en_US.iso88591
en_US.utf8

2) The wiki says...
> This time the focus is upon the files in the PGDATA directory,
> /etc/postgresql-9.3 , instead with primary focus on the
> postgresql.conf and pg_hba.conf files.

"ls /etc/postgresql-9.5/" returns...
postgresql.conf  psqlrc

but postgresql seems to want them in /var/lib instead...

> mv: cannot stat '/var/lib/postgresql/9.5/data/pg_hba.conf': No such
> file or directory
> mv: cannot stat '/var/lib/postgresql/9.5/data/pg_ident.conf': No
> such file or directory
> mv: cannot stat '/var/lib/postgresql/9.5/data/postgresql.conf':
> No such file or directory

  Can somebody please confirm the correct way to go?

  Why I want postgresql... I've been keeping a bunch of data in a
spreadsheet, and it's gotten too large.  The spreadsheet locks up my
system when I try to update it.  I've used "top" and watched as
gnumeric's memory consumption grows to eat all available ram.  It locks
up the system so I can't even ssh in.  This is on an X86_64 with 8 gigs
of RAM!  Fortunately, "magic-sysrq" allows a relatively clean shutdown.
While we're at it, is there a way for gnumeric to pull in data directly
from postgresql?  ODBC?  I'm aware of copying from postgresql to a CSV
file and importing that, but it's rather clunky.

  My main problem is that columns of several thousand rows are functions
based on other columns of several thousand rows.  For the time-being,
I've split up the spreadsheet into a few pieces, but a database is the
best solution.  If I could run the calculations in the database, and
pull in the final results as static numbers for graphing, that would
greatly reduce the strain on the spreadsheet.  Or is it possible to
graph directly from postgresql?

  I used to work with Oracle and PL/SQL before I retired, so I think I
know what I'm getting into as far as the database stuff is concerned.
Once I get past the Gentoo-specific install problems, I'll subscribe to
a postgresql mailing list, and ask postgresql-specific questions there.

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

Re: [gentoo-user] postgresql-9.1 start - ERROR

[J. Roeleveld]

On Monday, June 09, 2014 10:12:25 PM Joseph wrote:
 On 06/09/14 22:08, Joseph wrote:
 After upgrade when I try to start postgresql I get error:
 
 /etc/init.d/postgresql-9.1 start
 
  * Starting PostgreSQL ...
  * start-stop-daemon: did not create a valid pid in
  `/var/lib/postgresql/9.1/data/postmaster.pid' * Check the log for a
  possible explanation of the above error.
  * /var/lib/postgresql/9.1/data/postmaster.log 
   
 [
 !! ]
 
  * ERROR: postgresql-9.1 failed to start
 
 What is is looking for?
 
 more information from:
 /var/lib/postgresql/9.1/data/postmaster.log
 
 LOG:  database system was shut down at 2014-06-08 09:05:33 MDT
 LOG:  database system is ready to accept connections
 LOG:  autovacuum launcher started
 WARNING:  pgstat wait timeout
 WARNING:  pgstat wait timeout
 LOG:  received smart shutdown request
 LOG:  autovacuum launcher shutting down
 LOG:  shutting down
 LOG:  database system is shut down
 FATAL:  exceeded maxAllocatedDescs (16) while trying to open directory
 /usr/share/zoneinfo FATAL:  exceeded maxAllocatedDescs (16) while trying
 to open directory /usr/share/zoneinfo FATAL:  exceeded maxAllocatedDescs
 (16) while trying to open directory /usr/share/zoneinfo FATAL:  exceeded
 maxAllocatedDescs (16) while trying to open directory /usr/share/zoneinfo


Joseph,

Known issue.
It is unclear if the issue is with PostgeSQL or timezone data or how Gentoo 
packages it.

See bug:
https://bugs.gentoo.org/show_bug.cgi?id=486556

I see 3 possible solutions:
1) Upgrade to PostgreSQL 9.2
2) Downgrade timezone-data to: sys-libs/timezone-data-2013c
3) manually delete the problematic symlink: /usr/share/zoneinfo/posix

I have NOT tested any of these options myself. It is up to you to test this on 
your system.
I think options 1 or 2 have the least amount of risk.

--
Joost

Re: [gentoo-user] postgresql-9.1 start - ERROR

[Joseph]

On 06/10/14 14:54, J. Roeleveld wrote:

On Monday, June 09, 2014 10:12:25 PM Joseph wrote:

On 06/09/14 22:08, Joseph wrote:
After upgrade when I try to start postgresql I get error:

/etc/init.d/postgresql-9.1 start

 * Starting PostgreSQL ...
 * start-stop-daemon: did not create a valid pid in
 `/var/lib/postgresql/9.1/data/postmaster.pid' * Check the log for a
 possible explanation of the above error.
 * /var/lib/postgresql/9.1/data/postmaster.log

[
!! ]

 * ERROR: postgresql-9.1 failed to start

What is is looking for?

more information from:
/var/lib/postgresql/9.1/data/postmaster.log

LOG:  database system was shut down at 2014-06-08 09:05:33 MDT
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
WARNING:  pgstat wait timeout
WARNING:  pgstat wait timeout
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
FATAL:  exceeded maxAllocatedDescs (16) while trying to open directory
/usr/share/zoneinfo FATAL:  exceeded maxAllocatedDescs (16) while trying
to open directory /usr/share/zoneinfo FATAL:  exceeded maxAllocatedDescs
(16) while trying to open directory /usr/share/zoneinfo FATAL:  exceeded
maxAllocatedDescs (16) while trying to open directory /usr/share/zoneinfo



Joseph,

Known issue.
It is unclear if the issue is with PostgeSQL or timezone data or how Gentoo
packages it.

See bug:
https://bugs.gentoo.org/show_bug.cgi?id=486556

I see 3 possible solutions:
1) Upgrade to PostgreSQL 9.2
2) Downgrade timezone-data to: sys-libs/timezone-data-2013c
3) manually delete the problematic symlink: /usr/share/zoneinfo/posix

I have NOT tested any of these options myself. It is up to you to test this on
your system.
I think options 1 or 2 have the least amount of risk.

--
Joost


Thank you for the information. I removed the link /usr/share/zoneinfo/posix
and it worked.  I don't know what this posix link for?

On my system I only have stable postgresql 9.1 and 9.3 I don't have 9.2

[D] dev-db/postgresql-server
Available versions:  
(8.3)  8.3.18 8.3.19 ~8.3.19-r1 8.3.20

(8.4)  8.4.11 8.4.12 ~8.4.12-r1 8.4.13
(9.0)  9.0.7 9.0.8 ~9.0.8-r1 9.0.9
(9.1)  9.1.3 9.1.4 ~9.1.4-r1 9.1.5
(9.2)  ~9.2.0_beta1-r1 ~9.2.0_beta2 ~9.2.0_beta2-r1 ~9.2.0_beta3 ~9.2.0_rc1
(9.3)  **


--
Joseph

[gentoo-user] postgresql-9.1 start - ERROR

[Joseph]

After upgrade when I try to start postgresql I get error:

/etc/init.d/postgresql-9.1 start
* Starting PostgreSQL ...
* start-stop-daemon: did not create a valid pid in 
`/var/lib/postgresql/9.1/data/postmaster.pid'
* Check the log for a possible explanation of the above error.
* /var/lib/postgresql/9.1/data/postmaster.log   [ 
!! ]

* ERROR: postgresql-9.1 failed to start

What is is looking for?

--
Joseph

Re: [gentoo-user] postgresql-9.1 start - ERROR

[Joseph]

On 06/09/14 22:08, Joseph wrote:

After upgrade when I try to start postgresql I get error:

/etc/init.d/postgresql-9.1 start
* Starting PostgreSQL ...
* start-stop-daemon: did not create a valid pid in 
`/var/lib/postgresql/9.1/data/postmaster.pid'
* Check the log for a possible explanation of the above error.
* /var/lib/postgresql/9.1/data/postmaster.log   

[
!! ]
* ERROR: postgresql-9.1 failed to start

What is is looking for?


more information from: 
/var/lib/postgresql/9.1/data/postmaster.log


LOG:  database system was shut down at 2014-06-08 09:05:33 MDT
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
WARNING:  pgstat wait timeout
WARNING:  pgstat wait timeout
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
FATAL:  exceeded maxAllocatedDescs (16) while trying to open directory 
/usr/share/zoneinfo
FATAL:  exceeded maxAllocatedDescs (16) while trying to open directory 
/usr/share/zoneinfo
FATAL:  exceeded maxAllocatedDescs (16) while trying to open directory 
/usr/share/zoneinfo
FATAL:  exceeded maxAllocatedDescs (16) while trying to open directory 
/usr/share/zoneinfo

--
Joseph

Re: [gentoo-user] Postgresql emerge question

[Florian Philipp]

Am 03.03.2011 04:55, schrieb Walter Dnes:
   Am I supposed to emerge postgresql-server?  Any Gentoo-specific
 gotcha's that anyone's aware of?  This will be on pure 64-bit
 (no-multi-lib) Intel i3 with 8 gigs of ram.
 

Yes, if you want to run a postgresql server you emerge this package.
When emerge has finished, take a look at the output at the end. It
contains information on one last step before installation is complete.

I'm not aware of any issues.

Hope this helps,
Florian Philipp



signature.asc
Description: OpenPGP digital signature

[gentoo-user] Postgresql emerge question

[Walter Dnes]

  Am I supposed to emerge postgresql-server?  Any Gentoo-specific
gotcha's that anyone's aware of?  This will be on pure 64-bit
(no-multi-lib) Intel i3 with 8 gigs of ram.

-- 
Walter Dnes waltd...@waltdnes.org

[gentoo-user] postgresql-base-9.0_beta2-r1 and thread safety

[kelly hirai]

i'm getting the following compile error when emerging
postgresql-base-9.0_beta2-r1:

checking thread safety of required library functions... no
configure: error: thread test program failed
This platform is not thread-safe.  Check the file 'config.log' or compile
and run src/test/thread/thread_test for the exact reason.

so i followed the instructions:

cd
/var/tmp/portage/dev-db/postgresql-base-9.0_beta2-r1/work/postgresql-9.0beta2/
./configure
...
checking thread safety of required library functions... yes
...
make
...
cd src/test/thread
make
./thread_test

yields:

Your errno is thread-safe.
Your system has sterror_r();  it does not need strerror().
Your system has getpwuid_r();  it does not need getpwuid().
Your system has getaddrinfo();  it does not need gethostbyname()
  or gethostbyname_r().

Your platform is thread-safe.

any hints?

kelly hirai

[gentoo-user] postgresql 8.2.14 cannot stop

[Xi Shen]

hi,

my system is gentoo amd64, postgresql 8.2.14. the postgresql service
can start without any error; but when i try to stop it, it reports
failure. the process of postgresql are actually terminated. if i try
to stop the service again, i got

start-stop-daemon: fopen `/var/lib/postgresql/data/postmaster.pid':
No such file or direc

i will have to use zap to reset the service status. how to fix this?


-- 
Best Regards,
David Shen

http://twitter.com/davidshen84/

Re: [gentoo-user] postgresql 8.2.14 cannot stop

[Alan McKinnon]

On Tuesday 30 March 2010 15:31:50 Xi Shen wrote:
 hi,
 
 my system is gentoo amd64, postgresql 8.2.14. the postgresql service
 can start without any error; but when i try to stop it, it reports
 failure. the process of postgresql are actually terminated. if i try
 to stop the service again, i got
 
 start-stop-daemon: fopen `/var/lib/postgresql/data/postmaster.pid':
 No such file or direc
 
 i will have to use zap to reset the service status. how to fix this?


We don't need the error message from the second time you stop it. We know for 
sure that will not work. We need the error from the first failure. Without 
that, we can't help you.

Otherwise you are just asking how long is a piece of string? And we all know 
what the correct answer to that is.

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] postgresql 8.2.14 cannot stop

[Florian Philipp]

Am 30.03.2010 15:31, schrieb Xi Shen:
 hi,
 
 my system is gentoo amd64, postgresql 8.2.14. the postgresql service
 can start without any error; but when i try to stop it, it reports
 failure. the process of postgresql are actually terminated. if i try
 to stop the service again, i got
 
 start-stop-daemon: fopen `/var/lib/postgresql/data/postmaster.pid':
 No such file or direc
 
 i will have to use zap to reset the service status. how to fix this?
 
 

I guess it is just a timeout issue. Take a look at the stop()-function
in /etc/init.d/postgresql. It calls
start-stop-daemon [...]
--retry -TERM/${WAIT_FOR_DISCONNECT}/-INT/${WAIT_FOR_CLEANUP}/-QUIT [...]

I have the same issue with a virtual server. At the moment, I just
ignore it. I guess it would be safer to increase the timeout somehow.

Hope this helps,
Florian Philipp



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-user] postgresql 8.2.14 cannot stop

[Michael Higgins]

On Tue, 30 Mar 2010 21:31:50 +0800
Xi Shen davidshe...@googlemail.com wrote:

 hi,
 
 my system is gentoo amd64, postgresql 8.2.14. the postgresql service
 can start without any error; but when i try to stop it, it reports
 failure. the process of postgresql are actually terminated. if i try
 to stop the service again, i got
 
 start-stop-daemon: fopen `/var/lib/postgresql/data/postmaster.pid':
 No such file or direc
 
 i will have to use zap to reset the service status. how to fix this?
 
 

There are new start stop scripts for gentoo that work at
http://bugs.gentoo.org/show_bug.cgi?id=311047 

Enjoy!

-- 
 |\  /||   |  ~ ~  
 | \/ ||---|  `|` ?
 ||ichael  |   |iggins\^ /
 michael.higgins[at]evolone[dot]org

Re: [gentoo-user] postgresql 8.2.14 cannot stop

[Peter Humphrey]

On Tuesday 30 March 2010 20:15:41 Alan McKinnon wrote:

 ... how long is a piece of string? And we all know what the correct
 answer to that is.

Yeah. Twice as long as half a piece.

-- 
Rgds
Peter.

Re: [gentoo-user] postgresql 8.2.14 cannot stop

[Xi Shen]

On Wed, Mar 31, 2010 at 6:58 AM, Michael Higgins li...@evolone.org wrote:

 There are new start stop scripts for gentoo that work at
 http://bugs.gentoo.org/show_bug.cgi?id=311047


thanks. i think the new scripts can help me.


-- 
Best Regards,
David Shen

http://twitter.com/davidshen84/

Re: [gentoo-user] PostgreSQL server blocker

[Justin]

Stroller wrote:
 Hi there,

 !!! The following installed packages are masked:
 - dev-db/postgresql-8.1.11 (masked by: package.mask)
 /usr/portage/profiles/package.mask:
 # Patrick Lauer patr...@gentoo.org (03 Oct 2009)
 # Mask unsupported and obsolete postgres packages
 # Use postgresql-server (or postgresql-base) instead


So just unemerge your dev-db/postgresql and try again. everything should
be fine then.

jusitn



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-user] PostgreSQL server blocker

[Stroller]

On 4 Oct 2009, at 08:40, Justin wrote:

Stroller wrote:

Hi there,



!!! The following installed packages are masked:
- dev-db/postgresql-8.1.11 (masked by: package.mask)
/usr/portage/profiles/package.mask:
# Patrick Lauer patr...@gentoo.org (03 Oct 2009)
# Mask unsupported and obsolete postgres packages
# Use postgresql-server (or postgresql-base) instead



So just unemerge your dev-db/postgresql and try again. everything  
should

be fine then.


Uh, if I unmerge dev-db/postgresql then I won't have the database  
backend that my accounts system depends upon.


Did you read the part of my message where I said that:

 I'm running x86 (NOT ~x86), and all versions of postgresql-server  
are marked with a ~.


??

Stroller.

Re: [gentoo-user] PostgreSQL server blocker

[Justin]

Stroller wrote:
 
 On 4 Oct 2009, at 08:40, Justin wrote:
 Stroller wrote:
 Hi there,

 !!! The following installed packages are masked:
 - dev-db/postgresql-8.1.11 (masked by: package.mask)
 /usr/portage/profiles/package.mask:
 # Patrick Lauer patr...@gentoo.org (03 Oct 2009)
 # Mask unsupported and obsolete postgres packages
 # Use postgresql-server (or postgresql-base) instead


 So just unemerge your dev-db/postgresql and try again. everything should
 be fine then.
 
 Uh, if I unmerge dev-db/postgresql then I won't have the database
 backend that my accounts system depends upon.
 
 Did you read the part of my message where I said that:
 
  I'm running x86 (NOT ~x86), and all versions of postgresql-server
 are marked with a ~.
 
 ??
 
 Stroller.
 

You are right but the problem should be solved 'cause it isunmasked again.
I couldn't imagine a dev masking something where there is no stable
alternative.


justin





signature.asc
Description: OpenPGP digital signature

Re: [gentoo-user] PostgreSQL server blocker

[Stroller]

On 4 Oct 2009, at 15:54, Justin wrote:

...
Did you read the part of my message where I said that:


I'm running x86 (NOT ~x86), and all versions of postgresql-server
are marked with a ~.


You are right but the problem should be solved 'cause it isunmasked  
again.

I couldn't imagine a dev masking something where there is no stable
alternative.


I've just synced and it still seems to be ~ only.

Am I reading this wrong?
http://packages.gentoo.org/package/dev-db/postgresql-server

I'm thinking to file a no stable ebuild for PostgreSQL server, but  
I'm just cautious in case I'm overlooking something stupid and obvious.


Stroller.

Re: [gentoo-user] PostgreSQL server blocker

[Justin]

Stroller wrote:
 
 On 4 Oct 2009, at 15:54, Justin wrote:
 ...
 Did you read the part of my message where I said that:

 I'm running x86 (NOT ~x86), and all versions of postgresql-server
 are marked with a ~.

 You are right but the problem should be solved 'cause it isunmasked
 again.
 I couldn't imagine a dev masking something where there is no stable
 alternative.
 
 I've just synced and it still seems to be ~ only.
 
 Am I reading this wrong?
 http://packages.gentoo.org/package/dev-db/postgresql-server
 
 I'm thinking to file a no stable ebuild for PostgreSQL server, but I'm
 just cautious in case I'm overlooking something stupid and obvious.
 
 Stroller.
 
 
You missunderstood me. The keyword situation is the same, means -doc and
-base are  stable but -server not. But the masking of dev-db/postgresql
is reverted. So you will not have any problems.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-user] PostgreSQL server blocker

[Stroller]

On 4 Oct 2009, at 16:13, Justin wrote:

I'm running x86 (NOT ~x86), and all versions of postgresql-server
are marked with a ~.


You are right but the problem should be solved 'cause it isunmasked
again.
I couldn't imagine a dev masking something where there is no stable
alternative.


I've just synced and it still seems to be ~ only.

Am I reading this wrong?
http://packages.gentoo.org/package/dev-db/postgresql-server

I'm thinking to file a no stable ebuild for PostgreSQL server,  
but I'm

just cautious in case I'm overlooking something stupid and obvious.



You missunderstood me. The keyword situation is the same, means -doc  
and
-base are  stable but -server not. But the masking of dev-db/ 
postgresql

is reverted. So you will not have any problems.


Thanks! You're right. Updating world now. :D

I also checked dev-db/postgresql on packages.gentoo.org before  
posting, but I guess that must not be immediately updating.


Stroller.

[gentoo-user] PostgreSQL server blocker

[Stroller]

Hi there,

For the longest time I've had PostgreSQL server installed on one of my  
systems. I update world every month or two and today `emerge -upv  
world` shows this blocker:


...
[ebuild U ] app-admin/sudo-1.7.2_p1 [1.7.1-r1] USE=offensive pam - 
ldap (-selinux) -skey 753 kB
[ebuild  N] app-emacs/emacs-common-gentoo-1.2  USE=-X - 
emacs22icons 46 kB
[ebuild  NS   ] app-editors/emacs-23.1 [22.3] USE=gif jpeg png tiff  
xpm -X -Xaw3d -alsa -dbus -gpm -gtk -gzip-el -hesiod -kerberos -m17n- 
lib -motif -sound -source -svg -toolkit-scroll-bars -xft 33,577 kB

[ebuild U ] virtual/emacs-23 [22] 0 kB
[ebuild  N] app-admin/eselect-postgresql-0.3  3 kB
[ebuild  N] dev-db/postgresql-base-8.1.11  USE=doc nls pam  
readline ssl zlib -kerberos -pg-intdatetime -threads LINGUAS=-af -cs  
-de -es -fa -fr -hr -hu -it -ko -nb -pl -pt_BR -ro -ru -sk -sl -sv -tr  
-zh_CN -zh_TW 0 kB
[uninstall] dev-db/libpq-8.1.11  USE=nls pam readline ssl zlib - 
kerberos -pg-intdatetime -threads
[blocks b ] dev-db/libpq (dev-db/libpq is blocking dev-db/ 
postgresql-base-8.1.11)
[blocks b ] dev-db/libpq (dev-db/libpq is blocking app-admin/ 
eselect-postgresql-0.3)
[blocks b ] dev-db/postgresql-base (dev-db/postgresql-base is  
blocking dev-db/libpq-8.1.11)

[ebuild  NS   ] virtual/postgresql-base-8.1 [8.0] 0 kB
[ebuild U ] dev-perl/DBD-Pg-2.15.1 [1.49] 216 kB
[blocks B ] dev-db/postgresql (dev-db/postgresql is blocking dev- 
db/postgresql-base-8.1.11)


Total: 36 packages (26 upgrades, 5 new, 5 in new slots, 1 uninstall),  
Size of downloads: 116,469 kB

Conflict: 4 blocks (1 unsatisfied)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  ('ebuild', '/', 'dev-db/postgresql-base-8.1.11', 'merge') pulled in  
by
dev-db/postgresql-base:8.1 required by ('ebuild', '/', 'dev-perl/ 
DBD-Pg-2.15.1', 'merge')
dev-db/postgresql-base:8.1 required by ('ebuild', '/', 'virtual/ 
postgresql-base-8.1', 'merge')


  ('installed', '/', 'dev-db/postgresql-8.1.11', 'nomerge') pulled in  
by

dev-db/postgresql required by world


For more information about Blocked Packages, please refer to the  
following

section of the Gentoo Linux x86 Handbook (architecture is irrelevant):

http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?full=1#blocked


!!! The following installed packages are masked:
- dev-db/postgresql-8.1.11 (masked by: package.mask)
/usr/portage/profiles/package.mask:
# Patrick Lauer patr...@gentoo.org (03 Oct 2009)
# Mask unsupported and obsolete postgres packages
# Use postgresql-server (or postgresql-base) instead

For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.

$

Is anyone else seeing this, please?

It looks like I need to unmerge dev-db/postgresql (I'm currently  
running 8.1.11) and emerge dev-db/postgresql-server, instead.


But I'm running x86 (NOT ~x86), and all versions of postgresql-server  
are marked with a ~.


I'm guessing from the date on Mr Lauer's masking above that if I sync  
in a day's time then my major version (at least) of postgresql-server  
will be marked as stable, however I'm extremely wary because my long  
update period has ensured that I've never before seen a blocker which  
has not already been discussed on this list.


I would be grateful for any thoughts or comments,

Stroller.

Re: [gentoo-user] PostgreSQL server blocker

[Dirk Heinrichs]

Am Samstag 03 Oktober 2009 22:59:15 schrieb Stroller:
 For the longest time I've had PostgreSQL server installed on one of my  
 systems. I update world every month or two and today `emerge -upv  
 world` shows this blocker:

Postgresql packages have been split. They are now named postgresql-base, -doc 
and -server. You need to uninstall your current packages and install those.

HTH...

Dirk

Re: [gentoo-user] PostgreSQL server blocker

[Dirk Heinrichs]

Am Sonntag 04 Oktober 2009 07:34:24 schrieb Dirk Heinrichs:
 Am Samstag 03 Oktober 2009 22:59:15 schrieb Stroller:
  For the longest time I've had PostgreSQL server installed on one of my
  systems. I update world every month or two and today `emerge -upv
  world` shows this blocker:
 
 Postgresql packages have been split. They are now named postgresql-base,
  -doc and -server. You need to uninstall your current packages and install
  those.

Forgot to add: The new packages are also slotted now. Means you can install 
several major versions (back to 7.3.x) next to each other to ease upgrade:

Install new version, dump databases of old version, stop old, start new, 
import databases, uninstall old version.

Bye...

Dirk

[gentoo-user] PostgreSQL: dev-db/postgresql, dev-db/postgresql-server, or virtual/postgresql-server?

[Hilco Wijbenga]

I want to install PostgreSQL but I'm wondering which package to use.
The obvious choice (dev-db/postgresql) installs 8.2.7 but
dev-db/postgresql-server installs 8.3.5. (I see they even use the same
tarball.) Is postgresql-server is the proper way going forward?

What about virtual/postgresql-server? Should I prefer that over
dev-db/postgresql-server?

Why do we even have a virtual? I thought they were for adding a
service if you don't care which specific package supplies it.

Cheers,
Hilco

Re: [gentoo-user] PostgreSQL: dev-db/postgresql, dev-db/postgresql-server, or virtual/postgresql-server?

[Alan McKinnon]

On Sunday 21 December 2008 23:45:33 Hilco Wijbenga wrote:
 I want to install PostgreSQL but I'm wondering which package to use.
 The obvious choice (dev-db/postgresql) installs 8.2.7 but
 dev-db/postgresql-server installs 8.3.5. (I see they even use the same
 tarball.) Is postgresql-server is the proper way going forward?

Yes.

postgresql-server is not something new, it's a new way of packaging an 
existing product. It's like the monolithic/split KDE ebuilds, the new split 
postgresql packages make the dev's life easier, and make it possible for you 
to make more modular choices about what you want support for.

Plus, the old packages only go as far a version 8.3.1
Only the new packages are supported after that version

 What about virtual/postgresql-server? Should I prefer that over
 dev-db/postgresql-server?

If you emerge the virtual you will get the default method for your platform, 
whatever that is

 Why do we even have a virtual? I thought they were for adding a
 service if you don't care which specific package supplies it.

You have two packages supplying the same thing - monolithic postgresql and 
split postgresql. Your system does not care which ebuild you ran to get 
postgresql, as long as you have one.

This is no different to any other virtual, besides the fact that the source 
tarball is the same one for both.



-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] PostgreSQL: dev-db/postgresql, dev-db/postgresql-server, or virtual/postgresql-server?

[Graham Murray]

Alan McKinnon alan.mckin...@gmail.com writes:

 postgresql-server is not something new, it's a new way of packaging an 
 existing product. It's like the monolithic/split KDE ebuilds, the new split 
 postgresql packages make the dev's life easier, and make it possible for you 
 to make more modular choices about what you want support for.

 Plus, the old packages only go as far a version 8.3.1
 Only the new packages are supported after that version

The other advantage is that it makes the upgrade process considerably
easier and safer. The new postresql-[base|server|docs] ebuilds are
slotted. 

Re: [gentoo-user] PostgreSQL: dev-db/postgresql, dev-db/postgresql-server, or virtual/postgresql-server?

[Alan McKinnon]

On Monday 22 December 2008 00:07:01 Graham Murray wrote:
 Alan McKinnon alan.mckin...@gmail.com writes:
  postgresql-server is not something new, it's a new way of packaging an
  existing product. It's like the monolithic/split KDE ebuilds, the new
  split postgresql packages make the dev's life easier, and make it
  possible for you to make more modular choices about what you want support
  for.
 
  Plus, the old packages only go as far a version 8.3.1
  Only the new packages are supported after that version

 The other advantage is that it makes the upgrade process considerably
 easier and safer. The new postresql-[base|server|docs] ebuilds are
 slotted.

What's the rationale behind that? I can see why someone might want two or more 
versions of php, python, perl or mysql.

But postgresql? I can't imagine why it would be useful to the majority to have 
SLOTs for postgresql. People tend to run one version doing one major job, 
which is often not the case for the other examples above.

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] PostgreSQL: dev-db/postgresql, dev-db/postgresql-server, or virtual/postgresql-server?

[felix]

On Mon, Dec 22, 2008 at 02:01:59AM +0200, Alan McKinnon wrote:

 What's the rationale behind that? I can see why someone might want two or 
 more 
 versions of php, python, perl or mysql.
 
 But postgresql? I can't imagine why it would be useful to the majority to 
 have 
 SLOTs for postgresql. People tend to run one version doing one major job, 
 which is often not the case for the other examples above.

The only advantage I see is upgrades.  The dump and restore is a PITA,
and if you forget and install the new before dumping, you have to
reinstall the old, dump, and rereinstall the new.  You could probably
also set up Slony to manage both of them at once, running on different
ports, and upgrade that way, btu I have never tried Slony.

-- 
... _._. ._ ._. . _._. ._. ___ .__ ._. . .__. ._ .. ._.
 Felix Finch: scarecrow repairman  rocket surgeon / fe...@crowfix.com
  GPG = E987 4493 C860 246C 3B1E  6477 7838 76E9 182E 8151 ITAR license #4933
I've found a solution to Fermat's Last Theorem but I see I've run out of room o

Re: [gentoo-user] PostgreSQL: dev-db/postgresql, dev-db/postgresql-server, or virtual/postgresql-server?

[Alan McKinnon]

On Monday 22 December 2008 03:01:07 fe...@crowfix.com wrote:
 On Mon, Dec 22, 2008 at 02:01:59AM +0200, Alan McKinnon wrote:
  What's the rationale behind that? I can see why someone might want two or
  more versions of php, python, perl or mysql.
 
  But postgresql? I can't imagine why it would be useful to the majority to
  have SLOTs for postgresql. People tend to run one version doing one major
  job, which is often not the case for the other examples above.

 The only advantage I see is upgrades.  The dump and restore is a PITA,
 and if you forget and install the new before dumping, you have to
 reinstall the old, dump, and rereinstall the new.  You could probably
 also set up Slony to manage both of them at once, running on different
 ports, and upgrade that way, btu I have never tried Slony.

Ah yes, dump and restore. I hadn't considered that.

-- 
alan dot mckinnon at gmail dot com

[gentoo-user] Postgresql: failed to initialize lc_messages to

[Ricardo Bevilacqua]

Hi to all,

I'm setting up a new server for developing database-based applications.

So, I emerged PostgreSQL, but the problem comes when I try to
configure Postgre, this is what I get:

-
# emerge postgresql --config


Configuring pkg...

 * Creating the data directory ...
 * Initializing the database ...
The files belonging to this database system will be owned by user postgres.
This user must also own the server process.

The database cluster will be initialized with locale C.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating directory /var/lib/postgresql/data/global ... ok
creating directory /var/lib/postgresql/data/pg_xlog ... ok
creating directory /var/lib/postgresql/data/pg_xlog/archive_status ... ok
creating directory /var/lib/postgresql/data/pg_clog ... ok
creating directory /var/lib/postgresql/data/pg_subtrans ... ok
creating directory /var/lib/postgresql/data/base ... ok
creating directory /var/lib/postgresql/data/base/1 ... ok
creating directory /var/lib/postgresql/data/pg_tblspc ... ok
selecting default max_connections ... 10
selecting default shared_buffers ... 50
creating configuration files ... ok
creating template1 database in /var/lib/postgresql/data/base/1 ...
FATAL:  XX000: failed to initialize lc_messages to 
LOCATION:  InitializeGUCOptions, guc.c:2403
child process exited with exit code 1
initdb: removing contents of data directory /var/lib/postgresql/data
 *
 * You can use the '//etc/init.d/postgresql' script to run PostgreSQL
instead of 'pg_ctl'.
 *
-


I did some google research and found two possible solutions, neither
of those two worked for me.

The first one [1] said that the solution is to compile glibc with nls.
But, apparently, I already have glibc installed with nls.


-
# emerge -va glibc

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ] sys-libs/glibc-2.6.1  USE=gd nls -debug -glibc-omitfp
(-hardened) (-multilib) -profile (-selinux) -vanilla 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB

Would you like to merge these packages? [Yes/No]

-

The second solution [2] that I found consists on editing the
/etc/env.d/02locale file with UTF-8, like this (this is my actual
02locale file):

---/etc/env.d/02locale--

LANG=es...@euro

LC_CTYPE=es_ES.UTF-8
LC_NUMERIC=es_ES.UTF-8
LC_TIME=es_ES.UTF-8
LC_COLLATE=es_ES.UTF-8
LC_MONETARY=es_ES.UTF-8
LC_MESSAGES=es_ES.UTF-8
LC_PAPER=es_ES.UTF-8
LC_NAME=es_ES.UTF-8
LC_ADDRESS=es_ES.UTF-8
LC_TELEPHONE=es_ES.UTF-8
LC_MEASUREMENT=es_ES.UTF-8
LC_IDENTIFICATION=es_ES.UTF-8
LC_ALL=es_ES.UTF-8

-

Still the same error.

I hope someone helps me to fix this, ask me if you need to see any
other of my configuration files.

Thanks in advance!

Richard.



[1] 
http://forums.gentoo.org/viewtopic-p-1950119-highlight-fatal+xx000+initialize+lcmessages.html?sid=d01013a2790fc89ff6ebb6df84010702#1950119

[2] 
http://forum.soft32.com/linux/gentoo-postgres-postinstall-problem-ftopict334580.html

[gentoo-user] postgresql replication with slony1

[Zhu Sha Zang]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I need know if somebody here has installed/configured a postgresql 
replication where gentoo is a master host. I'm trying to make it using 
slony1, but have some mismatch configuration, look:

/
USER=postgres
CLUSTER=slave.host
DBUSER=snort
DBNAME=snort_log
DBHOST=master.host

LOGFILE=/var/lib/postgresql/data/slony1.log
LOGLEVEL=4   # 1(minimum)..4(maximum)/

I think that the error are in CLUSTER option looking in slony1's log:

/2008-08-28 15:16:57 BRT CONFIG main: slon version 1.2.10 starting up
2008-08-28 15:16:57 BRT DEBUG2 slon: watchdog process started
2008-08-28 15:16:57 BRT DEBUG2 slon: watchdog ready - pid = 22743
2008-08-28 15:16:57 BRT ERROR  cannot get sl_local_node_id - ERROR:  
schema _solaris.semfio.usp.br does not exi

st
2008-08-28 15:16:57 BRT FATAL  main: Node is not initialized properly - 
sleep 10s

2008-08-28 15:16:57 BRT DEBUG2 slon: worker process created - pid = 22751
2008-08-28 15:17:02 BRT DEBUG1 slon: shutdown requested
2008-08-28 15:17:02 BRT DEBUG2 slon: notify worker process to shutdown
2008-08-28 15:17:07 BRT DEBUG2 slon_retry() from pid=22751
2008-08-28 15:17:07 BRT DEBUG1 slon: retry requested
2008-08-28 15:17:07 BRT DEBUG2 slon: notify worker process to shutdown
2008-08-28 15:17:07 BRT FATAL  main: write to worker pipe failed -(9) 
Bad file descriptor

2008-08-28 15:17:07 BRT DEBUG2 slon: remove pid file
2008-08-28 15:17:07 BRT DEBUG2 slon: exit(-1)

/Then, if some smart guy has implemented yet this service have some 
trick or how-to to make it work in my gentoo box, i'll very gratefully.


If have any other method to make a postgresql repliction, like 
pgcluster, please talk to me.


Thanks for this time.

Zhu Sha Zang


-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAki29QcACgkQbLyL8mxGP9QO1gCgtv29yK1wiXcnGds/y+o4FieR
WpIAnjEyYlGmCnRoOEkBPBJqJjNI1sEF
=IIWx
-END PGP SIGNATURE-






___ 
Yahoo! Mail - Sempre a melhor opção para você! 
Experimente já e veja as novidades. 
http://br.yahoo.com/mailbeta/tudonovo/

[gentoo-user] Postgresql native replication problem

[Rafael Barrera Oro]

I tried to activate PostgreSQL native replication, but after emerging
postgresql and pgperl i found out that the following files (which are needed
to get the job done) were missing:

/usr/bin/DBMirror.pl
/usr/lib/postgresql/pending.so
/usr/share/postgresql/contrib/AddTrigger.sql
/usr/share/postgresql/contrib/MirrorSetup.sql
/usr/share/postgresql/contrib/slaveDatabase.conf

i am using this tutorial to guide myself:

http://rockfloat.com/howto/gentoo-postgres-replication-native.html

is there any other package i should emerge?

Any help would be greatly appreciated. Thanks in advance!

Rafael

[gentoo-user] postgresql weird incompatibility problem

[Rafael Barrera Oro]

I unmerged postgresql 8.0.3 in order to emerge 8.2.x, now when i try to do
anything i get the following message:

psql: FATAL:  database files are incompatible with server
DETAIL:  The data directory was initialized by PostgreSQL version 8.2, which
is not compatible with this version 8.0.13.

i deleted previous /var/lib/postgresql in order to be able to emerge the new
version, so i am clueless...

does this ring a bell for anyone?

as usual, thanks in advance to all

sincerely yours

Rafael

Re: [gentoo-user] postgresql weird incompatibility problem

[Arturo 'Buanzo' Busleiman]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Rafael Barrera Oro wrote:
 i deleted previous /var/lib/postgresql in order to be able to emerge the
 new version, so i am clueless...

So, you deleted your databases?? Please tell us you have a copy!

- --
Arturo Buanzo Busleiman - Consultor Independiente en Seguridad Informatica
Servicios Ofrecidos: http://www.buanzo.com.ar/pro/
Unase a los Foros GNU/Buanzo - La palabra Comunidad en su maxima expresion.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHFNJfAlpOsGhXcE0RCgAvAJ0R3DqD6pvutyyjYLrXNlom1jJ3wgCfQjn1
BgBsIuMvzqmg4To2TEm1jFY=
=ZTQG
-END PGP SIGNATURE-
-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-user] postgresql weird incompatibility problem

[Albert Hopkins]

On Tue, 2007-10-16 at 11:53 -0300, Rafael Barrera Oro wrote:
 I unmerged postgresql 8.0.3 in order to emerge 8.2.x, now when i try
 to do anything i get the following message:
 
 psql: FATAL:  database files are incompatible with server
 DETAIL:  The data directory was initialized by PostgreSQL version 8.2,
 which is not compatible with this version 8.0.13.
 
 i deleted previous /var/lib/postgresql in order to be able to emerge
 the new version, so i am clueless...
 
 does this ring a bell for anyone?

Did you stop postgres *before* uninstalling it and
removing /var/lib/postgresql?  Also did you emerge --config
postgresql-... *after* installing the new postgres?

-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-user] postgresql weird incompatibility problem

[Rafael Barrera Oro]

I just installed it so i did not have any useful data. I must admit though,
i did not stop the database before unmerging and emerging, however i did run
emerge --config afterwards.

Thanks for the replies :D

2007/10/16, Albert Hopkins [EMAIL PROTECTED]:


 On Tue, 2007-10-16 at 11:53 -0300, Rafael Barrera Oro wrote:
  I unmerged postgresql 8.0.3 in order to emerge 8.2.x, now when i try
  to do anything i get the following message:
 
  psql: FATAL:  database files are incompatible with server
  DETAIL:  The data directory was initialized by PostgreSQL version 8.2,
  which is not compatible with this version 8.0.13.
 
  i deleted previous /var/lib/postgresql in order to be able to emerge
  the new version, so i am clueless...
 
  does this ring a bell for anyone?

 Did you stop postgres *before* uninstalling it and
 removing /var/lib/postgresql?  Also did you emerge --config
 postgresql-... *after* installing the new postgres?

 --
 [EMAIL PROTECTED] mailing list

Re: [gentoo-user] postgresql weird incompatibility problem

[Albert Hopkins]

On Tue, 2007-10-16 at 12:01 -0300, Arturo 'Buanzo' Busleiman wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512
 
 Rafael Barrera Oro wrote:
  i deleted previous /var/lib/postgresql in order to be able to emerge the
  new version, so i am clueless...
 
 So, you deleted your databases?? Please tell us you have a copy!

I'm guessing he deleted them because he no longer needed them(?).

-a

-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-user] postgresql weird incompatibility problem

[Arturo 'Buanzo' Busleiman]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Albert Hopkins wrote:
 I'm guessing he deleted them because he no longer needed them(?).

Luckily, he just mentioned that it was a fresh 8.0 install, so the databases 
were useless anyway :)

- --
Arturo Buanzo Busleiman - Consultor Independiente en Seguridad Informatica
Servicios Ofrecidos: http://www.buanzo.com.ar/pro/
Unase a los Foros GNU/Buanzo - La palabra Comunidad en su maxima expresion.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHFNeHAlpOsGhXcE0RCtbnAJ9xHEmnuZZlDy5rxXZpoTRwtOti8wCfZlzL
9rtD5Mnnvpi3bDSqKFINuaY=
=O5Dv
-END PGP SIGNATURE-
-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-user] postgresql weird incompatibility problem

[Albert Hopkins]

On Tue, 2007-10-16 at 12:10 -0300, Rafael Barrera Oro wrote:
 I just installed it so i did not have any useful data. I must admit
 though, i did not stop the database before unmerging and emerging,
 however i did run emerge --config afterwards.
 

You should have stopped the database before you remove the database
files... Did you ever stop the database?  If not, you're still running
8.0 while your emerge installed 8.2 files and then the database
naturally got confused.  The --config probably tried to connect to the
running 8.0 service and told it do do something with the installed 8.2
files.

It doesn't really matter *what* happened exactly.  The point is you
should just start over.

  * Stop postgres. kill all postgres processes.
  * unmerge postgress
  * remove /var/lib/postgresql
  * re-install posgress




-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-user] postgresql weird incompatibility problem

[Rafael Barrera Oro]

That worked perfectly, thanks for all your help people.


2007/10/16, Albert Hopkins [EMAIL PROTECTED]:


 On Tue, 2007-10-16 at 12:10 -0300, Rafael Barrera Oro wrote:
  I just installed it so i did not have any useful data. I must admit
  though, i did not stop the database before unmerging and emerging,
  however i did run emerge --config afterwards.
 

 You should have stopped the database before you remove the database
 files... Did you ever stop the database?  If not, you're still running
 8.0 while your emerge installed 8.2 files and then the database
 naturally got confused.  The --config probably tried to connect to the
 running 8.0 service and told it do do something with the installed 8.2
 files.

 It doesn't really matter *what* happened exactly.  The point is you
 should just start over.

   * Stop postgres. kill all postgres processes.
   * unmerge postgress
   * remove /var/lib/postgresql
   * re-install posgress




 --
 [EMAIL PROTECTED] mailing list

Re: [gentoo-user] postgresql weird incompatibility problem

[Dirk Heinrichs]

Am Dienstag, 16. Oktober 2007 schrieb Rafael Barrera Oro:

 I unmerged postgresql 8.0.3 in order to emerge 8.2.x

Please be aware that in the future, once you have real data in your databases, 
you would need to dump all the databases (pg_dump_all) prior to a major 
update, ((8.x - 8.y), then, after update, re-initialize the database and 
import the dumps.

And that update is coming soon, 8.3 has entered beta phase.

Bye...

Dirk


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] postgresql problem

[Dirk Heinrichs]

Am Dienstag, 9. Oktober 2007 schrieb ext Rafael Barrera Oro:
 Hello, i'd like to ask you people if anyone has succeded in installing a
 postgresql version greater than 8.0.13 because i tried (unmasking by
 adding ~x86 to ACCEPT_KEYWORDS) and ran into a lot of problems. I don't
 have the logs with me, so i will post a description of such problems in
 the near future.

I'm running postgresql 8.2.x from the postgresql-experimental overlay w/o 
probs.

Bye...

Dirk
-- 
Dirk Heinrichs  | Tel:  +49 (0)162 234 3408
Configuration Manager   | Fax:  +49 (0)211 47068 111
Capgemini Deutschland   | Mail: [EMAIL PROTECTED]
Wanheimerstraße 68  | Web:  http://www.capgemini.com
D-40468 Düsseldorf  | ICQ#: 110037733
GPG Public Key C2E467BB | Keyserver: www.keyserver.net


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] postgresql problem

[Ow Mun Heng]

On Wed, 2007-10-10 at 08:07 +0200, Dirk Heinrichs wrote:
 Am Dienstag, 9. Oktober 2007 schrieb ext Rafael Barrera Oro:
  Hello, i'd like to ask you people if anyone has succeded in installing a
  postgresql version greater than 8.0.13 because i tried (unmasking by
  adding ~x86 to ACCEPT_KEYWORDS) and ran into a lot of problems. I don't
  have the logs with me, so i will post a description of such problems in
  the near future.
 
 I'm running postgresql 8.2.x from the postgresql-experimental overlay w/o 
 probs.
 
Running 8.2.4 here as well..

anyone got pgfouine to work? The overlay has issues. Wrong manifest is
one. But after that, can't get it installed properly. Complains about
not having apache. (i have apache 2)


-- 
[EMAIL PROTECTED] mailing list

[gentoo-user] postgresql problem

[Rafael Barrera Oro]

Hello, i'd like to ask you people if anyone has succeded in installing a
postgresql version greater than 8.0.13 because i tried (unmasking by adding
~x86 to ACCEPT_KEYWORDS) and ran into a lot of problems. I don't have the
logs with me, so i will post a description of such problems in the near
future.

Thanks in advance y'all

Rafael

PD: The system were i am installing is a Pentium III with 1GB of ram

Re: [gentoo-user] postgresql problem

[Rumen Yotov]

On Tue, 9 Oct 2007 16:22:10 -0300
Rafael Barrera Oro [EMAIL PROTECTED] wrote:

 Hello, i'd like to ask you people if anyone has succeded in
 installing a postgresql version greater than 8.0.13 because i tried
 (unmasking by adding ~x86 to ACCEPT_KEYWORDS) and ran into a lot of
 problems. I don't have the logs with me, so i will post a description
 of such problems in the near future.
 
 Thanks in advance y'all
 
 Rafael
 
 PD: The system were i am installing is a Pentium III with 1GB of ram
Hi,

The versions in ~x86 are not masked, just in testing (ebuild testing).
Check Bugzilla for any Bugs and (eventually) post a new one with logs.
HTH. Rumen
-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-user] Postgresql ODBC and JDBC in Gentoo

[Mark Kirkwood]

Brett I. Holcomb wrote:
I've installed postgres and am trying to get it to work with OpenOffice 2.0.3. 
After searching the web, postgres sites, unixODBC site, etc. I still haven't 
figured out how to get ODBC and JDBC to see postgres. How do I setup ODBC and 
JDBC so that my postgres databases are available?  Any docs, hints, etc. 
would be appreciated.





I'm not sure about ODBC, but for JDBC the general instructions are:

1/ ensure that the CLASSPATH includes the driver jar (e.g. 
CLASSPATH=.:/usr/local/pgsql/share/postgresql.jar)


2/ Check that postgres server has listen_address='*' in its 
configuration file (postgresql.conf)


3/ Set the connection url for the database you want (e.g. 
jdbc:postgresql://localhost:5432/postgres to connect to a server using 
port 5432 and a database called postgres)


Presumably openoffice lets you tell it 1/ and 3/, its up to you to make 
sure of 2/ !


Cheers

Mark
--
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Postgresql ODBC and JDBC in Gentoo

[Mark Kirkwood]

Mark Kirkwood wrote:


2/ Check that postgres server has listen_address='*' in its 
configuration file (postgresql.conf)





Should be listen_addresses='*'. Sorry.
--
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Postgresql ODBC and JDBC in Gentoo

[Brett I. Holcomb]

Thank you.  I'll see if I can get that going.  I finally got ODBC working and  
make some notes here for others who may search in the future.  I simply 
updated the /etc/unixODBC in files per the specs on the unixODBC site - 
although another part of their site says postgres doesn't use these ini 
files!  I also found that Gentoo does not have an ODBCConfig - it makes it 
gODBCConfig and you have to enable the gnome flag for it to be built.  

I'll see if I can get JDBC working now.

Thanks.


On Saturday September 16 2006 02:38, Mark Kirkwood wrote:
 Brett I. Holcomb wrote:
  I've installed postgres and am trying to get it to work with OpenOffice
  2.0.3. After searching the web, postgres sites, unixODBC site, etc. I
  still haven't figured out how to get ODBC and JDBC to see postgres. How
  do I setup ODBC and JDBC so that my postgres databases are available? 
  Any docs, hints, etc. would be appreciated.

 I'm not sure about ODBC, but for JDBC the general instructions are:

 1/ ensure that the CLASSPATH includes the driver jar (e.g.
 CLASSPATH=.:/usr/local/pgsql/share/postgresql.jar)

 2/ Check that postgres server has listen_address='*' in its
 configuration file (postgresql.conf)

 3/ Set the connection url for the database you want (e.g.
 jdbc:postgresql://localhost:5432/postgres to connect to a server using
 port 5432 and a database called postgres)

 Presumably openoffice lets you tell it 1/ and 3/, its up to you to make
 sure of 2/ !

 Cheers

 Mark

-- 

Brett I. Holcomb
-- 
gentoo-user@gentoo.org mailing list

[gentoo-user] Postgresql ODBC and JDBC in Gentoo

[Brett I. Holcomb]

I've installed postgres and am trying to get it to work with OpenOffice 2.0.3. 
After searching the web, postgres sites, unixODBC site, etc. I still haven't 
figured out how to get ODBC and JDBC to see postgres. How do I setup ODBC and 
JDBC so that my postgres databases are available?  Any docs, hints, etc. 
would be appreciated.


-- 

Brett I. Holcomb
-- 
gentoo-user@gentoo.org mailing list

[gentoo-user] postgresql

[pat]

HI all,

I've installed a postgres SQL and trying to connect into it. Does the
installation contains a testing DB ??? Is there a super user for the it
(somethink like sysdb, system etc. in oracle) and if yes what is its default 
passwd.

I've tryed google and documentation on the postgresql.org but without luck. The
DBs are not my cup of tea, but I need them :-|

Thanks

Pat
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] postgresql

[Bruno Lustosa]

On 5/12/06, pat [EMAIL PROTECTED] wrote:

I've installed a postgres SQL and trying to connect into it. Does the
installation contains a testing DB ??? Is there a super user for the it
(somethink like sysdb, system etc. in oracle) and if yes what is its default 
passwd.


The superuser for postgresql is 'postgres'. You can su to root, and
then 'su postgres' to connect to the database, as the postgres user
doesn't have a password by default.
You can use the template1 database to connect to the server, and then
create more databases. So:

$ su root
Password:
# su postgres
$ psql template1

Hope this helps

--
Bruno Lustosa [EMAIL PROTECTED]
http://www.lustosa.net/

--
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] postgresql

[pat]

On Fri, 12 May 2006 12:43:05 -0300, Bruno Lustosa wrote
 On 5/12/06, pat [EMAIL PROTECTED] wrote:
  I've installed a postgres SQL and trying to connect into it. Does the
  installation contains a testing DB ??? Is there a super user for the it
  (somethink like sysdb, system etc. in oracle) and if yes what is its
default passwd.
 
 The superuser for postgresql is 'postgres'. You can su to root, and
 then 'su postgres' to connect to the database, as the postgres user
 doesn't have a password by default.
 You can use the template1 database to connect to the server, and then
 create more databases. So:
 
 $ su root
 Password:
 # su postgres
 $ psql template1
 
 Hope this helps
 

Thanks, that helps.

 Pat
-- 
gentoo-user@gentoo.org mailing list