Re: [HACKERS] Detecting corrupted pages earlier
Kevin Brown <[EMAIL PROTECTED]> writes: > Shouldn't each variable listed in postgresql.conf.sample have comments > right above it explaining what it does anyway? Not really --- if you can't be bothered to consult the Admin Guide when in doubt, you have no business editing the config file. A word or two of hints is one thing, but circles and arrows and a paragraph on the back of each one just ain't gonna fly. We went down that path previously with pg_hba.conf, and finally realized that it was a waste of time to maintain what amounted to two separate sets of documentation. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Changing behavior of BEGIN...sleep...do something...COMMIT
Andreas, From the JDBC side it really doesn't make that much difference. The JDBC code needs to support both ways of doing it (explicit begin/commits for 7.2 and earlier servers, and set autocommit for 7.3 servers), so however it ends up for 7.4 it shouldn't be too much work to adopt. As Tom has mentioned elsewhere the key change is having the FE/BE protocol report the current transaction state. thanks, --Barry Zeugswetter Andreas SB SD wrote: Also, per other discussions, we are removing backend autocommit support in 7.4. It was the wrong way to do it. Somehow I did not see that conclusion made. I thought, at least for JDBC, it is already successfully used ? I think the backend autocommit is useful. Maybe only the installation/database/user wide GUC setting should be depricated/ disabled, so it is only used by a session SET ? Andreas ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Detecting corrupted pages earlier
Andrew Sullivan wrote: > On Thu, Apr 03, 2003 at 02:39:17PM -0500, Tom Lane wrote: > > just not listing zero_damaged_pages in postgresql.conf.sample? We > > already have several variables deliberately not listed there ... > > Hey, that might be a good solution. Of course, it doesn't solve the > "doomsday device" problem, but nobody who uses it can complain that > they didn't know what the thing would do. Shouldn't each variable listed in postgresql.conf.sample have comments right above it explaining what it does anyway? A self-documenting configuration file is a really handy thing to have. If it's documented that way in postgresql.conf.sample (and adjacent to the variable itself, even) then nobody who changed it would have grounds to complain about not knowing what the variable did. I'm much more in favor of being lucid and upfront about everything than hiding things just because they might be dangerous. That said, all sorts of warnings and such should be in that bit of documentation in postgresql.conf.sample, so that it's made abundantly clear that this particular option is not one to be messing with except when you know exactly what you're doing... -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] more contrib: log rotator
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Tom Lane writes: >> AFAICS, the only practical way to do this is to have a single process >> collecting the stdout/stderr from the postmaster and all its children. > I think not. It's a little tricky handling it directly in the child > processes, but it's been done before. A "little" tricky? Thanks, but no thanks ... for one thing, there'd be no easy way to know when all the children had switched over to writing the new file. Also, at least for not-too-long messages, writing on a single pipe gives atomicity guarantees that AFAIK do not exist when writing a file through multiple independently opened descriptors. In the latter case I think we'd have lots of trouble with interleaving of messages from different backends. >> If someone can offer a better alternative than Andrew's, great, let's >> see it. > How about the attached one, which I floated a while ago but which didn't > generate much interest. Seems like a good bare-bones file writer; but how about all those frammishes that people ask for like generating date-based filenames, switching every so many bytes, etc? Also, it'd be nice not to be dependent on a cron job to tickle the switchover. I do think there's an efficiency argument for having the log writer coded in C, so starting with what you have here and building up might be a better idea than starting with Andrew's perl script. But the important thing in my mind is to get something in there. We should also take a look at Apache's rotator to see if there's any need to reinvent the wheel at all. I have not seen it, am not even sure what it's written in... regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] more contrib: log rotator
Tom Lane writes:
> AFAICS, the only practical way to do this is to have a single process
> collecting the stdout/stderr from the postmaster and all its children.
I think not. It's a little tricky handling it directly in the child
processes, but it's been done before.
> If someone can offer a better alternative than Andrew's, great, let's
> see it.
How about the attached one, which I floated a while ago but which didn't
generate much interest.
--
Peter Eisentraut [EMAIL PROTECTED]
#include "c.h"
#include
#include
#include
#include
#include
#include
#include "pqsignal.h"
volatile static int hangup_flag = 0;
static void
signalhandler(SIGNAL_ARGS)
{
hangup_flag = 1;
}
#define BUF_SIZE 8192
#define MAX_ERRORS 200
#define MAX_ERRORS_CHECK() do { errcount++; if (max_errors > 0 && errcount >=
max_errors) exit(2); } while(0)
int
main(int argc, char *argv[])
{
const char *filename;
int fd = -1;
static char buf[BUF_SIZE];
unsigned int errcount = 0;
unsigned int max_errors = MAX_ERRORS;
if (argc != 2)
{
fprintf(stderr, "%s: missing required argument\n", argv[0]);
fprintf(stderr, "Try '%s --help' for more information.\n", argv[0]);
exit(1);
}
if (strcmp(argv[1], "--help")==0)
{
printf("this should be a help message...\n");
exit(0);
}
filename = argv[1];
fd = open(filename, O_WRONLY | O_CREAT | O_NOCTTY | O_APPEND, 0666);
if (fd < 0)
{
fprintf(stderr, "%s: could not open file %s: %s\n",
argv[0], filename, strerror(errno));
exit(1);
}
pqsignal(SIGUSR1, signalhandler);
for (;;)
{
int read_bytes;
int written_bytes;
char * buf_ptr;
read_bytes = read(0, buf, sizeof(buf));
if (read_bytes < 0)
{
if (errno == EINTR)
continue;
else
{
fprintf(stderr, "*** %s: read error from %s: %s\n",
argv[0], filename, strerror(errno));
MAX_ERRORS_CHECK();
}
}
if (read_bytes == 0)
{
/* end of file, postmaster exited? */
close(fd);
exit(0);
}
if (hangup_flag)
{
int fdnew;
fdnew = open(filename, O_WRONLY | O_CREAT | O_NOCTTY |
O_APPEND, 0666);
if (fdnew < 0)
{
fprintf(stderr, "*** %s: could not open new output
file %s: %s\n",
argv[0], filename, strerror(errno));
MAX_ERRORS_CHECK();
}
else
{
close(fd);
fd = fdnew;
}
hangup_flag = 0;
}
buf_ptr = buf;
do
{
written_bytes = write(fd, buf_ptr, read_bytes);
if (written_bytes < 0)
{
if (errno == EINTR)
continue;
fprintf(stderr, "*** %s: could not write to file %s:
%s\n",
argv[0], filename, strerror(errno));
MAX_ERRORS_CHECK();
break;
}
if (written_bytes < read_bytes)
{
buf_ptr += written_bytes;
read_bytes -= written_bytes;
continue;
}
break;
} while(1);
}
return 127;
}
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Stuff that doesn't work yet in IPv6 patch
Kurt Roeckx <[EMAIL PROTECTED]> writes: >> 2. SSL. Postmaster allows SSL for AF_INET but not AF_INET6. > Hmm, it really shouldn't matter if it uses AF_INET or AF_INET6 > ... I should look into that. Yeah, I suspect it just needs to replace the == AF_INET test with an isAF_INETx() test. But I don't have the facilities here to verify it. >> 4. pgstat code can only bind to 127.0.0.1 (v4 loopback). On a v6-only >> machine this would not exist, would it? > I'm not sure, but I think I changed something about that. Can > you point me to that code? src/backend/postmaster/pgstat.c regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Stuff that doesn't work yet in IPv6 patch
On Thu, Apr 03, 2003 at 04:47:45PM -0500, Tom Lane wrote: > The IPv6 patch seems to still be a few bricks shy of a load. Grepping > for places that handle AF_INET but not AF_INET6 revealed these > unimplemented features: > > 1. IDENT authorization. Fails if either local or remote address is IPv6. I've actually written this already. > 2. SSL. Postmaster allows SSL for AF_INET but not AF_INET6. Hmm, it really shouldn't matter if it uses AF_INET or AF_INET6 ... I should look into that. > 3. Client address display in backend's ps display seems to be v4 only. I'll take a look at that too. > 4. pgstat code can only bind to 127.0.0.1 (v4 loopback). On a v6-only >machine this would not exist, would it? I'm not sure, but I think I changed something about that. Can you point me to that code? Kurt ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Deadlock while doing VACUUM??
Kevin Brown <[EMAIL PROTECTED]> writes: > I finally got 7.3.2 installed, and confirmed that the problem does not > exist on that version. So this is something that's limited to the > 7.2.x tree. Which, I guess, means that it's not going to get fixed > for that tree (I assume that 7.2.x is effectively end-of-lifed)... I dug through the CVS logs, and could not find any entry between 7.2 and 7.3 that seemed like it might fix such a problem. If we knew what the problem was, maybe we could put together a back-patch. I'm not personally eager to spend more time on 7.2.*, though. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] more contrib: log rotator
On Fri, 4 Apr 2003, Tom Lane wrote:
> "scott.marlowe" <[EMAIL PROTECTED]> writes:
> > Hey, do you guys think that a setting of silent_mode = false might affect
> > no log files getting created?
>
> No, but setting it to true would be bad news.
That's what I'd meant actually. I had to turn of silent mode... You know
you're having a bad day when your email explaining how stupid you are is
factually incorrect. :-)
If anyone wants the diff, here it is:
22c22
< $CMDNAME start [-w] [-D DATADIR] [-s] [-l FILENAME] [-o \"OPTIONS\"]
---
> $CMDNAME start [-w] [-D DATADIR] [-s] [-r DURATION] [-l FILENAME]
[-o \"OPTIONS\"]
39a40,41
> -r DURATION invoke log rotation with DURATION seconds
> between rotation of files.
155a158,161
> -r)
> DURATION="$2"
> shift
> ;;
336c342,346
< "$po_path" ${1+"$@"} >$logfile 2>&1 &
---
> if [ -n "$DURATION" ]; then
> "$po_path" ${1+"$@"} &1| $PGPATH/rotatelogs
$logfile $DURATION 2>&1 &
> else
> "$po_path" ${1+"$@"} >$logfile 2>&1 &
> fi
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Deadlock while doing VACUUM??
Tom Lane wrote: > Kevin Brown <[EMAIL PROTECTED]> writes: > > When a heavy INSERT or UPDATE load on a table is occurring (lots of > > quick INSERTs or UPDATEs within a single transaction), a VACUUM > > ANALYZE (or just straight VACUUM) has a really good chance (10% or so) > > of causing either the INSERT/UPDATE or the VACUUM to fail with a > > "deadlock detected" error. > > I was unable to replicate this in CVS tip, using "pgbench -c 10 -t 1000" > as the load generator. I finally got 7.3.2 installed, and confirmed that the problem does not exist on that version. So this is something that's limited to the 7.2.x tree. Which, I guess, means that it's not going to get fixed for that tree (I assume that 7.2.x is effectively end-of-lifed)... On 7.3.2, a concurrent VACUUM appears to slow inserts down A LOT, but it won't deadlock them. -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] more contrib: log rotator
"scott.marlowe" <[EMAIL PROTECTED]> writes: > Hey, do you guys think that a setting of silent_mode = false might affect > no log files getting created? No, but setting it to true would be bad news. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] more contrib: log rotator
On Fri, 4 Apr 2003, Ed L. wrote: > On Friday April 4 2003 2:17, scott.marlowe wrote: > > > > OK, So I tried putting the 2>&1 before the | and all. No matter what I > > try, every from the | on is ignored. ps doesn't show it, and neither > > does pg_ctl status. Both show a command line of > > /usr/local/pgsql/bin/postmaster as the only input to start the server. > > Not clear if you're looking at it this way or if this is your problem, but > you can't really tell there is log rotation going on just by grepping ps > for postmaster because ps does not typically show the postmaster and the > rotatelogs together on the same line. I wouldn't expect pg_ctl status to > know anything at all about rotatelogs when you pipe it like this. Hey, do you guys think that a setting of silent_mode = false might affect no log files getting created? I had it right as soon as I added Tom's recommended 2>&1 but spent another 30 minutes figuring out why my log file wasn't getting created / filled. Thanks for the help. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] more contrib: log rotator
On Friday April 4 2003 2:17, scott.marlowe wrote: > > OK, So I tried putting the 2>&1 before the | and all. No matter what I > try, every from the | on is ignored. ps doesn't show it, and neither > does pg_ctl status. Both show a command line of > /usr/local/pgsql/bin/postmaster as the only input to start the server. Not clear if you're looking at it this way or if this is your problem, but you can't really tell there is log rotation going on just by grepping ps for postmaster because ps does not typically show the postmaster and the rotatelogs together on the same line. I wouldn't expect pg_ctl status to know anything at all about rotatelogs when you pipe it like this. Ed ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] more contrib: log rotator
On Fri, 4 Apr 2003, Ed L. wrote:
> On Friday April 4 2003 11:58, Tom Lane wrote:
> > "scott.marlowe" <[EMAIL PROTECTED]> writes:
> > > rotatelogs is in my path and all, it just never sees it.
> >
> > You mean the command fails? Or just that it doesn't capture output?
> >
> > > "$po_path" ${1+"$@"} > > 2>&1 &
> >
> > Most if not all of the postmaster's log output goes to stderr, so you'd
> > need
> >
> > "$po_path" ${1+"$@"} &1 | $PGPATH/rotatelogs ...
> >
> > to have any hope of useful results.
>
> Hmmm. I would have agreed 2>&1 was needed, too, but this command seems to
> routinely capture all output, including ERRORs:
>
> nohup pg_ctl start | nohup rotatelogs server_log.%a 86400
OK, So I tried putting the 2>&1 before the | and all. No matter what I
try, every from the | on is ignored. ps doesn't show it, and neither does
pg_ctl status. Both show a command line of
/usr/local/pgsql/bin/postmaster as the only input to start the server.
Now, the thing is, I've tried this with hardcoded values, like:
"$po_path" ${1+"$@"} &1 /usr/local/pgsql/bin/rotatelogs
/mnt/d1/data/logs/pglog 86400
where I know the logs directory exists. It works if I do:
pg_ctl start | rotatelogs $PGDATA/pglog 86400 2>1&
and puts the log files there.
I've copied rotatelogs into the /usr/local/pgsql/bin directory as well.
So, I'm thinking this is my weakness in shell scripting that's getting me
here, and that the shell is eating the |, not passing it out with the
postmaster to be used when it starts.
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [PERFORM] [HACKERS] OSS database needed for testing
Lamar, > I do still want to get CDBS in a PostgreSQL setup, with automatic nightly > import, at some point in time. Just probably not as quickly as Josh needs a > dataset to crank on. Oh, I don't know. I expect setting this up to take several weeks. And if we do the CDBS database as part of the test pack, then you can get my help without paying the $175/hour I normally charge . -- -Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] more contrib: log rotator
"Ed L." <[EMAIL PROTECTED]> writes: > Hmmm. I would have agreed 2>&1 was needed, too, but this command seems to > routinely capture all output, including ERRORs: > nohup pg_ctl start | nohup rotatelogs server_log.%a 86400 That's 'cause pg_ctl internally redirects the postmaster's stderr. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PERFORM] [HACKERS] OSS database needed for testing
On Friday 04 April 2003 14:54, Merlin Moncure wrote: > I can tell you, though; the land mobile database is much more > complicated. Getting it to run decently on pc hardware is a significant > engineering challenge. Then it sounds like it's a better fit for Josh's requirements. > ill-fated DTV rollout and the failed AM stereo. It's a conservative > industry. Tell me about it. Yet we get IBOCand the 30,000 translator apps in the one week window... Anyway, those topics more correctly belong to [EMAIL PROTECTED]; rather off-topic here. I do still want to get CDBS in a PostgreSQL setup, with automatic nightly import, at some point in time. Just probably not as quickly as Josh needs a dataset to crank on. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] more contrib: log rotator
On Fri, 4 Apr 2003, Tom Lane wrote: > "scott.marlowe" <[EMAIL PROTECTED]> writes: > > rotatelogs is in my path and all, it just never sees it. > > You mean the command fails? Or just that it doesn't capture output? The database starts, but rotatelogs doesn't get run. I.e. it's just like everything after the | symbol isn't there. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] [HACKERS] OSS database needed for testing
> I'm quite familiar with RadioSoft. Can't afford any of the software; > familiar > with the products... :-) > > I've been putting together open source tools to do much of the same stuff. > With the release of the FCC's Fortran source, I've been able to do > virtually > everything I need to do. > > But while the LMR dataset is larger, the MB dataset is just as varied. > I'm > interested in both, however. Peter's my father! At RadioSoft I worked over that same fortran code you talked about. We ported most of it to C. Of course, I would prefer not to write any software that would compete with my father's company, free or no :) I can tell you, though; the land mobile database is much more complicated. Getting it to run decently on pc hardware is a significant engineering challenge. OTOH, the broadcast database isn't too bad. For example, the last time I checked there were about 40k tv stations in the cdbs, compared with about 4m frequencies in the land mobile private. Also the lm database has a much wider scope of function. The requirements for broadcast have changed (like the nature of broadcast technology itself) very little in the last 50 years, except for the ill-fated DTV rollout and the failed AM stereo. It's a conservative industry. Merlin ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PERFORM] [HACKERS] OSS database needed for testing
On Friday 04 April 2003 14:23, Merlin Moncure wrote: > Up until about 6 months ago, I worked at a company called RadioSoft. > They are a provider of high quality database, engineering, and GIS > software. The company has its roots as source of engineering tools for > broadcast engineers. They currently offer several products and services > (including online web based database services), some of which are based > on postgres, some not. You might consider checking them out. I'm quite familiar with RadioSoft. Can't afford any of the software; familiar with the products... :-) I've been putting together open source tools to do much of the same stuff. With the release of the FCC's Fortran source, I've been able to do virtually everything I need to do. But while the LMR dataset is larger, the MB dataset is just as varied. I'm interested in both, however. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] [HACKERS] OSS database needed for testing
Lamar, > Also check out the cdbs files (which contain the broadcast stuff as well as > more) at /pub/Bureaus/Mass_Media/Databases/cdbs/ (which I would be more > interested in doing, since I am a broadcast engineer by profession) Hey, if you're willing to do the text --> postgres conversions, I'll use whichever tables you want ... -- -Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] [HACKERS] OSS database needed for testing
Lamar Owen wrote: > Also check out the cdbs files (which contain the broadcast stuff as well > as > more) at /pub/Bureaus/Mass_Media/Databases/cdbs/ (which I would be more > interested in doing, since I am a broadcast engineer by profession) > -- Up until about 6 months ago, I worked at a company called RadioSoft. They are a provider of high quality database, engineering, and GIS software. The company has its roots as source of engineering tools for broadcast engineers. They currently offer several products and services (including online web based database services), some of which are based on postgres, some not. You might consider checking them out. RadioSoft's flagship product, ComStudy, is the #1 tool for broadcast engineers on the market. I happen to be intimately familiar with the cdbs. I suggested the land mobile stuff because it is (much) bigger and (much) more complicated, but mostly draws on the same concepts, like haat, etc. You might get a kick out of this project. The last project I did before leaving there was an online database of directional patterns in xml format. I also made up a xml schema for directional patterns hoping to get some standardization in that regard. You can see that in the free section of the RadioSoft web page. Merlin ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Build error in cvs head
My build failed in interfaces/ecpg/compatlib because the Makefile references pgtypeslib instead of compatlib. I was wondering why nobody else got that problem. Maybe because they have an old version of informix.c lying around in pgtypeslib? Anyway, this patch should fix that problem. Servus Manfred diff -ruN ../base/src/interfaces/ecpg/compatlib/Makefile src/interfaces/ecpg/compatlib/Makefile --- ../base/src/interfaces/ecpg/compatlib/Makefile 2003-04-04 15:45:51.0 +0200 +++ src/interfaces/ecpg/compatlib/Makefile 2003-04-04 17:41:43.0 +0200 @@ -8,7 +8,7 @@ # #- -subdir = src/interfaces/ecpg/pgtypeslib +subdir = src/interfaces/ecpg/compatlib top_builddir = ../../../.. include $(top_builddir)/src/Makefile.global ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] more contrib: log rotator
On Friday April 4 2003 11:58, Tom Lane wrote:
> "scott.marlowe" <[EMAIL PROTECTED]> writes:
> > rotatelogs is in my path and all, it just never sees it.
>
> You mean the command fails? Or just that it doesn't capture output?
>
> > "$po_path" ${1+"$@"} > 2>&1 &
>
> Most if not all of the postmaster's log output goes to stderr, so you'd
> need
>
> "$po_path" ${1+"$@"} &1 | $PGPATH/rotatelogs ...
>
> to have any hope of useful results.
Hmmm. I would have agreed 2>&1 was needed, too, but this command seems to
routinely capture all output, including ERRORs:
nohup pg_ctl start | nohup rotatelogs server_log.%a 86400
Ed
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PERFORM] [HACKERS] OSS database needed for testing
On Friday 04 April 2003 11:47, Merlin Moncure wrote: > The location of the data of interest is at > /pub/Bureaus/Wireless/Databases/uls/. > wireless services. This includes most two way systems and point to > multipoint (microwave) but not broadcast (AM, FM, TV) and not advanced > radio. Also check out the cdbs files (which contain the broadcast stuff as well as more) at /pub/Bureaus/Mass_Media/Databases/cdbs/ (which I would be more interested in doing, since I am a broadcast engineer by profession) -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] more contrib: log rotator
"scott.marlowe" <[EMAIL PROTECTED]> writes:
> rotatelogs is in my path and all, it just never sees it.
You mean the command fails? Or just that it doesn't capture output?
> "$po_path" ${1+"$@"} &1 &
Most if not all of the postmaster's log output goes to stderr, so you'd need
"$po_path" ${1+"$@"} &1 | $PGPATH/rotatelogs ...
to have any hope of useful results.
regards, tom lane
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] more contrib: log rotator
On Fri, 4 Apr 2003, Ed L. wrote:
> On Friday April 4 2003 10:19, Tom Lane wrote:
> >
> > I feel we really ought to have *some* rotator included in the standard
> > distro, just so that the Admin Guide can point to a concrete solution
> > instead of having to arm-wave about what you can get off the net.
> > If someone can offer a better alternative than Andrew's, great, let's
> > see it.
>
> Out of curiosity, are there issues preventing inclusion of Apache's log
> rotation code? It seems you'd be hard-pressed to find a more
> battle-hardened log rotator.
>
> Obviously some people also wish to rotate based on log file size, so adding
> both to contrib at least seems sensible.
OK, I'm playing with the pg_ctl script that comes with 7.3, and trying to
make it startup with apaches rotatelog script, but this line won't pipe
output. I'm a total noob at bash shell scripting, so please feel free to
snicker when you answer.
rotatelogs is in my path and all, it just never sees it.
"$po_path" ${1+"$@"} &1 &
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] .sequence_name != ?
On Friday April 4 2003 10:24, Tom Lane wrote: > "Ed L." <[EMAIL PROTECTED]> writes: > > When a sequence is created in 7.3.2, it appears you get a new table for > > each sequence object. Is it ever possible for the sequence_name in a > > sequence relation not to match the name of the relation itself? > > In general I'd counsel that you should ignore the sequence_name field > anyway. It's vestigial. A related question: Is there a single generalized SQL query which can yield the set of (sequence_name, last_value) pairs for all sequence objects? The fact that each sequence is its own relation seems to block that, and the query constructed from grabbing sequence names from pg_class gets quite long for more than just a few sequence objects... Ed ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] more contrib: log rotator
On Friday April 4 2003 10:19, Tom Lane wrote: > > I feel we really ought to have *some* rotator included in the standard > distro, just so that the Admin Guide can point to a concrete solution > instead of having to arm-wave about what you can get off the net. > If someone can offer a better alternative than Andrew's, great, let's > see it. Out of curiosity, are there issues preventing inclusion of Apache's log rotation code? It seems you'd be hard-pressed to find a more battle-hardened log rotator. Obviously some people also wish to rotate based on log file size, so adding both to contrib at least seems sensible. Ed ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] [HACKERS] OSS database needed for testing
Josh Berkus wrote: > Cool. I'll tackle this in a week or two. Right now, I'm being paid to > convert a client's data and that'll keep me busy through the weekend ... I would suggest downloading the data now. I can help get you started with the create table statements and the import scripts. There are not very many ways to get the data in a reasonable timeframe: the spi functions or the copy command are a good place to start. Do not bother with running stuff through insert queries: take my word for it, it just won't work. Of course, if you use copy, you have to pre-format. Be aware that you will have many gigabytes (like more than 20) of data before you are done. Whatever you decide to do, document the process: the difficulty of getting large amounts of data into postgres quickly and easily has been a historical complaint of mine. Using mysql, it was a snap to get the data in but using *that* database I really felt it couldn't handle this much data. I can also get you started with some example queries that should be quite a challenge to set up to run quickly. After that, it's your ballgame. Merlin ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] [HACKERS] OSS database needed for testing
Merlin, > The fcc FTP site is ftp.fcc.gov > > The location of the data of interest is at > /pub/Bureaus/Wireless/Databases/uls/. Cool. I'll tackle this in a week or two. Right now, I'm being paid to convert a client's data and that'll keep me busy through the weekend ... -- -Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] .sequence_name != ?
"Ed L." <[EMAIL PROTECTED]> writes: > When a sequence is created in 7.3.2, it appears you get a new table for each > sequence object. Is it ever possible for the sequence_name in a sequence > relation not to match the name of the relation itself? ALTER TABLE RENAME on a sequence doesn't update the sequence_name. I think someone looked at doing that update, but we concluded it was too messy (mainly because ALTER RENAME is transactional but updates to a sequence tuple aren't). In general I'd counsel that you should ignore the sequence_name field anyway. It's vestigial. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] more contrib: log rotator
Peter Eisentraut <[EMAIL PROTECTED]> writes: > What would get me a whole lot more excited is if the server could write > directly to a file and do its own rotating (or at least reopening of > files). AFAICS, the only practical way to do this is to have a single process collecting the stdout/stderr from the postmaster and all its children. pglog-rotator is one implementation of that approach. I too would rather this functionality were integrated into the server, but I haven't noticed anyone stepping up to the plate to do it. > Considering that your rotator is tailored to a rather specific setup, it > doesn't do anything better compared to established ones, it prevents the > use of pg_ctl, it's written in Perl, and it doesn't do anything for > Windows users, I think it's not suitable for a general audience. These might be good arguments for not putting it into the mainstream, but I don't think they have any force if we consider it for contrib. I feel we really ought to have *some* rotator included in the standard distro, just so that the Admin Guide can point to a concrete solution instead of having to arm-wave about what you can get off the net. If someone can offer a better alternative than Andrew's, great, let's see it. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] more contrib: log rotator
On Fri, 4 Apr 2003, Andrew Sullivan wrote: > On Fri, Apr 04, 2003 at 09:16:39AM -0700, scott.marlowe wrote: > > where -r is the rotation period in seconds. If it's an external program > > Ours rotates based on size rather than time. I can see some > advantages to the time-based approach, but if you have wide > variations in traffic, you run the risk of rotating over useful files > with more or less empty ones if you use it. I would want time based for sure, and I can see the use for size based splitting as well. I wouldn't be hard to have it do both would it? I just like the idea of it being one of the dozens or so options for pg_ctl so it's painless to use for joe six pack. pg_ctl -r 86400 -l $PGDATA/logs/pgsql where -r is the rotation period OR pg_ctl -f 10M -l $PGDATA/logs/pgsql where -f is the max file size of a log I'd recommend that the nameing convnention should probably be: filenamespec.timestamp, like: $PGDATA/logs/pgsql.1049414400 for time rotated logs, and filename.incnumber like: $PGDATa/logs/pgsql.01 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] more contrib: log rotator
Peter Eisentraut wrote: > > Andrew Sullivan writes: > > > Is anyone interested in having pglog-rotator? > > What would get me a whole lot more excited is if the server could write > directly to a file and do its own rotating (or at least reopening of > files). >From a technical point of view I don't think that is desirable. The entire log traffic would have to be routed through the postmaster, as it is in LibertyRMS's log rotator now through the perl script. And we really try to keep everything outside the postmaster that does not absolutely have to be in there for stability reasons. We can discuss if the log rotator should be a child process of the postmaster or the other way round, but that will not change the flow of bytes between the processes in any way. I would say it's better the way it is, because it does not pollute the postmasters wait logic with another exception. My ideal solution would be to integrate the log rotators functionality into a C version of pg_ctl that forks and detaches from the control terminal in the way, daemons should. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] more contrib: log rotator
On Friday April 4 2003 10:04, Ed L. wrote: > By way of feature ideas, one very convenient but not widely used feature > of Apache's log rotator is the ability to specify a strftime() format > string for the file extension. For example, if I want to have my logs > rollover every 24 hours and be named log.Mon, log.Tue, log.Wed, I say > something like > > pg_ctl start | rotatelogs 86400 "%a" More accurately, something like this: pg_ctl start | rotatelogs 86400 "log.%a" Ed ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] more contrib: log rotator
On Friday April 4 2003 9:16, scott.marlowe wrote: > > That said, a log rotation capability built right into pg_ctl or > thereabouts would be a very nice feature. I.e. 'pg_ctl -r 86400 -l > $PGDATA/logs/pgsql start' > > where -r is the rotation period in seconds. If it's an external program > that pg_ctl calls that's fine, and it could even just be a carbon copy of > apache's log rotater if their license is compatible (isn't it?) By way of feature ideas, one very convenient but not widely used feature of Apache's log rotator is the ability to specify a strftime() format string for the file extension. For example, if I want to have my logs rollover every 24 hours and be named log.Mon, log.Tue, log.Wed, I say something like pg_ctl start | rotatelogs 86400 "%a" This causes the logs to overwrite themselves every seven days, taking log maintenance time to very near zero. We also customized our use of it to allow us to automatically move existing logs out of the way to "log.1", "log.2", or to simply overwrite existing logs. Ed ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
[HACKERS] .sequence_name != ?
When a sequence is created in 7.3.2, it appears you get a new table for each sequence object. Is it ever possible for the sequence_name in a sequence relation not to match the name of the relation itself? For example, suppose I create a table: CREATE TABLE t1(id serial); A new relation called 't1_id_seq' is created where t1_id_seq.sequence_name = 't1_id_seq' Is that always true? Ed ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] [HACKERS] OSS database needed for testing
The fcc FTP site is ftp.fcc.gov The location of the data of interest is at /pub/Bureaus/Wireless/Databases/uls/. There are zip files (pipe delimited) in complete and the daily changed files in daily. Theres lots of info in documentation which includes excel spreadsheets of the schema. These will have to be converted to sql statemtents. The ULS is the database system that holds the data for Fixed and Mobile wireless services. This includes most two way systems and point to multipoint (microwave) but not broadcast (AM, FM, TV) and not advanced radio. The database is really a database of applications. It contains application data submitted by wireless applicants. There are two families of tables, prefixed with 'a' and 'l'. The 'a' tables stand for application records that are pending being granted by the fcc. The 'l' tables have received licenses and may or may not be operating. Combined, the 'a' and 'l' zipfiles represent a specific service. For example, 'a_micro' and 'l_micro' contain the applications and licensed data for microwave systems. The different services have slightly different layouts because they have different requirements. I strongly suggest looking at LMcomm and LMpriv first. These are the fixed land mobile systems, and 90% of the entire database. They also have identical layouts. There are a great deal of files in each zipfile, but here are the most interesting: hd: header data ad: application detail an: antenna data lo: location data fr: frequency data em: emission data There are others. I can help you write meaningful queries that are quite complex and will require optimization techniques. Merlin ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] more contrib: log rotator
On Fri, Apr 04, 2003 at 05:13:13PM +0200, Peter Eisentraut wrote: > use of pg_ctl, it's written in Perl, and it doesn't do anything for > Windows users, I think it's not suitable for a general audience. It doesn't prevent the use of pg_ctl, although it does indeed prevent the use of pg_ctl for startup. I'm not sufficiently familiar with Windows to know how this does or does not help them. Could you elaborate? And what's wrong with Perl? A -- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada <[EMAIL PROTECTED]> M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] more contrib: log rotator
On Fri, Apr 04, 2003 at 09:16:39AM -0700, scott.marlowe wrote: > where -r is the rotation period in seconds. If it's an external program Ours rotates based on size rather than time. I can see some advantages to the time-based approach, but if you have wide variations in traffic, you run the risk of rotating over useful files with more or less empty ones if you use it. A -- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada <[EMAIL PROTECTED]> M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] more contrib: log rotator
On Fri, 4 Apr 2003, Peter Eisentraut wrote: > Andrew Sullivan writes: > > > Is anyone interested in having pglog-rotator? > > What would get me a whole lot more excited is if the server could write > directly to a file and do its own rotating (or at least reopening of > files). > > Considering that your rotator is tailored to a rather specific setup, it > doesn't do anything better compared to established ones, it prevents the > use of pg_ctl, it's written in Perl, and it doesn't do anything for > Windows users, I think it's not suitable for a general audience. That said, a log rotation capability built right into pg_ctl or thereabouts would be a very nice feature. I.e. 'pg_ctl -r 86400 -l $PGDATA/logs/pgsql start' where -r is the rotation period in seconds. If it's an external program that pg_ctl calls that's fine, and it could even just be a carbon copy of apache's log rotater if their license is compatible (isn't it?) ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PERFORM] [HACKERS] OSS database needed for testing
Jeff, Mlw, > Absolutely. We could just use one large state or several small ones and > let folks download the whole thing if they wanted. Using that technique > you could control the size of the test quite closely and still make > something potentially quite valuable as a contribution beyond the bench. Hold on a second. The FCC database is still a better choice because it is more complex with a carefully defined schema. The Tiger database would be good for doing tests of type 1 and 3, but not for tests of types 2 and 4. It would certainly be interesting to use the Tiger database as the basis for an additional type of test: 6) Very Large Data Set: querying, then updating, 300+ selected rows from a 2,000,000 + row table. ... but I still see the FCC database as our best candidate for the battery of tests 1-5. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] more contrib: log rotator
Andrew Sullivan writes: > Is anyone interested in having pglog-rotator? What would get me a whole lot more excited is if the server could write directly to a file and do its own rotating (or at least reopening of files). Considering that your rotator is tailored to a rather specific setup, it doesn't do anything better compared to established ones, it prevents the use of pg_ctl, it's written in Perl, and it doesn't do anything for Windows users, I think it's not suitable for a general audience. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Changing behavior of BEGIN...sleep...do something...COMMIT
"Zeugswetter Andreas SB SD" <[EMAIL PROTECTED]> writes: > Somehow I did not see that conclusion made. > I thought, at least for JDBC, it is already successfully used ? Barry, at least, seemed to be happy with removing it, given the planned protocol change to report current transaction state after every query. > I think the backend autocommit is useful. Maybe only the > installation/database/user wide GUC setting should be depricated/ > disabled, so it is only used by a session SET ? That wouldn't really solve any of the problems. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Changing behavior of BEGIN...sleep...do something...COMMIT
> Also, per other discussions, we are removing backend autocommit support > in 7.4. It was the wrong way to do it. Somehow I did not see that conclusion made. I thought, at least for JDBC, it is already successfully used ? I think the backend autocommit is useful. Maybe only the installation/database/user wide GUC setting should be depricated/ disabled, so it is only used by a session SET ? Andreas ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] [OT][ANNOUNCEMENT]Announcing first public release of Open Application Server
Hello all, I am very glad to announce first public release of Open Application Server. This is an application framework built in C++, to make use of existing APIs in internet application. It provides * A thread based request delivery architecture * support of request handlers loaded from external libraries * Support of additional application API such as http, SMTP etc. The features: * Written in C++ * mutlithreaded application * High performance and scalable * Provides object-packing technology * Native interface with apache to act as web application server The project is available from http://oasserver.sourceforge.net. The mailing list is not active as yet but it should be up in short time(by tomorrow hopefully). A complete web application built with OAS+postgresql+apache is also available from CVS. This is a issue tracking and resource booking system. There are no packages/tarballs available right now. Please use anonymous cvs. I plan to relase packages in short time. The CVS modules are oasserver and phd respectively. This is done so that I can update install documentation to cater for variety of build platforms. Right now, I can test the build only on slackware/mandrake/freeBSD. Shridhar ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
