Re: [HACKERS] Subtle pg_dump problem...
Did you guys find any solution to this in the end? Chris Oleg Bartunov wrote: Thanks Christopher, we'll look into the issue. Oleg On Fri, 7 May 2004, Christopher Kings-Lynne wrote: I have a table with a tsearch2 index on it. Now, I have all the tsearch2 stuff installed into a 'contrib' schema. I have had to change the default database schema to include the contrib schema as behind-the-scenes, tsearch2 looks for its tables, and cannot find them even if the function itself is schema-qualfified. This might well be a tsearc2 bug. Anyway, this means the table is dumped like this: SET SESSION AUTHORIZATION 'auadmin'; SET search_path = public, pg_catalog; COPY ... Which give this error upon restoring: ERROR: relation pg_ts_cfg does not exist CONTEXT: COPY food_categories, line 1: 79 102 Vegetables, Salads Legumes\N 'legum':3 'salad':2 'veget':1 It's because the search_path needs to be like this for it to work: SET search_path = public, contrib, pg_catalog; Chris ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] XLog: how to log?
On Wed, 2004-05-12 at 04:47, Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I think this argument is largely a red herring ... but if it makes you feel better, we could change the contents of the commit timestamp to be gettimeofday() output (seconds+microseconds) instead of just time() output. That should be precise enough for practical purposes. I am saying timestamp as used for specifying a recovery location might not be unique enough, no? Why not? I don't think there are going to be practical situations where the user knows that he wants transactions up till exactly 6:48:52 PM anyway. He'll be lucky if he knows that the junior DBA dropped the wrong table around 6:30 :-(. It's even less likely that he desperately needs to revert to before such a disaster but still get in a transaction that happened to commit at the same second. Take it down to the microsecond level and the use-case becomes vanishingly small. Here is my logic. Once they have a way to dump the WAL contents, folks trying to recover to a specific point in the past are going to look at the WAL dump and hopefully identify the transaction that was bad. They then will want to roll back to just before that transaction. Do they subtract one second from the transaction? Seems it would be easier to just pick the xid that was just before the bad one. Also, considering the various time formats and timezone issues that it is simpler to just have them specify an xid. Well, I think I agree with both sides of this debate. Solution: provide both timestamp AND Xid capability. We assume that if they specify Xid, it is because they know and, for whatever reason, care, about the exact specification of where recovery stops. If you know a large statement just executed in error, then you want to restore back to just before the error. My earlier angst was based upon mistaking that there was no timestamp. There is now a simple choice of recovery targets and fairly simple to implement both. Design is now clear for me. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Probably security hole in postgresql-7.4.1
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Should we be thinking about a 7.4.3? I'm not panicking over this particular bug ... but it does seem like we've accumulated enough fixes since 7.4.2 that it may be time to start thinking about another dot-release. Maybe set a date towards the end of the month? regards, tom lane Industry practices dictate that we do issue SOMETHING now. The bug is now public, and can be exploited. This does not necessarily have to be 7.4.3. We can issue 7.4.2.1, containing only this fix, so that people who need to expose their database are not left open to attacks. Also, if we want greater flexibility in handling these cases in the future, we should set up an invite-only list for reporting security bugs, and advertise it on the web site as the place to report security issues. Had this vulnerability been reported there, we could reasonably hold on without releasing a fix until 7.4.3 was ready. If you need help in that list, I have a lot of experience with code security, but very little experience with the Postgresql code. Also, it would be a good idea to invite all the distro-packagers to be on that list. Shachar -- Shachar Shemesh Lingnu Open Source Consulting http://www.lingnu.com/ ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Module dependency on PostgeSQL version
This is what I currently have: #if (CATALOG_VERSION_NO = 200211021) #define PG_VERSION_73_COMPAT #elif (CATALOG_VERSION_NO = 200310211) #define PG_VERSION_74_COMPAT #else #define PG_VERSION_75_COMPAT #endif Since CATALOG_VERSION_NO doesn't change between major releases, it seems to work pretty well. Joe Ok, thanks for the tip. Somewhat cleaner than what I suggested. Your solution doesn't cover the situations when you have dependencies on patch versions though, does it? There's no way to distinguish between 7.4.1 and 7.4.2 if the catalog version doesn't change. Such situations will arise. A module author finds a bug in the backend, reports it, and creates a less than perfekt work-around in his own code. At some point the bug gets properly fixed and included in some patch release. Using that release, the work-around is obsolete. I had an example when this happened a while back (concerning loss of MemoryContext between function ivocations). My workaround was really ugly (using the parent MemoryContext) and could potentially waste a lot of memory so it was imperative to actually get rid of it as soon as the patch was released. regards, Thomas Hallgren ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Module dependency on PostgeSQL version
Greg Sabino Mullane [EMAIL PROTECTED] wrote in message
DBD::Pg uses pg_config --version and parses the output to set the
version information before compiling. It finds pg_config by using
PG envrironment variables, or looks in the path. Once running, it
does a SELECT version() to find out what server it is running
against. Whenever possible, is now uses the new protocol version
information. It's much handier to just say:
if (protocol=3) {
...
}
Great! The SELECT version() approach has the advantage of having an
already compiled version adapt to different versions of the backend. I'll
try and use that to overcome differences in patch levels.
Thanks,
Thomas Hallgren
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] XLog: how to log?
I think this argument is largely a red herring ... but if it makes you feel better, we could change the contents of the commit timestamp to be gettimeofday() output (seconds+microseconds) instead of just time() output. That should be precise enough for practical purposes. I am saying timestamp as used for specifying a recovery location might not be unique enough, no? Maybe the api should allow a xid only in addition to a timestamp, for use when timestamp alone is not precise enough. That would solve the problem of not finding the xid. I would stay away from microseconds. Andreas ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Subtle pg_dump problem...
On Wed, 12 May 2004, Christopher Kings-Lynne wrote: Did you guys find any solution to this in the end? Not yet. Could you send me a simple test suite ? I'm not experienced with schema and don't know how to load tsearch2 into specific schema. Oleg Chris Oleg Bartunov wrote: Thanks Christopher, we'll look into the issue. Oleg On Fri, 7 May 2004, Christopher Kings-Lynne wrote: I have a table with a tsearch2 index on it. Now, I have all the tsearch2 stuff installed into a 'contrib' schema. I have had to change the default database schema to include the contrib schema as behind-the-scenes, tsearch2 looks for its tables, and cannot find them even if the function itself is schema-qualfified. This might well be a tsearc2 bug. Anyway, this means the table is dumped like this: SET SESSION AUTHORIZATION 'auadmin'; SET search_path = public, pg_catalog; COPY ... Which give this error upon restoring: ERROR: relation pg_ts_cfg does not exist CONTEXT: COPY food_categories, line 1: 79 102 Vegetables, Salads Legumes\N 'legum':3 'salad':2 'veget':1 It's because the search_path needs to be like this for it to work: SET search_path = public, contrib, pg_catalog; Chris ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Subtle pg_dump problem...
Christopher, I don't quite understand the problem Did you move pg_ts_* table to schema 'contrib' ? What functions you schema-qualified and how ? It's always required to set search_path properly. Oleg On Wed, 12 May 2004, Christopher Kings-Lynne wrote: Did you guys find any solution to this in the end? Chris Oleg Bartunov wrote: Thanks Christopher, we'll look into the issue. Oleg On Fri, 7 May 2004, Christopher Kings-Lynne wrote: I have a table with a tsearch2 index on it. Now, I have all the tsearch2 stuff installed into a 'contrib' schema. I have had to change the default database schema to include the contrib schema as behind-the-scenes, tsearch2 looks for its tables, and cannot find them even if the function itself is schema-qualfified. This might well be a tsearc2 bug. Anyway, this means the table is dumped like this: SET SESSION AUTHORIZATION 'auadmin'; SET search_path = public, pg_catalog; COPY ... Which give this error upon restoring: ERROR: relation pg_ts_cfg does not exist CONTEXT: COPY food_categories, line 1: 79 102 Vegetables, Salads Legumes\N 'legum':3 'salad':2 'veget':1 It's because the search_path needs to be like this for it to work: SET search_path = public, contrib, pg_catalog; Chris ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Configure redux.
[EMAIL PROTECTED] writes: They say the absolute path is a security issue, I honestly don't see how Because it allows someone with only CREATEDB privilege to tell the backend to write anywhere (that it can write). That's an ability that should be reserved to superusers. The environment-variable-based variant is actually not a lot better. Consider CREATEDB WITH LOCATION 'HOME'. Or 'PWD'. Depending on your platform, env | grep =/ may reveal other interesting possibilities. The problem is that a postmaster's environment will contain a lot of strings that happen to look like absolute paths. There is no way for the code to know which ones were really intended by the DBA to be used as locations, and which ones are just part of the standard environment on a particular platform. Either way you slice it, WITH LOCATION is fundamentally bogus because it allows users to create databases in unintended locations. I don't think it can be used to overwrite existing files, so it's not a security hole in the sense of allowing attacks, but it is a security hole in the sense of not giving the DBA a reliable way to control database locations. I have every intention of ripping that code out, root and branch, as soon as we have a real tablespace feature (which I'm expecting will provide means within the system for defining allowed physical locations). I don't recommend spending any time on improving the feature, because its days are numbered. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Module dependency on PostgeSQL version
Joe Conway wrote: #if (CATALOG_VERSION_NO = 200211021) #define PG_VERSION_73_COMPAT Since CATALOG_VERSION_NO doesn't change between major releases, it seems to work pretty well. That approach also has the nice property that it already works for all releases back to 7.0, whereas anything we add now would only be useful starting in 7.5 ... Thomas Hallgren [EMAIL PROTECTED] writes: Your solution doesn't cover the situations when you have dependencies on patch versions though, does it? There's no way to distinguish between 7.4.1 and 7.4.2 if the catalog version doesn't change. Perhaps not, but I think it would be pretty risky to depend on an #ifdef to tell you which backend minor version you're running in anyway. Addon libraries are unlikely to get upgraded at the same times that the backend does, so it's certainly plausible that you could be running in a later minor release than you were compiled with. If people copy executables between machines then the reverse is easily possible too. The only real version tiedown we have at the moment is to put extension libraries into a PG-version-specific $libdir, and that is not going to narrow things more closely than the PG major version in most setups. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Subtle pg_dump problem...
OK, I'll try to explain it better. 1. Tsearch2 requires access to several tables. 2. You can edit the tsearch2.sql script and change the set schema = ... to contrib. 3. You load all the tsearch2 objects into contrib. 4. You create a table in the public schema with a column of type contrib.vector, and a trigger of contrib.tsearch2. 5. You pg_dump that table, you get: SET search_path = public, pg_catalog; COPY ... (Because the table is in the public schema) 6. However, it is now not possible to restore the sql script as it was dumped, as you get this error: ERROR: relation pg_ts_cfg does not exist 7. You get this error because the tsearch2 code depends on the current search path, and since contrib is not in the search path, the restore fails. 8. This problem occurs because tsearch2 is dependent on the current user's search_path. Instead, it should be independent of the current user's search path, and instead try to find its configuration tables in the same schema in which the vector type or the tsearch2 trigger function resides. This assumes that the user has installed all the tsearch2 objects into the same schema, which I think is reasonable. This problem will occur for anyone who has multiple schemas and tries to create vector columns in tables that refer to the vector type in another schema. Does that make sense? Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Configure redux.
[EMAIL PROTECTED] writes: They say the absolute path is a security issue, I honestly don't see how Because it allows someone with only CREATEDB privilege to tell the backend to write anywhere (that it can write). That's an ability that should be reserved to superusers. I guess I can see that. The environment-variable-based variant is actually not a lot better. Consider CREATEDB WITH LOCATION 'HOME'. Or 'PWD'. Depending on your platform, env | grep =/ may reveal other interesting possibilities. The problem is that a postmaster's environment will contain a lot of strings that happen to look like absolute paths. There is no way for the code to know which ones were really intended by the DBA to be used as locations, and which ones are just part of the standard environment on a particular platform. Either way you slice it, WITH LOCATION is fundamentally bogus because it allows users to create databases in unintended locations. I don't think it can be used to overwrite existing files, so it's not a security hole in the sense of allowing attacks, but it is a security hole in the sense of not giving the DBA a reliable way to control database locations. I have every intention of ripping that code out, root and branch, as soon as we have a real tablespace feature (which I'm expecting will provide means within the system for defining allowed physical locations). I don't recommend spending any time on improving the feature, because its days are numbered. Well, is there a doc on how tables spaces will be implemented? Do you intend to have postmaster initialize the tablespace, or will the admin be required to perform an initlocation? I already have a keyword in my config patch called 'volume' but 'tablespace' would probably be a better name for it. A simple name change would work. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL pre-fork speedup
I ran the new Pgpool-1.2.2 and it was a bit faster on the TCP but still slower than on UNIX socket. I used the same script as before. TCP Socket (Pgpool 1.2.0) -- 2.39 sec TCP Socket (Pgpool 1.2.2) -- 0.80 sec 0.80 sec 0.79 sec UNIX Socket (Pgpool 1.2.2) --- 0.026 sec 0.027 sec 0.027 sec Direct TCP connection (no pgpool) - 0.16 sec 0.15 sec 0.16 sec PgPool on TCP is still slower than direct connection but much faster than v1.2. Any other areas that can be improved? Regards, __ Do you Yahoo!? Yahoo! Movies - Buy advance tickets for 'Shrek 2' http://movies.yahoo.com/showtimes/movie?mid=1808405861 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Subtle pg_dump problem...
Christopher, On Wed, 12 May 2004, Christopher Kings-Lynne wrote: OK, I'll try to explain it better. 1. Tsearch2 requires access to several tables. 2. You can edit the tsearch2.sql script and change the set schema = ... to contrib. Aha, this is what I thought about. 3. You load all the tsearch2 objects into contrib. createdb qq psql qq -c create schema contrib psql qq tsearch2_contrib.sql 4. You create a table in the public schema with a column of type contrib.vector, and a trigger of contrib.tsearch2. qq=# create table test ( a text, fts contrib.tsvector); CREATE TABLE 5. You pg_dump that table, you get: SET search_path = public, pg_catalog; COPY ... (Because the table is in the public schema) done. 6. However, it is now not possible to restore the sql script as it was dumped, as you get this error: ERROR: relation pg_ts_cfg does not exist No problem, [EMAIL PROTECTED]:~/app/pgsql/tsearch2/test_scheme$ createdb qq CREATE DATABASE [EMAIL PROTECTED]:~/app/pgsql/tsearch2/test_scheme$ psql qq -c create schema contrib CREATE SCHEMA psql qq ./tsearch2_contrib.sql psql qq ./test.dump SET SET SET SET CREATE TABLE But I get error later: qq=# insert into test(a) values( 'the hot dog'); INSERT 3478544 1 qq=# update test set fts=contrib.to_tsvector(a); ERROR: relation pg_ts_cfg does not exist after setting proper search_path it worked: qq=# set search_path to public,contrib; SET qq=# update test set fts=contrib.to_tsvector(a); UPDATE 1 7. You get this error because the tsearch2 code depends on the current search path, and since contrib is not in the search path, the restore fails. 8. This problem occurs because tsearch2 is dependent on the current user's search_path. Instead, it should be independent of the current user's search path, and instead try to find its configuration tables in the same schema in which the vector type or the tsearch2 trigger function resides. This assumes that the user has installed all the tsearch2 objects into the same schema, which I think is reasonable. This problem will occur for anyone who has multiple schemas and tries to create vector columns in tables that refer to the vector type in another schema. Does that make sense? Sorry, I don't see the problem. I just pg_dump whole db and recreated without any problem. For working with tsearch2 I should set correct search_path, but what's wrong with this ? pg_dump qq qq.dump dropdb qq createdb qq psql qq qq.dump qq=# set search_path to public,contrib; SET qq=# update test set fts=contrib.to_tsvector(a); UPDATE 1 works like a charm :) One remark: I applied regprocedure_7.4.patch.gz to be able dump/restore without issue with OIDs. Upgrading existed tsearch2 installation should be easy ( thanks Andrew for his script ): Actually, for playing with schema I added set search_path = contrib; to his script. Original script is available from http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/regprocedure_update.sql psql qq regprocedure_update.sql ( first line is set search_path = contrib;) now, database qq could be dumped/restored without problem. Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Subtle pg_dump problem...
6. However, it is now not possible to restore the sql script as it was dumped, as you get this error: ERROR: relation pg_ts_cfg does not exist No problem, [EMAIL PROTECTED]:~/app/pgsql/tsearch2/test_scheme$ createdb qq CREATE DATABASE [EMAIL PROTECTED]:~/app/pgsql/tsearch2/test_scheme$ psql qq -c create schema contrib CREATE SCHEMA psql qq ./tsearch2_contrib.sql psql qq ./test.dump SET SET SET SET CREATE TABLE Is that because you didn't insert any data into the table before dumping it? You will get the same error that follows: But I get error later: qq=# insert into test(a) values( 'the hot dog'); INSERT 3478544 1 qq=# update test set fts=contrib.to_tsvector(a); ERROR: relation pg_ts_cfg does not exist after setting proper search_path it worked: qq=# set search_path to public,contrib; SET qq=# update test set fts=contrib.to_tsvector(a); UPDATE 1 My point is that if you pg_dump a table that has data in it, pg_dump will set yoru search_path for you, and so the restore will fail. pg_dump qq qq.dump dropdb qq createdb qq psql qq qq.dump qq=# set search_path to public,contrib; SET qq=# update test set fts=contrib.to_tsvector(a); UPDATE 1 works like a charm :) I bet you don't have any data in the table. Chris ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] threads stuff/UnixWare
At the risk of getting my butt kicked again, is there any way we can talk about how to deal with threads on UnixWare and the libpq stuff? Has any other platform come up with a need to look for the real pthread_* calls from libpq? I would REALLY like us to support threaded programs in UnixWare, and would like to NOT antagonize the community, but would like to have a direction I can go to make this happen in 7.5 before we freeze. Thanks, LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Subtle pg_dump problem...
Christopher,
here is a cut'n paste from test script (patch applied):
dropdb qq
createdb qq
psql qq -c create schema contrib;
psql qq tsearch2_contrib-2.sql
psql qq -c create table test ( a text, fts contrib.tsvector);
psql qq -c insert into test(a) values ('I hit a dog');
psql qq -c set search_path = public,contrib; update test set fts = to_tsvector(a);
pg_dump qq qq.dump
There's certainly one record and after restoring I could use tsearch2
as usual (of course, setting search_path properly).
Oleg
On Wed, 12 May 2004, Christopher Kings-Lynne wrote:
6. However, it is now not possible to restore the sql script as it was
dumped, as you get this error:
ERROR: relation pg_ts_cfg does not exist
No problem,
[EMAIL PROTECTED]:~/app/pgsql/tsearch2/test_scheme$ createdb qq
CREATE DATABASE
[EMAIL PROTECTED]:~/app/pgsql/tsearch2/test_scheme$ psql qq -c create schema
contrib
CREATE SCHEMA
psql qq ./tsearch2_contrib.sql
psql qq ./test.dump
SET
SET
SET
SET
CREATE TABLE
Is that because you didn't insert any data into the table before dumping
it? You will get the same error that follows:
But I get error later:
qq=# insert into test(a) values( 'the hot dog');
INSERT 3478544 1
qq=# update test set fts=contrib.to_tsvector(a);
ERROR: relation pg_ts_cfg does not exist
after setting proper search_path it worked:
qq=# set search_path to public,contrib;
SET
qq=# update test set fts=contrib.to_tsvector(a);
UPDATE 1
My point is that if you pg_dump a table that has data in it, pg_dump
will set yoru search_path for you, and so the restore will fail.
pg_dump qq qq.dump
dropdb qq
createdb qq
psql qq qq.dump
qq=# set search_path to public,contrib;
SET
qq=# update test set fts=contrib.to_tsvector(a);
UPDATE 1
works like a charm :)
I bet you don't have any data in the table.
Chris
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Regards,
Oleg
_
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Subtle pg_dump problem...
No problem, Actually, I did some more testing and I properly understand the problem now - and it won't happen in the general restoring case. What fails is if you pg_dump -a to just dump the DATA from a table containing a tsearch2 trigger that is in a different schema. Then you delete all the rows from the table. Then you try to execute the sql script created from pg_dump to restore the data. It will fail because the sql script will automatically set the search_path to public, pg_catalog. And then as the COPY command inserts each row, it will fail immediately as the tsearch2 trigger will not be able to find its config table. Does that make sense? Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Subtle pg_dump problem...
On Wed, 12 May 2004, Christopher Kings-Lynne wrote: No problem, Actually, I did some more testing and I properly understand the problem now - and it won't happen in the general restoring case. What fails is if you pg_dump -a to just dump the DATA from a table containing a tsearch2 trigger that is in a different schema. Then you delete all the rows from the table. Then you try to execute the sql script created from pg_dump to restore the data. It will fail because the sql script will automatically set the search_path to public, pg_catalog. And then as the COPY command inserts each row, it will fail immediately as the tsearch2 trigger will not be able to find its config table. Does that make sense? Hmm, what other hackers thinks ? This is not just a tsearch2 problem, it could happens with any such kind of things, like defining user defined type in one scheme, using it in another, dumping separate data. Could pg_dump be enough smart to set search_path properly ? Chris Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] threads stuff/UnixWare
On Wed, 12 May 2004, Larry Rosenman wrote: At the risk of getting my butt kicked again, is there any way we can talk about how to deal with threads on UnixWare and the libpq stuff? Has any other platform come up with a need to look for the real pthread_* calls from libpq? I would REALLY like us to support threaded programs in UnixWare, and would like to NOT antagonize the community, but would like to have a direction I can go to make this happen in 7.5 before we freeze. Perfect time to start talking about it, at least :) Do you have a proposed patch to look at for doing this? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Subtle pg_dump problem...
Oleg Bartunov [EMAIL PROTECTED] writes: Hmm, what other hackers thinks ? This is not just a tsearch2 problem, it could happens with any such kind of things, like defining user defined type in one scheme, using it in another, dumping separate data. Could pg_dump be enough smart to set search_path properly ? It could not. I think the fundamental point here is that it is a real bad idea for the tsearch routines to make any assumptions about the current search path. What I would suggest is that the internal objects used by the tsearch routines (such as pg_ts_cfg) should be required to live in a specific schema (tsearch2 seems like a good name) and that all the internal references inside the tsearch functions should be fully qualified names. You could perhaps make this private schema name be selectable at the time tsearch is built ... but I'm not sure it's worth the trouble. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 12:57:10 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: At the risk of getting my butt kicked again, is there any way we can talk about how to deal with threads on UnixWare and the libpq stuff? Has any other platform come up with a need to look for the real pthread_* calls from libpq? I would REALLY like us to support threaded programs in UnixWare, and would like to NOT antagonize the community, but would like to have a direction I can go to make this happen in 7.5 before we freeze. Perfect time to start talking about it, at least :) Do you have a proposed patch to look at for doing this? No, as I believe the way I want to do it was frowned upon. I'd LIKE to be able to have PG wrappers for those functions, and have the first invocation of them look via dlsym() for the real ones, and if they are NOT there, use fake functions that assume we are NOT threaded. If they ARE present, indirect to the real functions. I am looking for direction before expending a lot of effort on it, if it will ultimately be rejected. Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 12:58:58 -0400 Tom Lane [EMAIL PROTECTED] wrote: Larry Rosenman [EMAIL PROTECTED] writes: At the risk of getting my butt kicked again, is there any way we can talk about how to deal with threads on UnixWare and the libpq stuff? In what way does the current thread stuff not work for you? regards, tom lane In the initdb compile: Undefined first referenced symbol in file pthread_mutex_unlocklibpq.so pthread_getspecific libpq.so pthread_mutex_lock libpq.so pthread_key_create libpq.so pthread_oncelibpq.so pthread_setspecific libpq.so UX:ld: ERROR: Symbol referencing errors. No output written to initdb gmake[3]: *** [initdb] Error 1 gmake[3]: Leaving directory `/home/ler/pg-dev/pgsql-server/src/bin/initdb' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/home/ler/pg-dev/pgsql-server/src/bin' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory `/home/ler/pg-dev/pgsql-server/src' gmake: *** [all] Error 2 $ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] threads stuff/UnixWare
Larry Rosenman [EMAIL PROTECTED] writes: At the risk of getting my butt kicked again, is there any way we can talk about how to deal with threads on UnixWare and the libpq stuff? In what way does the current thread stuff not work for you? regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] threads stuff/UnixWare
Larry Rosenman [EMAIL PROTECTED] writes: --On Wednesday, May 12, 2004 12:58:58 -0400 Tom Lane [EMAIL PROTECTED] In what way does the current thread stuff not work for you? In the initdb compile: Undefined first referenced symbol in file pthread_mutex_unlocklibpq.so pthread_getspecific libpq.so pthread_mutex_lock libpq.so pthread_key_create libpq.so pthread_oncelibpq.so pthread_setspecific libpq.so Hmm. And that means what? Does Unixware not have these functions? Are we just failing to suck in the needed library? regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 14:14:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: I'd LIKE to be able to have PG wrappers for those functions, and have the first invocation of them look via dlsym() for the real ones, and if they are NOT there, use fake functions that assume we are NOT threaded. Wouldn't it be easier to have a #define? Correct me if I'm wrong here, but the problem is that we are expecting thread functinos to be called x_, while Unixware defines them as pthread_, right? so make Unixware specific #defines that map x_ to pthread_ ... No, we are expecting them to have pthread_*. On UnixWare, to get the pthread_* functions, you need to invoke a compiler option (-Kpthread), to cause libpthread to be linked in. Since libpq.so now REQUIRES the function, we need to either: 1) force ANY program that uses libpq to be compiled/linked with -Kpthread or 2) dynamically determine if libpthread is loaded and use the real pthread_* functions if they are there (libpthread is in the image), or use fake singlethread versions of the pthread_* calls if libpthread is NOT in the image. Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] threads stuff/UnixWare
On Wed, 12 May 2004, Larry Rosenman wrote: I'd LIKE to be able to have PG wrappers for those functions, and have the first invocation of them look via dlsym() for the real ones, and if they are NOT there, use fake functions that assume we are NOT threaded. Wouldn't it be easier to have a #define? Correct me if I'm wrong here, but the problem is that we are expecting thread functinos to be called x_, while Unixware defines them as pthread_, right? so make Unixware specific #defines that map x_ to pthread_ ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Probably security hole in postgresql-7.4.1
Shachar Shemesh [EMAIL PROTECTED] writes: Also, if we want greater flexibility in handling these cases in the future, we should set up an invite-only list for reporting security bugs, and advertise it on the web site as the place to report security issues. Had this vulnerability been reported there, we could reasonably hold on without releasing a fix until 7.4.3 was ready. A lot of people would be unhappy with that approach. A) they don't know the people on the invite-only list and have no basis to trust them and B) Often when a white hat reports the problem the black hats have known about it for much longer already. -- greg ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 13:18:35 -0400 Tom Lane [EMAIL PROTECTED] wrote: Larry Rosenman [EMAIL PROTECTED] writes: --On Wednesday, May 12, 2004 12:58:58 -0400 Tom Lane [EMAIL PROTECTED] In what way does the current thread stuff not work for you? In the initdb compile: Undefined first referenced symbol in file pthread_mutex_unlocklibpq.so pthread_getspecific libpq.so pthread_mutex_lock libpq.so pthread_key_create libpq.so pthread_oncelibpq.so pthread_setspecific libpq.so Hmm. And that means what? Does Unixware not have these functions? Are we just failing to suck in the needed library? regards, tom lane No, we don't pass the -Kpthread (thread CFLAGS) to the C links. This is the whole discussion we had back in January/February about forcing -Kpthread for *ALL* libpq using programs, or dynamically determining if the image already is linked -Kpthread, or not supporting threads at all on UW. #3 is unacceptable to me. LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] psql variables
Neil Conway [EMAIL PROTECTED] writes: The historical origins of the feature are no excuse for its deficiencies. On the other hand, the alleged deficiencies are not bad enough to justify making non-backwards-compatible changes. If we were getting routine complaints from the field I might be willing to break things in order to address the issue. But we're not --- AFAIR you are the first to raise the point at all. On second thought, there's another alternative. Rather than improving \set, we could invent a new mechanism for setting psql-internal variables, and leave the \set stuff to user-defined variables. I was toying with the idea of inventing a \declare foo command (which would error out if the variable foo already exists), along with an optional setting that makes psql complain about either use of or assignment to an undeclared variable. As long as latter setting is false the behavior is backwards-compatible. By setting it true you get the sort of error checking you're after. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Probably security hole in postgresql-7.4.1
On Wed, May 12, 2004 at 10:46:00 +0300, Shachar Shemesh [EMAIL PROTECTED] wrote: Industry practices dictate that we do issue SOMETHING now. The bug is now public, and can be exploited. The description of the problem indicates that it can only be exploited after you have authenticated to the database. Since people who can connect to a postgres database can already cause denial of service attacks, this problem isn't a huge deal. It makes breaches in other programs (web server process especially) worse and provides another way for authorized users to cause problems. A release should probably be made soon, as a way to advertise the problem so that people are aware of it and can take appropiate steps. I don't think that this problem warrants bypassing normal minor release proceedure. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] threads stuff/UnixWare
On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 14:14:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: I'd LIKE to be able to have PG wrappers for those functions, and have the first invocation of them look via dlsym() for the real ones, and if they are NOT there, use fake functions that assume we are NOT threaded. Wouldn't it be easier to have a #define? Correct me if I'm wrong here, but the problem is that we are expecting thread functinos to be called x_, while Unixware defines them as pthread_, right? so make Unixware specific #defines that map x_ to pthread_ ... No, we are expecting them to have pthread_*. On UnixWare, to get the pthread_* functions, you need to invoke a compiler option (-Kpthread), to cause libpthread to be linked in. Since libpq.so now REQUIRES the function, we need to either: 1) force ANY program that uses libpq to be compiled/linked with -Kpthread Ummm, shouldn't that be added to the port specific Makefile? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 15:02:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 14:14:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: I'd LIKE to be able to have PG wrappers for those functions, and have the first invocation of them look via dlsym() for the real ones, and if they are NOT there, use fake functions that assume we are NOT threaded. Wouldn't it be easier to have a #define? Correct me if I'm wrong here, but the problem is that we are expecting thread functinos to be called x_, while Unixware defines them as pthread_, right? so make Unixware specific #defines that map x_ to pthread_ ... No, we are expecting them to have pthread_*. On UnixWare, to get the pthread_* functions, you need to invoke a compiler option (-Kpthread), to cause libpthread to be linked in. Since libpq.so now REQUIRES the function, we need to either: 1) force ANY program that uses libpq to be compiled/linked with -Kpthread Ummm, shouldn't that be added to the port specific Makefile? See my reply to Tom. It forces ALL libpq using programs to be linked with -Kpthread, which was deemed unacceptable. Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] Probably security hole in postgresql-7.4.1
Greg Stark [EMAIL PROTECTED] writes: Shachar Shemesh [EMAIL PROTECTED] writes: Also, if we want greater flexibility in handling these cases in the future, we should set up an invite-only list for reporting security bugs, A lot of people would be unhappy with that approach. A) they don't know the people on the invite-only list and have no basis to trust them and B) Often when a white hat reports the problem the black hats have known about it for much longer already. Past procedure for sensitive bugs has been for people to send reports to the core committee (pgsql-core at postgresql dot org). If you don't trust us you probably shouldn't be using Postgres ;-) As per other comments, I don't find this bug compelling enough to justify an instant release. Also, the original reporter is still running his analysis tool and has found some other things that might be worth patching. (Again, nothing compelling yet... but ...) I'm inclined to wait a bit longer and see if we can't include some more fixes in 7.4.3. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 15:39:34 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 15:02:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 14:14:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: I'd LIKE to be able to have PG wrappers for those functions, and have the first invocation of them look via dlsym() for the real ones, and if they are NOT there, use fake functions that assume we are NOT threaded. Wouldn't it be easier to have a #define? Correct me if I'm wrong here, but the problem is that we are expecting thread functinos to be called x_, while Unixware defines them as pthread_, right? so make Unixware specific #defines that map x_ to pthread_ ... No, we are expecting them to have pthread_*. On UnixWare, to get the pthread_* functions, you need to invoke a compiler option (-Kpthread), to cause libpthread to be linked in. Since libpq.so now REQUIRES the function, we need to either: 1) force ANY program that uses libpq to be compiled/linked with -Kpthread Ummm, shouldn't that be added to the port specific Makefile? See my reply to Tom. It forces ALL libpq using programs to be linked with -Kpthread, which was deemed unacceptable. deemed unacceptable by whom? Sounds to me alot simpler of a solution then making wrappers for the pthread_* functions just to accommodate one OS ... I could see it if this was a wide-spread problem, but it doesn't appear to be ... Bruce didn't want to muck ip all the .c programs in the distribution to do that. As well as any users using libpq. I'm all ears if the COMMUNITY wants to do it. LER Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] threads stuff/UnixWare
On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 15:02:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 14:14:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: I'd LIKE to be able to have PG wrappers for those functions, and have the first invocation of them look via dlsym() for the real ones, and if they are NOT there, use fake functions that assume we are NOT threaded. Wouldn't it be easier to have a #define? Correct me if I'm wrong here, but the problem is that we are expecting thread functinos to be called x_, while Unixware defines them as pthread_, right? so make Unixware specific #defines that map x_ to pthread_ ... No, we are expecting them to have pthread_*. On UnixWare, to get the pthread_* functions, you need to invoke a compiler option (-Kpthread), to cause libpthread to be linked in. Since libpq.so now REQUIRES the function, we need to either: 1) force ANY program that uses libpq to be compiled/linked with -Kpthread Ummm, shouldn't that be added to the port specific Makefile? See my reply to Tom. It forces ALL libpq using programs to be linked with -Kpthread, which was deemed unacceptable. deemed unacceptable by whom? Sounds to me alot simpler of a solution then making wrappers for the pthread_* functions just to accommodate one OS ... I could see it if this was a wide-spread problem, but it doesn't appear to be ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 15:59:19 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: Ummm, shouldn't that be added to the port specific Makefile? See my reply to Tom. It forces ALL libpq using programs to be linked with -Kpthread, which was deemed unacceptable. deemed unacceptable by whom? Sounds to me alot simpler of a solution then making wrappers for the pthread_* functions just to accommodate one OS ... I could see it if this was a wide-spread problem, but it doesn't appear to be ... Bruce didn't want to muck ip all the .c programs in the distribution to do that. I'm confused now ... what .c programs would have to be mucked up if you add -Kpthread to the Unixware specific Makefile? initdb, et al. See these comments in src/template/unixware: # Disabled because flags are required for all apps using libpq. # Waiting to see if other platforms need this too. 2004-03-22 THREAD_SUPPORT=yes # verified UnixWare 7.1.4 2004-03-18 STRERROR_THREADSAFE=yes GETPWUID_THREADSAFE=yes GETHOSTBYNAME_THREADSAFE=yes $ I turned it back on to see if the new stuff had fixed it. Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] threads stuff/UnixWare
On Wed, 12 May 2004, Larry Rosenman wrote: Ummm, shouldn't that be added to the port specific Makefile? See my reply to Tom. It forces ALL libpq using programs to be linked with -Kpthread, which was deemed unacceptable. deemed unacceptable by whom? Sounds to me alot simpler of a solution then making wrappers for the pthread_* functions just to accommodate one OS ... I could see it if this was a wide-spread problem, but it doesn't appear to be ... Bruce didn't want to muck ip all the .c programs in the distribution to do that. I'm confused now ... what .c programs would have to be mucked up if you add -Kpthread to the Unixware specific Makefile? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane [EMAIL PROTECTED] wrote: Larry Rosenman [EMAIL PROTECTED] writes: This is the whole discussion we had back in January/February about forcing -Kpthread for *ALL* libpq using programs, or dynamically determining if the image already is linked -Kpthread, or not supporting threads at all on UW. Oh, that business :-(. AFAIR we did not much care for any of the proposed alternatives. Have you thought of any new options? At this point I'd settle for saying that --enable-thread-safety on Unixware will generate a library that requires -Kpthread. This is kinda grungy but it seems that any more-pleasant solution would require a disproportionate amount of work. If I did the work for the dlsym() stuff would you and the rest of core@ accept it? regards, tom lane -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] threads stuff/UnixWare
Larry Rosenman [EMAIL PROTECTED] writes: This is the whole discussion we had back in January/February about forcing -Kpthread for *ALL* libpq using programs, or dynamically determining if the image already is linked -Kpthread, or not supporting threads at all on UW. Oh, that business :-(. AFAIR we did not much care for any of the proposed alternatives. Have you thought of any new options? At this point I'd settle for saying that --enable-thread-safety on Unixware will generate a library that requires -Kpthread. This is kinda grungy but it seems that any more-pleasant solution would require a disproportionate amount of work. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] threads stuff/UnixWare
On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 15:59:19 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: Ummm, shouldn't that be added to the port specific Makefile? See my reply to Tom. It forces ALL libpq using programs to be linked with -Kpthread, which was deemed unacceptable. deemed unacceptable by whom? Sounds to me alot simpler of a solution then making wrappers for the pthread_* functions just to accommodate one OS ... I could see it if this was a wide-spread problem, but it doesn't appear to be ... Bruce didn't want to muck ip all the .c programs in the distribution to do that. I'm confused now ... what .c programs would have to be mucked up if you add -Kpthread to the Unixware specific Makefile? initdb, et al. See these comments in src/template/unixware: # Disabled because flags are required for all apps using libpq. # Waiting to see if other platforms need this too. 2004-03-22 THREAD_SUPPORT=yes # verified UnixWare 7.1.4 2004-03-18 STRERROR_THREADSAFE=yes GETPWUID_THREADSAFE=yes GETHOSTBYNAME_THREADSAFE=yes $ I turned it back on to see if the new stuff had fixed it. k, I just looked at the template/unixware file myself, and it looks to me like -Kpthread is already defined, and included as part of THREAD_CPPFLAGS ... is THREAD_CPPFLAGS not honor'd in all of the various Makefile's (ie. initdb?) ... Please correct me if I'm wrong here, but so far, the issue seems to be that now that libpq uses pthread_* by default, Unixware requires -Kpthread to be used when building all the various support programs that use libpq ... and -Kpthread is defined in template/unixware ... so I'm *really* confused as to what is actually broken, except maybe the Makefile's themselves ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] threads stuff/UnixWare
Larry Rosenman [EMAIL PROTECTED] writes: --On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane [EMAIL PROTECTED]=20 wrote: At this point I'd settle for saying that --enable-thread-safety on Unixware will generate a library that requires -Kpthread. This is kinda grungy but it seems that any more-pleasant solution would require a disproportionate amount of work. If I did the work for the dlsym() stuff would you and the rest of core@ accept it? How invasive a change are we talking about? I'd be inclined to reject a patch that makes libpq materially less readable ... regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 16:00:48 -0400 Tom Lane [EMAIL PROTECTED] wrote: Larry Rosenman [EMAIL PROTECTED] writes: --On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane [EMAIL PROTECTED]=20 wrote: At this point I'd settle for saying that --enable-thread-safety on Unixware will generate a library that requires -Kpthread. This is kinda grungy but it seems that any more-pleasant solution would require a disproportionate amount of work. If I did the work for the dlsym() stuff would you and the rest of core@ accept it? How invasive a change are we talking about? I'd be inclined to reject a patch that makes libpq materially less readable ... regards, tom lane I was thinking of pq_pthread_* calls, and that function would set a static flag for calling either the real pthread_* function or a statically named version in libpgport.a that is a single thread wrapper. I know, this sucks, but, I don't see any other way, other than linking *ALL* libpq-using programs (including initdb and friends) with -K pthread. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] threads stuff/UnixWare
Larry Rosenman [EMAIL PROTECTED] writes: I was thinking of pq_pthread_* calls, and that function would set a static flag for calling either the real pthread_* function or a statically named version in libpgport.a that is a single thread wrapper. And how will you avoid having a link-time dependency on the real pthread function? You muttered about dlsym but how much code will that take, and what kind of runtime penalty will we incur? (IIRC the pthread functions are performance critical.) Even more to the point, can you make it work at all? I seem to recall from the prior discussion that -Kpthread actually changes some code generation details on that platform. Are -Kpthread and non -Kpthread libraries interoperable at all? I know, this sucks, but, I don't see any other way, other than linking *ALL* libpq-using programs (including initdb and friends) with -K pthread. -Kpthread doesn't sound that bad to me, as long as it's documented. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 16:22:58 -0400 Tom Lane [EMAIL PROTECTED] wrote: Larry Rosenman [EMAIL PROTECTED] writes: I was thinking of pq_pthread_* calls, and that function would set a static flag for calling either the real pthread_* function or a statically named version in libpgport.a that is a single thread wrapper. And how will you avoid having a link-time dependency on the real pthread function? You muttered about dlsym but how much code will that take, and what kind of runtime penalty will we incur? (IIRC the pthread functions are performance critical.) The first call to ANY of the pthread_* functions would set the flag, and would cache the dlsym() info. Even more to the point, can you make it work at all? I seem to recall from the prior discussion that -Kpthread actually changes some code generation details on that platform. Are -Kpthread and non -Kpthread libraries interoperable at all? Yes, this is how libc deals with it. We wouldn't have a hard symbol for the pthread_* calls. I can ask the compiler guys at SCO if you want. I know, this sucks, but, I don't see any other way, other than linking *ALL* libpq-using programs (including initdb and friends) with -K pthread. -Kpthread doesn't sound that bad to me, as long as it's documented. regards, tom lane -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 17:29:30 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 16:00:48 -0400 Tom Lane [EMAIL PROTECTED] wrote: Larry Rosenman [EMAIL PROTECTED] writes: --On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane [EMAIL PROTECTED]=20 wrote: At this point I'd settle for saying that --enable-thread-safety on Unixware will generate a library that requires -Kpthread. This is kinda grungy but it seems that any more-pleasant solution would require a disproportionate amount of work. If I did the work for the dlsym() stuff would you and the rest of core@ accept it? How invasive a change are we talking about? I'd be inclined to reject a patch that makes libpq materially less readable ... regards, tom lane I was thinking of pq_pthread_* calls, and that function would set a static flag for calling either the real pthread_* function or a statically named version in libpgport.a that is a single thread wrapper. I know, this sucks, but, I don't see any other way, other than linking *ALL* libpq-using programs (including initdb and friends) with -K pthread. k, a change that 'sucks', vs linking against -Kpthread ... I'm for the -Kpthread route myself, which still sounds the 'clean' solution ... that was rejected back in Jan-Mar. BUT, I agree it would work. I tried to submit the patch, and it was killed. LER Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] threads stuff/UnixWare
On Wed, 12 May 2004, Larry Rosenman wrote: --On Wednesday, May 12, 2004 16:00:48 -0400 Tom Lane [EMAIL PROTECTED] wrote: Larry Rosenman [EMAIL PROTECTED] writes: --On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane [EMAIL PROTECTED]=20 wrote: At this point I'd settle for saying that --enable-thread-safety on Unixware will generate a library that requires -Kpthread. This is kinda grungy but it seems that any more-pleasant solution would require a disproportionate amount of work. If I did the work for the dlsym() stuff would you and the rest of core@ accept it? How invasive a change are we talking about? I'd be inclined to reject a patch that makes libpq materially less readable ... regards, tom lane I was thinking of pq_pthread_* calls, and that function would set a static flag for calling either the real pthread_* function or a statically named version in libpgport.a that is a single thread wrapper. I know, this sucks, but, I don't see any other way, other than linking *ALL* libpq-using programs (including initdb and friends) with -K pthread. k, a change that 'sucks', vs linking against -Kpthread ... I'm for the -Kpthread route myself, which still sounds the 'clean' solution ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Probably security hole in postgresql-7.4.1
Bruno Wolff III wrote: On Wed, May 12, 2004 at 10:46:00 +0300, Shachar Shemesh [EMAIL PROTECTED] wrote: Industry practices dictate that we do issue SOMETHING now. The bug is now public, and can be exploited. The description of the problem indicates that it can only be exploited after you have authenticated to the database. Since people who can connect to a postgres database can already cause denial of service attacks, this problem isn't a huge deal. My take on this is different. To me, a DoS is a nuisance, but an arbitrary code execution vulnerability means information leak, and a major escalation (from which further escalation may be possible). It makes breaches in other programs (web server process especially) worse and provides another way for authorized users to cause problems. Not to mention being another chain. A release should probably be made soon, as a way to advertise the problem so that people are aware of it and can take appropiate steps. I don't think that this problem warrants bypassing normal minor release proceedure. Ok. How about an official patch against 7.4.2 that fixes it, so that packagers can make their own informed decision. Also, has anybody checked what other versions are affected? Is 7.3? 7.2? Some people can't afford to upgrade due to data inconsistancy. For those reasons I suggested a seperate mailing list. Shachar -- Shachar Shemesh Lingnu Open Source Consulting http://www.lingnu.com/ ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Linux 2.6.6 also
Gregory Stark wrote: This patch also looks relevant to Postgres for two reasons. This part seems like it might expose some bugs that otherwise might have remained hidden: This affects I/O scheduling potentially quite significantly. It is no longer the case that the kernel will submit pages for I/O in the order in which the application dirtied them. We instead submit them in file-offset order all the time. The part about part-file fdatasync calls seems like could be really useful. It seems like that's just speculation about future directions though? Correct. The kernel could do that now, but it's not exposed to user space. But the change highlights one point: the order in which file blocks are written to disk is undefined. Theoretically the wal checkpoint record could be on the platter, but the preceeding pages were not written. Is that case handled by the wal replay code? -- Manfred ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Probably security hole in postgresql-7.4.1
Shachar Shemesh [EMAIL PROTECTED] writes: Ok. How about an official patch against 7.4.2 that fixes it, so that packagers can make their own informed decision. The official patch is available to anyone who wants it from our CVS server. http://developer.postgresql.org/cvsweb.cgi/pgsql-server/src/backend/lib/stringinfo.c.diff?r1=1.36r2=1.36.4.1 BTW, all the principal packagers read this list and have doubtless made their informed decisions already ... Also, has anybody checked what other versions are affected? Nothing before 7.4, at least by the known implications of this issue. Again, if we wait a while and let Ken keep running his analysis tool, he might turn up other stuff we need to fix. Maybe even stuff that needs a fix much worse than this does. Industry practices dictate that we do issue SOMETHING now. The bug is now public, and can be exploited. I frankly think that this discussion is emblematic of all the worst tendencies of the security community. Have you forgotten the fable about the boy who cried wolf? If you demand a Chinese fire drill for every issue that could conceivably be exploited, you'll soon find yourself unable to get peoples' attention for problems that are really serious. I repeat: in my estimation this is not a bug that needs a fix yesterday. AFAICS it would be very difficult to cause more than a nuisance DOS with it, and there are plenty of other ways for authenticated database users to cause those. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Probably security hole in postgresql-7.4.1
On Wed, May 12, 2004 at 23:36:49 +0300, Shachar Shemesh [EMAIL PROTECTED] wrote: My take on this is different. To me, a DoS is a nuisance, but an arbitrary code execution vulnerability means information leak, and a major escalation (from which further escalation may be possible). A DOS is generally more than a nuisance in production environments. In most cases you aren't going to be giving direct access to your DB to people that aren't fairly trusted. The exception may be some of the web hosting places that provide DB backed web pages. Not to mention being another chain. This isn't very significant as you have to authenticate to the DB first to exploit it. That's a lot less of a problem than something directly accessible by anyone from the net such as a web server. Ok. How about an official patch against 7.4.2 that fixes it, so that packagers can make their own informed decision. Also, has anybody checked what other versions are affected? Is 7.3? 7.2? Some people can't afford to upgrade due to data inconsistancy. Hasn't it already been committed to 7.4 stable? If so, just grab an update from CVS. Something should probably be done about 7.2 and 7.3 if the same bug exists in those versions. Nobody should be running anything earlier than that. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Probably security hole in postgresql-7.4.1
Tom Lane wrote: Shachar Shemesh [EMAIL PROTECTED] writes: Also, has anybody checked what other versions are affected? Nothing before 7.4, at least by the known implications of this issue. Again, if we wait a while and let Ken keep running his analysis tool, he might turn up other stuff we need to fix. Maybe even stuff that needs a fix much worse than this does. and also I frankly think that this discussion is emblematic of all the worst tendencies of the security community. Have you forgotten the fable about the boy who cried wolf? I totally agree. That's why I suggested preventing the automatic public disclosure for Ken's next bugs, as well as anyone else's. This way, if we do need a few extra days, we can have them while still limiting the window of exposure. I repeat: in my estimation this is not a bug that needs a fix yesterday. AFAICS it would be very difficult to cause more than a nuisance DOS with it, and there are plenty of other ways for authenticated database users to cause those. I'm sorry. Maybe it's spending too many years in the security industry (I've been Check Point's oh my god we have a security problem process manager for over two years). Maybe it's knowing how to actually exploit these problems. Maybe it's just seeing many of the good guys (OpenBSD's Theo included) fall flat on their faces after saying This is a DoS only. In my book, a buffer overrun=arbitrary code execution. For a now famous example of a bug declared non exploitable, followed by an exploit, see http://www.theinquirer.net/?article=4053. I have been on the mailing lists at the time. The problem was declared unexploitable on i386 by some of the best known names in the security industry of the time. regards, tom lane Please. I'm not saying Release now. I'm saying get a mechanism for smarter handling of future events. Shachar -- Shachar Shemesh Lingnu Open Source Consulting http://www.lingnu.com/ ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Linux 2.6.6 also
Manfred Spraul [EMAIL PROTECTED] writes: But the change highlights one point: the order in which file blocks are written to disk is undefined. Theoretically the wal checkpoint record could be on the platter, but the preceeding pages were not written. Is that case handled by the wal replay code? We don't care, unless of course the kernel returns claiming that fsync is done when it's not all done yet. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] threads stuff/UnixWare
On Wed, 12 May 2004, Larry Rosenman wrote: k, a change that 'sucks', vs linking against -Kpthread ... I'm for the -Kpthread route myself, which still sounds the 'clean' solution ... that was rejected back in Jan-Mar. BUT, I agree it would work. I tried to submit the patch, and it was killed. Please re-submit since I don't recall the patch ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 7.5 features
On Wed, 12 May 2004, Jan Wieck wrote: Can we please make move all replication software out of the release an official open item for the 7.5 release? Agreed ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] threads stuff/UnixWare
Larry Rosenman [EMAIL PROTECTED] writes: Please save us all time searching by providing a URL ... I can't find my posts on archives.postgresql.org, but can find it in MY archives. Well, then we won't be able to find 'em either, so please repost. This is heading down the same path I was back on 3/22. I think at the time we were hoping for a cleaner solution. Since none has emerged, it would be good of you to refresh our memories of where things stood. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] UnixWare/Threads stuff [repost from March]
This is a re-post, and still the state of things. There were a few posts after this because of my STRENUOUS objection below. I still think we should enable threads somehow on this platform. --On Monday, March 22, 2004 11:14:59 -0600 Larry Rosenman [EMAIL PROTECTED] wrote: --On Monday, March 22, 2004 09:52:54 -0500 Bruce Momjian [EMAIL PROTECTED] wrote: Bruce Momjian wrote: Larry Rosenman wrote: The a.out (not any library) should be linked with -Kpthread (not -lpthread). This will force libthread to be linked in the right order relative to libc, libC, networking libraries, etc. In other words, the entire application either is or is not linked with threads; it's not a property of an individual library. SO, IF we are using the threads flags, we need to use them on ALL libpq-using programs, ours or the users. Seems we have a few options for making threaded libpq on Unixware: o remove thread-safe SIGPIPE code, which calls thread library o create a threaded and non-threaded libpq library o add a libpq function that enables threading, and do dynamic linking of thread calls based on that function o Add thread flags to all builds on that platform, including the backend o Add the ability to specify compile/link flags for everything but the backend As I remember, libcrypt used to be required by all libpq builds on various platforms. This seems to be a similar case. I think the last option might be the best. Somehow create different cppflags/libs for the backend and non-backend programs. One other option is to disable threads on this platform for 7.5 unless we find another platforms that need this to use threads. That is the direction I will take for the moment. If someone needs a threaded libpq on this platform, then can enable threads, compile libpq alone, and rename it with a thread extension. This will be a REGRESSION from 7.4. I object STRENUOUSLY to this tack. I want to see threads enabled for this platform since it is a threaded kernel. What do we have to do to get there? What code do I need to write for y'all? LER -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] Probably security hole in postgresql-7.4.1
On Thu, May 13, 2004 at 00:54:19 +0300, Shachar Shemesh [EMAIL PROTECTED] wrote: I'm sorry. Maybe it's spending too many years in the security industry (I've been Check Point's oh my god we have a security problem process manager for over two years). Maybe it's knowing how to actually exploit these problems. Maybe it's just seeing many of the good guys (OpenBSD's Theo included) fall flat on their faces after saying This is a DoS only. In my book, a buffer overrun=arbitrary code execution. But it is still a local user exploit, not a remote user exploit. That makes a big difference in how the bug should be treated. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] threads stuff/UnixWare
Tom Lane wrote: Larry Rosenman [EMAIL PROTECTED] writes: Please save us all time searching by providing a URL ... I can't find my posts on archives.postgresql.org, but can find it in MY archives. Well, then we won't be able to find 'em either, so please repost. This is heading down the same path I was back on 3/22. I think at the time we were hoping for a cleaner solution. Since none has emerged, it would be good of you to refresh our memories of where things stood. [ Sorry I have been away from email today. ] Larry, now that I have put the thread testing into configure, I am ready to deal with Unixware. In fact I posted to the list asking you about it but was too lazy to look up your email address. Anyway, I think the only solution is going to be to use the -K flag on all compiles on that platform if you want threads, so anything that uses libpq will need it. We don't have a client-level flags option at this point, but we can add it, or just the flag for the backend compile too. I think that is our best option. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] UnixWare/Threads stuff [repost from March]
I don't see a patch in here ... forget to attach it? On Wed, 12 May 2004, Larry Rosenman wrote: This is a re-post, and still the state of things. There were a few posts after this because of my STRENUOUS objection below. I still think we should enable threads somehow on this platform. --On Monday, March 22, 2004 11:14:59 -0600 Larry Rosenman [EMAIL PROTECTED] wrote: --On Monday, March 22, 2004 09:52:54 -0500 Bruce Momjian [EMAIL PROTECTED] wrote: Bruce Momjian wrote: Larry Rosenman wrote: The a.out (not any library) should be linked with -Kpthread (not -lpthread). This will force libthread to be linked in the right order relative to libc, libC, networking libraries, etc. In other words, the entire application either is or is not linked with threads; it's not a property of an individual library. SO, IF we are using the threads flags, we need to use them on ALL libpq-using programs, ours or the users. Seems we have a few options for making threaded libpq on Unixware: o remove thread-safe SIGPIPE code, which calls thread library o create a threaded and non-threaded libpq library o add a libpq function that enables threading, and do dynamic linking of thread calls based on that function o Add thread flags to all builds on that platform, including the backend o Add the ability to specify compile/link flags for everything but the backend As I remember, libcrypt used to be required by all libpq builds on various platforms. This seems to be a similar case. I think the last option might be the best. Somehow create different cppflags/libs for the backend and non-backend programs. One other option is to disable threads on this platform for 7.5 unless we find another platforms that need this to use threads. That is the direction I will take for the moment. If someone needs a threaded libpq on this platform, then can enable threads, compile libpq alone, and rename it with a thread extension. This will be a REGRESSION from 7.4. I object STRENUOUSLY to this tack. I want to see threads enabled for this platform since it is a threaded kernel. What do we have to do to get there? What code do I need to write for y'all? LER -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] UnixWare/Threads stuff [repost from March]
--On Wednesday, May 12, 2004 22:14:33 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: I don't see a patch in here ... forget to attach it? It was way back in the thread, and this was the upshot of the discussion. See the note I just replied to Bruce with. LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] Parser change needed?
Hi Thomas, Please don't get too disheartened that a developer hasn't commented on your stuff yet. Everyone's very busy at the moment. Just hang in there! Chris Thomas Hallgren wrote: Yes, this is another vain attempt to get some attention to the custom config variables patch that I've submitted (twice) :-) But it's also a relevant question. Given that the design is accepted and a custom variable will use qualified names I'd propose a change to the SQL parser so that the names doesn't need to be quoted in the SET and SHOW commands. Today one must write: SET pljava.vmoptions TO '-Xmx128M' I think it would be more clean if one could write: SET pljava.vmoptions TO '-Xmx128M' What's your thoughts? Kind regards, Thomas Hallgren ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Subtle pg_dump problem...
It could not. I think the fundamental point here is that it is a real bad idea for the tsearch routines to make any assumptions about the current search path. What I would suggest is that the internal objects used by the tsearch routines (such as pg_ts_cfg) should be required to live in a specific schema (tsearch2 seems like a good name) and that all the internal references inside the tsearch functions should be fully qualified names. I think a better solution is to change tsearch2 to have two assumptions: 1. All tsearch2 objects will be loaded in the same schema, name not important. 2. When an object foo is called and needs to refer to another object bar, it should assume that bar exists in the same schema as foo, and NOT in the current search_path. Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 21:08:25 -0400 Bruce Momjian [EMAIL PROTECTED] wrote: Tom Lane wrote: Larry Rosenman [EMAIL PROTECTED] writes: Please save us all time searching by providing a URL ... I can't find my posts on archives.postgresql.org, but can find it in MY archives. Well, then we won't be able to find 'em either, so please repost. This is heading down the same path I was back on 3/22. I think at the time we were hoping for a cleaner solution. Since none has emerged, it would be good of you to refresh our memories of where things stood. [ Sorry I have been away from email today. ] Larry, now that I have put the thread testing into configure, I am ready to deal with Unixware. In fact I posted to the list asking you about it but was too lazy to look up your email address. Anyway, I think the only solution is going to be to use the -K flag on all compiles on that platform if you want threads, so anything that uses libpq will need it. We don't have a client-level flags option at this point, but we can add it, or just the flag for the backend compile too. I think a client-level flag would be best, that way the backend doesn't get the wrapped functions from libthread. Can you do the patch? I think that is our best option. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] threads stuff/UnixWare
Larry Rosenman wrote: [ Sorry I have been away from email today. ] Larry, now that I have put the thread testing into configure, I am ready to deal with Unixware. In fact I posted to the list asking you about it but was too lazy to look up your email address. Anyway, I think the only solution is going to be to use the -K flag on all compiles on that platform if you want threads, so anything that uses libpq will need it. We don't have a client-level flags option at this point, but we can add it, or just the flag for the backend compile too. I think a client-level flag would be best, that way the backend doesn't get the wrapped functions from libthread. Can you do the patch? OK. Can someone think of a clean way to add a flag to everything but the backend build? Hold, what about /port. Larry, if we compile anything with threads does everything that links with that object file require the flag, even if there are no thread flags in the object file? I don't think so, but I am just checking. Actually, my bet is that it might do that because if it didn't, there would be no problem with using that flag on the backend because there are no threading calls in there. I am afraid of libpgport. Those object files get used by the backend, and by the client stuff. Larry can -K and non-K object files be mixed in the same binary? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 21:55:40 -0400 Bruce Momjian [EMAIL PROTECTED] wrote: Larry Rosenman wrote: [ Sorry I have been away from email today. ] Larry, now that I have put the thread testing into configure, I am ready to deal with Unixware. In fact I posted to the list asking you about it but was too lazy to look up your email address. Anyway, I think the only solution is going to be to use the -K flag on all compiles on that platform if you want threads, so anything that uses libpq will need it. We don't have a client-level flags option at this point, but we can add it, or just the flag for the backend compile too. I think a client-level flag would be best, that way the backend doesn't get the wrapped functions from libthread. Can you do the patch? OK. Can someone think of a clean way to add a flag to everything but the backend build? Hold, what about /port. Larry, if we compile anything with threads does everything that links with that object file require the flag, even if there are no thread flags in the object file? I don't think so, but I am just checking. Actually, my bet is that it might do that because if it didn't, there would be no problem with using that flag on the backend because there are no threading calls in there. Yes, there would still be the overhead, because the functions that libthread wraps would go through that overhead since libthread does it's magic at _ini time. Y'all were concerned with overhead in previous discussions. If you want to link the backend with -Kpthread, you can, but basically the client code (initdb, et al) should be -Kpthread, and the backend should NOT since there are no threads calls in the backend. I am afraid of libpgport. Those object files get used by the backend, and by the client stuff. Larry can -K and non-K object files be mixed in the same binary? Yes, they can. But, you MUST link ANY pthread_* using code with -K pthread. See above discussion. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
Re: [HACKERS] Subtle pg_dump problem...
Christopher Kings-Lynne [EMAIL PROTECTED] writes: 2. When an object foo is called and needs to refer to another object bar, it should assume that bar exists in the same schema as foo, and NOT in the current search_path. That would be great if a C function could find out what schema it had been declared in, but I don't think it can readily do so. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] threads stuff/UnixWare
Larry Rosenman wrote: Yes, there would still be the overhead, because the functions that libthread wraps would go through that overhead since libthread does it's magic at _ini time. Y'all were concerned with overhead in previous discussions. If you want to link the backend with -Kpthread, you can, but basically the client code (initdb, et al) should be -Kpthread, and the backend should NOT since there are no threads calls in the backend. I am afraid of libpgport. Those object files get used by the backend, and by the client stuff. Larry can -K and non-K object files be mixed in the same binary? Yes, they can. But, you MUST link ANY pthread_* using code with -K pthread. OK, can someone come up with a cute way to add flags only to things that use libpq, meaning /interfaces and /bin? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Subtle pg_dump problem...
That would be great if a C function could find out what schema it had been declared in, but I don't think it can readily do so. There's no context information available to it at all? Even if you go contrib.tsearch2 qualfication? How about making it so that the default context for functions is their own schema? :) Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Rough draft for Unicode-aware UPPER()/LOWER()/INITCAP()
I got tired of reading complaints about how upper/lower don't work with
Unicode, so I went and prototyped a solution. The attached code uses
the C99-standard functions mbstowcs and wcstombs to convert to and from
a wchar_t[] representation that can be fed to the also-C99 functions
towupper, towlower, etc.
This code will only work if the database is running under an LC_CTYPE
setting that implies the same encoding specified by server_encoding.
However, I don't see that as a fatal objection, because in point of fact
the existing upper/lower code assumes the same thing. When they don't
match, this code may deliver an invalid multibyte character error
rather than silently producing a wrong answer, but is that really a step
backward?
Note this patch is *not* meant for application to CVS yet. It's not
autoconfiscated. But if you have a platform that has mbstowcs and
friends, please try it and let me know about any portability gotchas
you see.
Also, as a character-set-impaired American, I'm probably not the best
qualified person to judge whether the patch actually does what's wanted.
It seemed to do the right sorts of conversions in my limited testing,
but does it do what *you* want it to do?
regards, tom lane
PS: the patch works against either 7.4 or CVS tip.
*** src/backend/utils/adt/oracle_compat.c.orig Sat Feb 28 12:53:23 2004
--- src/backend/utils/adt/oracle_compat.c Wed May 12 21:19:33 2004
***
*** 15,21
*/
#include postgres.h
! #include ctype.h
#include utils/builtins.h
#include mb/pg_wchar.h
--- 15,22
*/
#include postgres.h
! #include wchar.h
! #include wctype.h
#include utils/builtins.h
#include mb/pg_wchar.h
***
*** 26,31
--- 27,124
bool doltrim, bool dortrim);
+ /*
+ * Convert a TEXT value into a palloc'd wchar string.
+ */
+ static wchar_t *
+ texttowcs(const text *txt)
+ {
+ int nbytes = VARSIZE(txt) - VARHDRSZ;
+ char *workstr;
+ wchar_t*result;
+ size_t ncodes;
+
+ /* Overflow paranoia */
+ if (nbytes 0 ||
+ nbytes (int) (INT_MAX / sizeof(wchar_t)) - 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+errmsg(out of memory)));
+
+ /* Need a null-terminated version of the input */
+ workstr = (char *) palloc(nbytes + 1);
+ memcpy(workstr, VARDATA(txt), nbytes);
+ workstr[nbytes] = '\0';
+
+ /* Output workspace cannot have more codes than input bytes */
+ result = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t));
+
+ /* Do the conversion */
+ ncodes = mbstowcs(result, workstr, nbytes + 1);
+
+ if (ncodes == (size_t) -1)
+ {
+ /*
+* Invalid multibyte character encountered. We try to give a useful
+* error message by letting pg_verifymbstr check the string. But
+* it's possible that the string is OK to us, and not OK to mbstowcs
+* --- this suggests that the LC_CTYPE locale is different from the
+* database encoding. Give a generic error message if verifymbstr
+* can't find anything wrong.
+*/
+ pg_verifymbstr(workstr, nbytes, false);
+ ereport(ERROR,
+ (errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
+errmsg(invalid multibyte character for locale)));
+ }
+
+ Assert(ncodes = (size_t) nbytes);
+
+ return result;
+ }
+
+
+ /*
+ * Convert a wchar string into a palloc'd TEXT value. The wchar string
+ * must be zero-terminated, but we also require the caller to pass the string
+ * length, since it will know it anyway in current uses.
+ */
+ static text *
+ wcstotext(const wchar_t *str, int ncodes)
+ {
+ text *result;
+ size_t nbytes;
+
+ /* Overflow paranoia */
+ if (ncodes 0 ||
+ ncodes (int) ((INT_MAX - VARHDRSZ) / MB_CUR_MAX) - 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+errmsg(out of memory)));
+
+ /* Make workspace certainly large enough for result */
+ result = (text *) palloc((ncodes + 1) * MB_CUR_MAX + VARHDRSZ);
+
+ /* Do the conversion */
+ nbytes = wcstombs((char *) VARDATA(result), str,
+ (ncodes + 1) * MB_CUR_MAX);
+
+ if (nbytes == (size_t) -1)
+ {
+ /* Invalid multibyte character encountered ... shouldn't happen */
+ ereport(ERROR,
+ (errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
+errmsg(invalid multibyte character for locale)));
+ }
+
+ Assert(nbytes = (size_t) (ncodes
Re: [HACKERS] threads stuff/UnixWare
--On Wednesday, May 12, 2004 22:26:03 -0400 Bruce Momjian [EMAIL PROTECTED] wrote: Larry Rosenman wrote: Yes, there would still be the overhead, because the functions that libthread wraps would go through that overhead since libthread does it's magic at _ini time. Y'all were concerned with overhead in previous discussions. If you want to link the backend with -Kpthread, you can, but basically the client code (initdb, et al) should be -Kpthread, and the backend should NOT since there are no threads calls in the backend. I am afraid of libpgport. Those object files get used by the backend, and by the client stuff. Larry can -K and non-K object files be mixed in the same binary? Yes, they can. But, you MUST link ANY pthread_* using code with -K pthread. OK, can someone come up with a cute way to add flags only to things that use libpq, meaning /interfaces and /bin? What about where -lpq is specified? (I'm NOT a GNU Make guru). LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 pgp0.pgp Description: PGP signature
[HACKERS] relcache refcount
Hackers, I'm stuck trying to figure out how to decrease reference counting for relcache entries at subtransaction abort. Initially I thought I could just drop them all to zero, because a subtransaction boundary should be enough warranty that the entries are no longer needed. However I now think this is bogus, because maybe a function could open a new transaction and abort it; and a surrounding query would need the previous relcache entry. So this cannot possibly work (if I'm wrong I'll be very happy because this is the easiest way). Keeping a list of all entries the current subtrans holds and its local refcount sounds ridiculous, doesn't it? We would need one hash per subtransaction; this is very bad. Any ideas out there? Incidentally, I assume that LWLocks are not going to be needed across subtransaction boundaries -- I release them all on abort, just as it's done on main transaction abort. Same for catcache entries. Does anyone think this is incorrect? -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) No hay cielo posible sin hundir nuestras raíces en la profundidad de la tierra(Malucha Pinto) ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] anoncvs moved
A couple of months back, in order to deal with a client that was being DDoS'd, we setup an 'offsite' server that we have about 1.4TB of bandwidth into ... the machine itself isn't super powerful, but we also have very very little on it ... We moved bittorrent and ftp over to it about a month ago, and tonight I just moved over anoncvs ... both ftp and anoncvs are updated hourly, and I've tested to make sure that I can access/checkout from anoncvs, but if anyone notices any problems, please let me know ... I've also added that server as a mail relay, which should take some load off of those that, to date, have been generous enough to allow us to use their servers ... Nothing on this server is primary ... all that I'm putting over there is a mirror of what already exists on the local servers, so if for some reason that machine does blow up, its a matter of repointing the DNS back ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 8: explain analyze is your friend
