Re: [HACKERS] Threads vs Processes
When the address-of operator is applied to a thread-local variable, it is evaluated at run-time and returns the address of the current thread's instance of that variable. An address so obtained may be used by any thread. When a thread terminates, any pointers to thread-local variables in that thread become invalid. Bummer, I would have thought one advantage of using TLS must surely be memory protection ? So the only for pg useful usage for TLS seems to be __declspec(thread) and __declspec(thread) static (both for stuff that do not need runtime preinitialization). Maybe the techniques of electric fence could be used for protecting the shmem at least a little bit. Andreas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Threads vs Processes
Tom Lane wrote:
Shridhar Daithankar [EMAIL PROTECTED] writes:
One thing that can be done is to arrange all globals/statics in a
structure and make that structure thread local.
That's about as far from non-invasive as I can imagine :-(
I know.
I have following half baked solution. Obviously it won't be as good and
automatic as __declspec(thread). But still... Just for starters
1. Put all static/global variables in .h file. No more in .c files
2. Encapsulate them in a #define.
3. Create a new header files which includes lot of macro invocations. That way
we expand a macro definition, the structure definition changes on recompile.
Like in a.h
a.h
---
#define module1 int a;\
int b;
---
b.h
---
typedef struct
{
module1
module2
} globalDataStructure
---
Whoever includes a.h earlier, needs to include b.h now. That way things will
flow smoothly. This could be optional but I would rather not include a.h anymore
as a good practice.
4. Add a macro definition for initialization function. The global initialization
function will again invoke macro implementations which are free to redirect it
to any function of choice.
That way we can maintain variables in modules that they are but still put them
in a single compile unit when they get compiled. Only one file b.h, need to be
maintained for addition of new files. The variables would be taken care of in
the respective modules. ( This could nicely be done in C++ using classes but
thats a
altogether different subject.)
The changes to actual C code obviously need to follow.
I agree that's messy and non transparent. but it still retains distributed
nature of variable declarations and maintenance thereof..
But I feel we are missing the point. Last time threads for unix builds were
discussed, one of the good suggestion was to use them to distribute load across
CPU's rather than using thread per connection. Personally I don't think
postgresql is missing lot of performance by using process/connection rather than
thread/connection(Except for slowaris and windows perhaps).
Even on windows, the time taken for a new connection is unlikely to be an issue,
I feel.
We really don't need threads to replace existing functionality. That would be
dog work.
If we are writing new functionality, we can do it clean and portable way rather
than fiddling with existing code. Changing working code is always a messy
business irrespective of technicalities involved. As a support engineer, I can
attest that.
Shridhar
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Threads vs Processes
Shridhar Daithankar [EMAIL PROTECTED] writes: We really don't need threads to replace existing functionality. That would be dog work. No, that's not the point at all. The problem we are facing at the moment with the Windows port is lack of fork(), which means there's no way for separate-subprocess backends to inherit variable values from the postmaster. Bruce has been trying to fix that by having the subprocesses somehow reload or re-deduce all those variables; which is messy, bug-prone, and probably race-condition-prone too. In a threaded implementation it would maybe be relatively easy to initialize a new thread's TLS by copying the postmaster thread's TLS, in which case a whole pile of as-yet-unwritten Windows-only code won't be needed. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Threads vs Processes
On Friday 26 September 2003 20:19, Tom Lane wrote: Shridhar Daithankar [EMAIL PROTECTED] writes: We really don't need threads to replace existing functionality. That would be dog work. No, that's not the point at all. The problem we are facing at the moment with the Windows port is lack of fork(), which means there's no way for separate-subprocess backends to inherit variable values from the postmaster. Bruce has been trying to fix that by having the subprocesses somehow reload or re-deduce all those variables; which is messy, bug-prone, and probably race-condition-prone too. In a threaded implementation it would maybe be relatively easy to initialize a new thread's TLS by copying the postmaster thread's TLS, in which case a whole pile of as-yet-unwritten Windows-only code won't be needed. Umm.. I understand child process created by createProcess does not inherit variable values from parent process. That's where problem originates.. We can simply create a registry key that would contain shared memory id from where a child process should get the variable values. And that would need initialization function I talked about earlier. And since anyways TLS-TLS copy is still needed anyways, I think this approach can still save us dealing with threads. God.. it doesn't get any less messy..I hope this is of some value.. Shridhar ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Threads vs Processes
Tom Lane wrote: Shridhar Daithankar [EMAIL PROTECTED] writes: We really don't need threads to replace existing functionality. That would be dog work. No, that's not the point at all. The problem we are facing at the moment with the Windows port is lack of fork(), which means there's no way for separate-subprocess backends to inherit variable values from the postmaster. Bruce has been trying to fix that by having the subprocesses somehow reload or re-deduce all those variables; which is messy, bug-prone, and probably race-condition-prone too. In a threaded implementation it would maybe be relatively easy to initialize a new thread's TLS by copying the postmaster thread's TLS, in which case a whole pile of as-yet-unwritten Windows-only code won't be needed. I haven't said much in this thread yet because I wasn't sure what to say. I worked 2-3 weeks on Win32 back in March/April, and a few days last month getting it to compile again, and for the interfaces/clients to run under Win32. I haven't had any time to work on the fork/exec, though I now have a few weeks to spend on it, so it isn't that I have been trying to get fork/exec working, and failing, it is that I haven't even tried lately. My plan is to pass a few values to the child via the command line: #ifdef EXEC_BACKEND Assert(UsedShmemSegID != 0 UsedShmemSegAddr != NULL); /* database name at the end because it might contain commas */ snprintf(pbuf, NAMEDATALEN + 256, %d,%d,%d,%p,%s, port-sock, canAcceptConnections(), UsedShmemSegID, UsedShmemSegAddr, port-database_name); av[ac++] = pbuf; #else and pass the GUC settings via a special binary file. (Those are already in main CVS.) The other values I plan to regenerate in the child the same way the postmaster does it at initialization time. The easy part of that is that I only have to worry about postmaster variables. All my current fork/exec work is marked by #ifdef EXEC_BACKEND in current CVS, so it can be easily ripped out. One solution is for me to continue with this in the Win32 CVS version until I have fork/exec() working on Unix, then test on Win32. I think that could be done in a few weeks, if not less. Another solution, already mentioned, is to use threads and TLS. This is what SRA's code uses. I know SRA wants to contribute that code back to the community, so I can ask them to see if they are ready to release it. That would show us all the changes needed to do threading. Their code is based on 7.3.X, rather than PeerDirect's which is based on 7.2. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Threads vs Processes
Bruce Momjian [EMAIL PROTECTED] writes: One solution is for me to continue with this in the Win32 CVS version until I have fork/exec() working on Unix, then test on Win32. I think that could be done in a few weeks, if not less. Another solution, already mentioned, is to use threads and TLS. This is what SRA's code uses. I know SRA wants to contribute that code back to the community, so I can ask them to see if they are ready to release it. If you are willing to expend the effort, I think it would be worth the time to pursue both approaches. We don't yet have enough info to decide which one will be cleaner, so we need to push forward on both until we can make a realistic comparison. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Threads vs Processes
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: One solution is for me to continue with this in the Win32 CVS version until I have fork/exec() working on Unix, then test on Win32. I think that could be done in a few weeks, if not less. Another solution, already mentioned, is to use threads and TLS. This is what SRA's code uses. I know SRA wants to contribute that code back to the community, so I can ask them to see if they are ready to release it. If you are willing to expend the effort, I think it would be worth the time to pursue both approaches. We don't yet have enough info to decide which one will be cleaner, so we need to push forward on both until we can make a realistic comparison. I think I know enough to get the fork/exec working, particularly because most of the work can be tested under Unix. I don't know enough to get the threads working, and I have had no offers of help since I started. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Threads vs Processes
-Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2003 9:27 AM To: Bruce Momjian Cc: Shridhar Daithankar; [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [HACKERS] Threads vs Processes Bruce Momjian [EMAIL PROTECTED] writes: One solution is for me to continue with this in the Win32 CVS version until I have fork/exec() working on Unix, then test on Win32. I think that could be done in a few weeks, if not less. Another solution, already mentioned, is to use threads and TLS. This is what SRA's code uses. I know SRA wants to contribute that code back to the community, so I can ask them to see if they are ready to release it. If you are willing to expend the effort, I think it would be worth the time to pursue both approaches. We don't yet have enough info to decide which one will be cleaner, so we need to push forward on both until we can make a realistic comparison. I think the ideal situation would be a server that both threads and forks. Sometimes, we want the server to take on a new personality (with the rights of the attaching user). In such a case, threading is not a sufficient answer. Also, a forked process is a bit safer (though you can put a try/catch around threads). For performance with multiple queries from a single user, threads are going to be faster than forking. I think that the best model will be a server that does both. I know that there is some aversion to using C++, but the ACE framework offers a consistent threading model that works for just about every computer under the sun. http://www.cs.wustl.edu/~schmidt/ACE.html The license is basically BSD (it is called ACE, but it works exactly like a BSD license). With ACE, you program to a single API, and the code works the same on every platform with a simple recompile. It might even be worthwhile to use the ACE higher level components to create a server that supports multiple models of connection and threading. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Threads vs Processes
We really don't need threads to replace existing functionality. That would be dog work. No, that's not the point at all. The problem we are facing at the moment with the Windows port is lack of fork(), which means there's no way for separate-subprocess backends to inherit variable values from the postmaster. Bruce has been trying to fix that by having the subprocesses somehow reload or re-deduce all those variables; which is messy, bug-prone, and probably race-condition-prone too. In a threaded implementation it would maybe be relatively easy to initialize a new thread's TLS by copying the postmaster thread's TLS, in which case a whole pile of as-yet-unwritten Windows-only code won't be needed. Kepp in mind though all the cool things that could be done if we had thread capabilities. eg. evaluating different subexpressings on fdifferent cpuis for the one query, etc. Chris ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Threads vs Processes (was: NuSphere and PostgreSQL for windows)
Claudio Natoli [EMAIL PROTECTED] writes: FWIW, I've got a threaded version of the WIN32_DEV branch more or less running (it is a terrible hack job, so NO, no patches... yet :-), as a proof of concept. Still a work in progress (ok, I've qualified it enough), but it is showing enough promise to convince me that threading is the way to go for the Win32 port. How are you dealing with the issue of wanting some static variables to be per-thread and others not? regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Threads vs Processes (was: NuSphere and PostgreSQL for window
Claudio Natoli [EMAIL PROTECTED] writes: FWIW, I've got a threaded version of the WIN32_DEV branch more or less running (it is a terrible hack job, so NO, no patches... yet :-), as a proof of concept. Still a work in progress (ok, I've qualified it enough), but it is showing enough promise to convince me that threading is the way to go for the Win32 port. How are you dealing with the issue of wanting some static variables to be per-thread and others not? regards, tom lane To be perfectly honest, I'm still trying to familiarize myself with the code sufficiently well so that I can tell which variables need to be per-thread and which are shared (and, in turn, which of these need to be protected from concurrent access). So, in short, I'm not dealing with the issue (and, hence, it is only running in the very loosest sense of the word). Unfortunately, I'm not yet even in a position to propose a reasonable model, let alone one that'd play well with the existing code base. Perhaps some time soon, hopefully... (I really want to get involved in this.) Cheers, Claudio --- WE HAVE MOVED - PLEASE NOTE OUR NEW CONTACT DETAILS: THE BASEMENT, 33 EWELL STREET, BALMAIN NSW 2041 TEL: +61 2 9555 1544 FAX: +61 2 9555 6911 Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see http://www.memetrics.com/emailpolicy.html ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Threads vs Processes (was: NuSphere and PostgreSQL for window s)
Claudio Natoli [EMAIL PROTECTED] writes: How are you dealing with the issue of wanting some static variables to be per-thread and others not? To be perfectly honest, I'm still trying to familiarize myself with the code sufficiently well so that I can tell which variables need to be per-thread and which are shared (and, in turn, which of these need to be protected from concurrent access). Well, the first-order approximation would be to duplicate the current fork semantics: *all* static variables are per-thread, and should be copied from the parent thread at thread creation. If there is some reasonably non-invasive way to do that, we'd have a long leg up on the problem. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Threads vs Processes
Tom Lane wrote: Claudio Natoli [EMAIL PROTECTED] writes: How are you dealing with the issue of wanting some static variables to be per-thread and others not? To be perfectly honest, I'm still trying to familiarize myself with the code sufficiently well so that I can tell which variables need to be per-thread and which are shared (and, in turn, which of these need to be protected from concurrent access). Well, the first-order approximation would be to duplicate the current fork semantics: *all* static variables are per-thread, and should be copied from the parent thread at thread creation. If there is some reasonably non-invasive way to do that, we'd have a long leg up on the problem. Hmm.. I was looking for some fast tutorials on thread local storage. I found this one.. http://publib16.boulder.ibm.com/pseries/en_US/aixprggd/genprogc/thread_specific_data.htm Basically, in a process we are free to declare as many globals as we can. Converting them to thread local is not an easy job because each variable would need it's own key and there is limit on how many keys can be allocated. One thing that can be done is to arrange all globals/statics in a structure and make that structure thread local. We need to change all invocations of any of those variables to use a pointer. We just need only one global variable. And some macro trickery possibly so that we can extend that structure easily and automatically. Upshot is duplicating environment is easy. We need to do a huge memcpy and any specific depp copy of strings on thread creation. Besides even in process model, this kind of initialization will allow to put all variables on heap instead of stack. But then we need to add initialization code explicitly. Something like int a=10; can not be added just like that. If globals are less than 100 in numbers, I think it should be reasonably blind job of converting them to a structure type stuff. Don't know really though. My estimations are always 10% of what it takes..:-) I hope I got it correct.. Shridhar ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Threads vs Processes (was: NuSphere and PostgreSQL
Tom Lane wrote: Claudio Natoli [EMAIL PROTECTED] writes: How are you dealing with the issue of wanting some static variables to be per-thread and others not? To be perfectly honest, I'm still trying to familiarize myself with the code sufficiently well so that I can tell which variables need to be per-thread and which are shared (and, in turn, which of these need to be protected from concurrent access). No. Not protected from concurrent access. Each thread must have it's own copy. Well, the first-order approximation would be to duplicate the current fork semantics: *all* static variables are per-thread, and should be copied from the parent thread at thread creation. If there is some reasonably non-invasive way to do that, we'd have a long leg up on the problem. There is a declspec(thread) that makes a global variable per-thread. AFAIK it uses linker magic to replace the actual memory accesses with calls to TlsAlloc() etc. Note that declspec(thread) doesn't work from within dynamic link libraries, but that shouldn't be a big problem. -- Manfred ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Threads vs Processes (was: NuSphere and PostgreSQL for windows)
Typically variables that you want to be per-thread are stored in what Microsoft calls Thread Local Storage (TLS). Variables that you want shared you can just treat as globals and statics with the appropriate threading synchronization primitives. With Windows 2000 and later you have up to 1088 TLS locations that you can use, of course these can be pointers to memory which can store whatever you want. Keith -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom Lane Sent: Thursday, September 25, 2003 9:17 AM To: Claudio Natoli Cc: Robert Treat; Bruce Momjian; [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [HACKERS] Threads vs Processes (was: NuSphere and PostgreSQL for windows) Claudio Natoli [EMAIL PROTECTED] writes: FWIW, I've got a threaded version of the WIN32_DEV branch more or less running (it is a terrible hack job, so NO, no patches... yet :-), as a proof of concept. Still a work in progress (ok, I've qualified it enough), but it is showing enough promise to convince me that threading is the way to go for the Win32 port. How are you dealing with the issue of wanting some static variables to be per-thread and others not? regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Threads vs Processes (was: NuSphere and PostgreSQL for
Keith Bottner wrote: Typically variables that you want to be per-thread are stored in what Microsoft calls Thread Local Storage (TLS). Variables that you want shared you can just treat as globals and statics with the appropriate threading synchronization primitives. With Windows 2000 and later you have up to 1088 TLS locations that you can use, of course these can be pointers to memory which can store whatever you want. Goes GCC on Windows support TLS, or only Microsoft compilers? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Threads vs Processes
Shridhar Daithankar [EMAIL PROTECTED] writes: One thing that can be done is to arrange all globals/statics in a structure and make that structure thread local. That's about as far from non-invasive as I can imagine :-( I really, really want to avoid doing anything like the above, because it would force us to expose to the whole backend many data structures and state variables that are currently local to individual .c files. That complicates understanding and debugging tremendously, not to mention slowing the edit/compile/debug cycle when you are changing such structures. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Threads vs Processes
Both Microsoft and windows compilers support thread local storage. *If* you guys go with the threading model and *if* it does not introduce any serious portability issues with gcc (big ifs, and I'm not familiar with gcc), than IMO TLS is really the way to go. I don't think any reorganization of postgres's static variables is necessary. TLS is implemented in the win32 API, not the C Libs, so by giving up the syntax sugar you can make direct calls and keep compiler independence in win32. Microsoft syntax is __desclspec(thread) and Borland syntax is simply __thread. All TLS variables *must* be static (or implicitly static through extern, i.e. no 'auto' variables) and their addresses can not be assumed to be constant. Taking addresses of TLS variables should be considered illegal, as well as pointers to TLS variables. Another gotcha is that DLLs that have __thread variables will GPF if loaded with LoadLibrary (they should be static linked). Of course, pg does not use dlls, but it's worth noting. Merlin ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Threads vs Processes
Actually you can use a DLL with LoadLibrary as long as you do the following.
When a process uses load-time linking with this DLL, the entry-point
function is sufficient to manage the thread local storage. Problems can
occur with a process that uses run-time linking because the entry-point
function is not called for threads that exist before the LoadLibrary
function is called, so TLS memory is not allocated for these threads. The
following example solves this problem by checking the value returned by the
TlsGetValue function and allocating memory if the value indicates that the
TLS slot for this thread is not set.
LPVOID lpvData;
// Retrieve a data pointer for the current thread.
lpvData = TlsGetValue(dwTlsIndex);
// If NULL, allocate memory for this thread.
if (lpvData == NULL)
{
lpvData = (LPVOID) LocalAlloc(LPTR, 256);
if (lpvData != NULL)
TlsSetValue(dwTlsIndex, lpvData);
}
Unless gcc has extension as Borland and Microsoft do you will have to
utilize the API and not the compiler/linker customizations that make access
variables declared as __declspec(thread) or __thread under Borland.
Keith
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Merlin Moncure
Sent: Thursday, September 25, 2003 2:49 PM
To: Tom Lane
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; Bruce
Momjian; Shridhar Daithankar; Claudio Natoli
Subject: Re: [HACKERS] Threads vs Processes
Both Microsoft and windows compilers support thread local storage. *If* you
guys go with the threading model and *if* it does not introduce any serious
portability issues with gcc (big ifs, and I'm not familiar with gcc), than
IMO TLS is really the way to go. I don't think any reorganization of
postgres's static variables is necessary. TLS is implemented in the win32
API, not the C Libs, so by giving up the syntax sugar you can make direct
calls and keep compiler independence in win32.
Microsoft syntax is __desclspec(thread) and Borland syntax is simply
__thread. All TLS variables *must* be static (or implicitly static through
extern, i.e. no 'auto' variables) and their addresses can not be assumed to
be constant.
Taking addresses of TLS variables should be considered illegal, as well as
pointers to TLS variables. Another gotcha is that DLLs that have __thread
variables will GPF if loaded with LoadLibrary (they should be static
linked). Of course, pg does not use dlls, but it's worth noting.
Merlin
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
Re: [HACKERS] Threads vs Processes
On Thursday, September 25, 2003, at 10:03 AM, Tom Lane wrote: Shridhar Daithankar [EMAIL PROTECTED] writes: One thing that can be done is to arrange all globals/statics in a structure and make that structure thread local. That's about as far from non-invasive as I can imagine :-( I really, really want to avoid doing anything like the above, because it would force us to expose to the whole backend many data structures and state variables that are currently local to individual .c files. That complicates understanding and debugging tremendously, not to mention slowing the edit/compile/debug cycle when you are changing such structures. Another option would be to create thread local hashtable or other lookup structure to which you would register a structure for a particular .c file or group of files. You could then define the structures you need locally without affecting other parts of the codebase. Myron Scott ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Threads vs Processes
Merlin Moncure [EMAIL PROTECTED] writes: All TLS variables *must* be static (or implicitly static through extern, i.e. no 'auto' variables) I assume you mean static as in not-auto, rather than static as in not-global. Otherwise we have a problem here. and their addresses can not be assumed to be constant. Surely the addresses can be assumed constant within a thread. Otherwise we have a problem here too. Taking addresses of TLS variables should be considered illegal, Sorry, no can accept that restriction. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Threads vs Processes
Tom Lane wrote: I assume you mean static as in not-auto, rather than static as in not-global. Otherwise we have a problem here. [...] Surely the addresses can be assumed constant within a thread. Otherwise we have a problem here too. [...] Taking addresses of TLS variables should be considered illegal, Sorry, no can accept that restriction. I think you are okay on all 3 fronts, from http://gcc.gnu.org/onlinedocs/gcc/Thread-Local.html#Thread-Local : The __thread specifier may be used alone, with the extern or static specifiers, but with no other storage class specifier. When used with extern or static, __thread must appear immediately after the other storage class specifier. and When the address-of operator is applied to a thread-local variable, it is evaluated at run-time and returns the address of the current thread's instance of that variable. An address so obtained may be used by any thread. When a thread terminates, any pointers to thread-local variables in that thread become invalid. Also see ISO/IEC 9899:1999 Edits for Thread-Local Storage : http://gcc.gnu.org/onlinedocs/gcc/C99-Thread-Local-Edits.html#C99%20Thread-Local%20Edits and ELF Handling For Thread-Local Storage, http://people.redhat.com/drepper/tls.pdf may be of interest. Cheers, Kurt. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Threads vs Processes
Another option would be to create thread local hashtable or other lookup structure to which you would register a structure for a particular .c file or group of files. You could then define the structures you need locally without affecting other parts of the codebase. Myron Scott A slight variant on this idea would be to mask the variables themselves, using macrology on a hash keyed on the threadID. To illustrate: MyProc-errType would, on a threaded system, become something like: ((PGProc*)ThreadLocalHash(MyProc,GetCurrentThreadID()))-errType I'm imagining that an approach like this would fit in very nicely with the existing code. Not the most performance friendly solution however (and I'm guessing that there might be a place or two where this might be important :-). Just a thought, Claudio --- WE HAVE MOVED - PLEASE NOTE OUR NEW CONTACT DETAILS: THE BASEMENT, 33 EWELL STREET, BALMAIN NSW 2041 TEL: +61 2 9555 1544 FAX: +61 2 9555 6911 Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see http://www.memetrics.com/emailpolicy.html ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Threads vs processes - The Apache Way (Re: Path to PostgreSQL
Robert wrote: Hi, Win32 threads support are both going to be a lot of work and maybe we'll need in the future one or both - is there any chance Postgres developers look at the Apache experience? Briefly, Apache 2 had the some problems as are discussed here (need to support Win, problems with Win32 fork, questionable cygwin etc) and they decided to solve it once and for all with their Apache Portable Runtime and Multi-Processing Modules. APR was already mentioned here - now how about MPMs? I am starting to come to the conclusion that the PostgreSQL group is satisfied with cygwin, and the will to create a native Win32 version does not exist outside of a few organizations that are paying developers to create one. Without some buy-in from the core team, I'm not sure I am willing to spend my time on it. If someone would be willing to fund the 100 or so man-hours required to do it, then that would be a different story. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Threads vs processes - The Apache Way (Re: Path to PostgreSQL
mlw [EMAIL PROTECTED] writes: Without some buy-in from the core team, I'm not sure I am willing to spend my time on it. If someone would be willing to fund the 100 or so man-hours required to do it, then that would be a different story. You are not going to get any buy-in with such ridiculous claims as that. If the total cost of a native Windows port were O(100 hours), it'd have been done long since. Add a couple zeroes on the end and I'd start to believe that you might have some grasp of the problem. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Threads vs processes - The Apache Way (Re: Path to PostgreSQL
Tom Lane wrote: mlw [EMAIL PROTECTED] writes: Without some buy-in from the core team, I'm not sure I am willing to spend my time on it. If someone would be willing to fund the 100 or so man-hours required to do it, then that would be a different story. You are not going to get any buy-in with such ridiculous claims as that. If the total cost of a native Windows port were O(100 hours), it'd have been done long since. Add a couple zeroes on the end and I'd start to believe that you might have some grasp of the problem. I was basing my estimates on a couple things. Please feel free to correct me where I'm wrong. Dann Corbit mentioned a number of, I think I recall, a couple hundred man-hours for their port. My approach would be to find all the global variables setup by postmaster, not all the globals, mind you. Just the ones initialized by postmaster. Move them to a structure. That structure would be capable of being copied to the child process. In the area where forking the postgres process happens, I would ifdef that area with an HAS_FORK The Windows portion would use CreateProcess. The Windows version of postgres would contact the postmaster and get its copy of the globals struct. The code to transfer ownership of sockets, files, and memory would have to be written also. I would only minimally change the back-end code, it would still be built with cygwin tools only directed not to link against the cygwin.dll. (The same goes for the utilities as well.) A thin port layer could then be constructed by either implementing sysv/UNIX replacements, or a more simple API as needed in the code, like your shared memory and semaphore APIs. Does that sound like an unworkable plan? ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
