AW: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> The other category is run-time behaviour, which is the present case of > testing whether mktime() behaves a certain way when given certain > arguments. Another item that has been thrown around over the years is > whether the system supports Unix domain sockets (essentially a run-time > behaviour check of socket()). You do not need to check these things in > configure; you can just do it when the program runs and adjust > accordingly. I think in PostgreSQL there is also a third category: "features" that affect on disk storage of data. Those can definitely not be changed at runtime, because a change would corrupt that database. It is very hard to judge what "feature" really has an on disk effect. It can e.g. be an application that selected a row that was inserted before the feature existed, do some calculations and insert a new row with the feature now available. Thus I think available features should be as immune to OS feature changes as possible, but let us postpone that discussion until after 7.1 release. > So I would currently suggest that you define > > #ifdef AIX || IRIX > # define NO_DST_BEFORE_1970 > #endif That would be OK for me. Where do we put that ? Imho a good place would be config.h. One problem though is, that I don't have a patch, that really turns off DST before 1970 (which would be considerably more work). I only have a patch that makes platforms, that don't have a working mktime for values before 1970, behave like NO_DST_... Thus more wanting a #define NO_MKTIME_BEFORE_1970. Andreas
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Peter Eisentraut writes: > What if someone has a binary PostgreSQL package installed, then > updates his time library to something supposedly binary compatible > and finds out that PostgreSQL still doesn't use the enhanced > capabilities? You are too generous. If someone downloads a binary package it should not be expected to be able to take advantage of non standard features of the platform. It is reasonable that they should compile it from source to get the most from it. -- Pete Forman -./\.- Disclaimer: This post is originated WesternGeco -./\.- by myself and does not represent [EMAIL PROTECTED] -./\.- opinion of Schlumberger, Baker http://www.crosswinds.net/~petef -./\.- Hughes or their divisions.
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Peter Eisentraut <[EMAIL PROTECTED]> writes: > > > This would seem to be the right answer, but unfortunately Autoconf is not > > > smart enough to detect marginal cross-compilation cases in all situations. > > > Someone had zlib installed in a location where gcc would find it (compiles > > > okay) but the run-time linker would not (does not run). This is not > > > detected when AC_PROG_CC runs, but only later on after you have checked > > > for the libraries. > > > > Hmmm. I would not describe that as a cross-compilation case at all. > > The build machine and the host machine are the same. > > Only for small values of "same". ;-) Sameness is not defined by the names > being spelled identically or the physical coincidence of the hardware. > There are a million things you can do to a system that supposedly preserve > binary compatibility, such as installing vendor patches or changing a > setting in /etc. But if you run a test program is such a situation you're > testing the wrong system. I believe that terminology is important in technical issues, and in any case I'm pedantic by nature. So I am going to disagree. When you build a program on the system on which it is going to run, that is not cross-compilation. I agree that if you build a program on one system, and then copy it (either physically or via something like NFS) to another system, and run it on that other system, then that may be cross-compilation. It may be that the program in this case was being moved from one machine to another, and I missed it. However, unless that was the case, a difference in search path between the compiler and the runtime linker is not a cross-compilation issue. A difference in where a library appears from one machine to another is a cross-compilation issue. I believe that it is important to describe problems correctly in order to understand how to fix them correctly. If the problem is ``autoconf does not correctly diagnose a case of cross-compilation,'' that suggests one sort of fix. If the problem is ``the compiler and the runtime linker have different search paths,'' that suggests a different sort of fix. To return to the issue which started this thread, I believe it was an issue concerning the behaviour of the mktime library call. The behaviour of that library call is not going to be affected by the location of a shared library. The approach I suggested should suffice to address the particular problem at hand. Do you disagree? The issue in which the compiler and the runtime linker use different search paths is fixed by either changing the compiler search path, to by default only choose shared libraries from directories which the runtime linker will search, or by changing the library installation procedure, either put the library in /usr/lib, or on systems which support ld.so.conf arrange to put the directory in ld.so.conf. Note that if you link with -lz, at least some versions of the GNU linker will issue a warning that the library will not be found by the runtime linker. Ian
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Ian Lance Taylor writes: > > This would seem to be the right answer, but unfortunately Autoconf is not > > smart enough to detect marginal cross-compilation cases in all situations. > > Someone had zlib installed in a location where gcc would find it (compiles > > okay) but the run-time linker would not (does not run). This is not > > detected when AC_PROG_CC runs, but only later on after you have checked > > for the libraries. > > Hmmm. I would not describe that as a cross-compilation case at all. > The build machine and the host machine are the same. Only for small values of "same". ;-) Sameness is not defined by the names being spelled identically or the physical coincidence of the hardware. There are a million things you can do to a system that supposedly preserve binary compatibility, such as installing vendor patches or changing a setting in /etc. But if you run a test program is such a situation you're testing the wrong system. > I would describe that as a case where the compiler search path and the > run time library search path are not the same. The assumption is surely that the user would set LD_LIBRARY_PATH or configure his linker before running the program. But nothing guarantees that he'll actually set the search path to /usr/local/lib, which is what gcc was searching in this situation. > Clearly differences between search paths matter, but they are not the > same as cross-compilation. It's not the same as the classical Canadian Cross, but it's still cross enough to concern me. ;-) -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Ian Lance Taylor writes: > > > An approach I've followed in the past is to use three-way logic. If > > configuring for a native system, compile and run a program which > > provides a yes or no answer. When using cross-configuration, set the > > configuration variable to ``don't know'' (or, since this a database > > group, NULL). > > This would seem to be the right answer, but unfortunately Autoconf is not > smart enough to detect marginal cross-compilation cases in all situations. > Someone had zlib installed in a location where gcc would find it (compiles > okay) but the run-time linker would not (does not run). This is not > detected when AC_PROG_CC runs, but only later on after you have checked > for the libraries. Hmmm. I would not describe that as a cross-compilation case at all. The build machine and the host machine are the same. I would describe that as a case where the compiler search path and the run time library search path are not the same. The autoconf tests don't use any extra libraries, so any discrepancies of this sort should not matter to them. Autoconf tests whether it can compile and run a simple program; if it can, it assumes that it is not in a cross-compilation situation. Clearly differences between search paths matter, but they are not the same as cross-compilation. Ian
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Ian Lance Taylor writes: > An approach I've followed in the past is to use three-way logic. If > configuring for a native system, compile and run a program which > provides a yes or no answer. When using cross-configuration, set the > configuration variable to ``don't know'' (or, since this a database > group, NULL). This would seem to be the right answer, but unfortunately Autoconf is not smart enough to detect marginal cross-compilation cases in all situations. Someone had zlib installed in a location where gcc would find it (compiles okay) but the run-time linker would not (does not run). This is not detected when AC_PROG_CC runs, but only later on after you have checked for the libraries. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Peter Eisentraut <[EMAIL PROTECTED]> writes: > More importantly, you *should* *not* do these tests in configure because > these tests will be unreliable in a cross-compilation situation. > Cross-compilation in this context does not only mean compiling between > completely different platforms, but it includes any setup where the build > system is configured differently from the system you're going to run the > system on, including building on a noexec file system, misconfigured > run-time linkers, different user id, or just a different file system > layout on an otherwise identical platform. An approach I've followed in the past is to use three-way logic. If configuring for a native system, compile and run a program which provides a yes or no answer. When using cross-configuration, set the configuration variable to ``don't know'' (or, since this a database group, NULL). Pull all code which needs to make this test into a single routine. Let it check the configuration variable. If the variable is ``don't know,'' then compile in a static variable, and run the required test once, at run time, and set the static variable accordingly. Then test the static variable in all future calls. This is the approach I used in my UUCP code to look for a bad version of ftime on some versions of SCO Unix--the ftime result would sometimes run backward. Ian Co-author of GNU Autoconf, Automake, and Libtool
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Thomas Lockhart writes: > I'm not sure I fully appreciate the distinction here. configure will > check for "behavior" of various kinds, including "behavior assumptions" > in the PostgreSQL code. There are two general categories of platform dependencies: One is compilation system features, for example presence of certain functions, libraries, header files, argument types, compiler flags. These sort of things must be determined before compilation can begin. Autoconf is used to test for these things. The other category is run-time behaviour, which is the present case of testing whether mktime() behaves a certain way when given certain arguments. Another item that has been thrown around over the years is whether the system supports Unix domain sockets (essentially a run-time behaviour check of socket()). You do not need to check these things in configure; you can just do it when the program runs and adjust accordingly. More importantly, you *should* *not* do these tests in configure because these tests will be unreliable in a cross-compilation situation. Cross-compilation in this context does not only mean compiling between completely different platforms, but it includes any setup where the build system is configured differently from the system you're going to run the system on, including building on a noexec file system, misconfigured run-time linkers, different user id, or just a different file system layout on an otherwise identical platform. I'm not making these things up. Just yesterday there was a message on this list where someone ran into this problem and his configure run misbehaved badly. (PostgreSQL currently violates these rules, but that does not mean that we should make it harder now to clean it up at some later date.) Admittedly, these situations sound somewhat exotic, but that does not mean they do not happen, it may merely mean that PostgreSQL is not fit to perform in these situations. > So the issue for this case is simply whether it is appropriate to check > for behavior at run time on all platforms, even those known to "never" > exhibit this problematic result, You can make the run-time check #ifdef one or the other platform. Or you don't check at all and just postulate the misfeature for a set of platforms. We have done this in at least two cases: The list of platforms known not to support Unix domain sockets (HAVE_UNIX_SOCKETS), and the list of platforms known to have a peculiar bug in the accept() implementation (happen to be both from SCO). I think this is an appropriate solution in cases where the affected systems can be classified easily. But as soon as some versions of some of these systems start implementing Unix sockets (which is indeed the situation we're currently facing) or SCO fixes their kernels we're going to have to think harder. So I would currently suggest that you define #ifdef AIX || IRIX # define NO_DST_BEFORE_1970 #endif but when these systems get fixed you might have to run a simple check once when the system starts up. This is not really a "hit". > Andreas, Pete and I were advocating the latter view: that the majority > of platforms which happen to be well behaved should run code optimized > for them, while the bad actors can be supported without "#ifdef __AIX__ > || __IRIX__" in our code, but rather with a more helpful "#ifdef > NO_DST_BEFORE_1970" or whatever. Yeah. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> Exactly. What if someone has a binary PostgreSQL package installed, then > updates his time library to something supposedly binary compatible and > finds out that PostgreSQL still doesn't use the enhanced capabilities? > Runtime behaviour checks are done at runtime, it's as simple as that. I'm not sure I fully appreciate the distinction here. configure will check for "behavior" of various kinds, including "behavior assumptions" in the PostgreSQL code. So the issue for this case is simply whether it is appropriate to check for behavior at run time on all platforms, even those known to "never" exhibit this problematic result, or whether we put in a configure-time check to save the day for the (two) platforms known to have trouble -- without the other supported platforms taking the hit by having to check even though the check will never fail. Andreas, Pete and I were advocating the latter view: that the majority of platforms which happen to be well behaved should run code optimized for them, while the bad actors can be supported without "#ifdef __AIX__ || __IRIX__" in our code, but rather with a more helpful "#ifdef NO_DST_BEFORE_1970" or whatever. - Thomas
AW: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> > I agree that configure is the way to go. What if someone has > > installed a third party library to provide a better mktime() and > > localtime()? > > Exactly. What if someone has a binary PostgreSQL package installed, then > updates his time library to something supposedly binary compatible and > finds out that PostgreSQL still doesn't use the enhanced capabilities? > Runtime behaviour checks are done at runtime, it's as simple as that. So, you are suggesting to call mktime after every call to localtime, just to check whether to output the time in DST before 1970 on all platforms ? Beleive me, we need a configure check. Besides, mktime is in libc on AIX and all available versions show the same behavior, thus I do not think that it is going to change soon. The only way would be to link with a third party lib, and that would definitely need a new configure run anyway. This configure is needed to avoid an otherwise necessary #if defined(_AIX) || defined(__sgi) If, and only if, you say the above is better then we don't need a configure check. Andreas
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Pete Forman writes: > I agree that configure is the way to go. What if someone has > installed a third party library to provide a better mktime() and > localtime()? Exactly. What if someone has a binary PostgreSQL package installed, then updates his time library to something supposedly binary compatible and finds out that PostgreSQL still doesn't use the enhanced capabilities? Runtime behaviour checks are done at runtime, it's as simple as that. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
Zeugswetter Andreas SB writes: > The down side is, that I did not do a configure test, and did not > incooperate IRIX, since I didn't know what define to check. > > The correct thing to do instead of the #if defined (_AIX) would be > to use something like #ifdef NO_NEGATIVE_MKTIME and set that with a > configure. I agree that configure is the way to go. What if someone has installed a third party library to provide a better mktime() and localtime()? But to answer your question, #if defined (__sgi) is a good test for IRIX, at least with the native compiler. I can't speak for gcc. -- Pete Forman -./\.- Disclaimer: This post is originated WesternGeco -./\.- by myself and does not represent [EMAIL PROTECTED] -./\.- opinion of Schlumberger, Baker http://www.crosswinds.net/~petef -./\.- Hughes or their divisions.
AW: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> > The correct thing to do instead of the #if defined (_AIX) would be to use
> > something like #ifdef NO_NEGATIVE_MKTIME and set that with a configure.
> > Thomas, are you volunteering ?
>
> Actually, I can volunteer to be supportive of your efforts ;) I'm
> traveling at the moment, and don't have the original thread(s) which
> describe in detail what we need to do for platforms I don't have.
>
> If Peter E. would be willing to do a configure test for this mktime()
> problem, then you or I can massage the actual code. Peter, is this
> something you could pick up?
>
> I do not have the original thread where Andreas describes the behavior
> of mktime() on his machine. Andreas, can you suggest a simple configure
> test to be used?
#include
int main()
{
struct tm tt, *tm=&tt;
int i = -5000;
tm = localtime (&i);
i = mktime (tm);
if (i != -5000) /* on AIX this check could also be (i == -1) */
{
printf("ERROR: mktime(3) does not correctly support datetimes before 1970\n");
return(1);
}
}
Andreas
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> The correct thing to do instead of the #if defined (_AIX) would be to use > something like #ifdef NO_NEGATIVE_MKTIME and set that with a configure. > Thomas, are you volunteering ? Actually, I can volunteer to be supportive of your efforts ;) I'm traveling at the moment, and don't have the original thread(s) which describe in detail what we need to do for platforms I don't have. If Peter E. would be willing to do a configure test for this mktime() problem, then you or I can massage the actual code. Peter, is this something you could pick up? I do not have the original thread where Andreas describes the behavior of mktime() on his machine. Andreas, can you suggest a simple configure test to be used? - Thomas
AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> > > > > Yes, the annoyance is, that localtime works for dates before 1970 > > > > > but mktime doesn't. Best would probably be to assume no DST before > > > > > 1970 on AIX and IRIX. > > > > > > > > That seems like a reasonable answer to me, especially since we have > > > > other platforms that behave that way. How can we do this --- just > > > > test for isdst = -1 after the call, and assume that means failure? Here is a patch that is incremental to the previous patch and makes AIX not use DST before 1970. The results are now consistent with other no-DST-before-1970 platforms. The down side is, that I did not do a configure test, and did not incooperate IRIX, since I didn't know what define to check. The correct thing to do instead of the #if defined (_AIX) would be to use something like #ifdef NO_NEGATIVE_MKTIME and set that with a configure. Thomas, are you volunteering ? Andreas aix_no-DST-before-1970.patch
AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> > > > Yes, the annoyance is, that localtime works for dates before 1970 > > > > but mktime doesn't. Best would probably be to assume no DST before > > > > 1970 on AIX and IRIX. > > > > > > That seems like a reasonable answer to me, especially since we have > > > other platforms that behave that way. How can we do this --- just > > > test for isdst = -1 after the call, and assume that means failure? > > Are you working on this, or can you point me to the parts of the code, > that would need change ? Here is a patch that should make AIX and IRIX more happy. It changes all checks for tm_isdst to (tm_isdst > 0) and fixes the expected horology file for AIX. I just now realized, that the new expected file (while still bogous) is more correct than the old one. Thanks to Tom for mentioning that the hour should stay the same when subtracting days from a timestamp. Please apply. Andreas aixisdst.patch
AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> > > Yes, the annoyance is, that localtime works for dates before 1970 > > > but mktime doesn't. Best would probably be to assume no DST before > > > 1970 on AIX and IRIX. > > > > That seems like a reasonable answer to me, especially since we have > > other platforms that behave that way. How can we do this --- just > > test for isdst = -1 after the call, and assume that means failure? Are you working on this, or can you point me to the parts of the code, that would need change ? Andreas
AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> > Yes, the annoyance is, that localtime works for dates before 1970 > > but mktime doesn't. Best would probably be to assume no DST before > > 1970 on AIX and IRIX. > > That seems like a reasonable answer to me, especially since we have > other platforms that behave that way. How can we do this --- just > test for isdst = -1 after the call, and assume that means failure? Since we have tests of the form (tm_isdst > 0) ? ... in other parts of the code this would imho be sufficient. BTW, I would use the above check for DST in all parts of the code. Currently we eighter have (tm_isdst ? ) or the above form. Andreas
AW: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> > As I see it, the Linux results are also not 100 % correct > in respect to dates > > before 1970. (based on assumption that Solaris is correct) > > <| Sat May 10 23:59:12 1947 PST > > >| Sat May 10 23:59:12 1947 PDT > > Was 1947 PDT or PST ? In eighter case one result is one hour off, Solaris or Linux. > > Yes, I've seen this before. As I mentioned, Solaris does a particularly > good job of including the variations in DST definitions around WWII and > earlier. One of the two results is still off by one hour if the noted hour is 23 in both the PDT and PST case. One issue is determining DST and the other is printing the hour. For a given time in history the hour can't be the same if printed in PDT or PST. Andreas
Re: AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> As I see it, the Linux results are also not 100 % correct in respect to dates > before 1970. (based on assumption that Solaris is correct) > <| Sat May 10 23:59:12 1947 PST > >| Sat May 10 23:59:12 1947 PDT > Was 1947 PDT or PST ? In eighter case one result is one hour off, Solaris or Linux. Yes, I've seen this before. As I mentioned, Solaris does a particularly good job of including the variations in DST definitions around WWII and earlier. In the US, there are several different conventions used in those years, presumably based on a need for energy conservation and perhaps to maximize production efficiency. > This raises another issue. Why do we distribute expected files with bogus results > in them? It depends on what you mean by "bogus". imho we should (and do) distribute "expected" files which reflect the results expected for a particular machine -- based on a careful analysis of the results from that machine from an expert, such as you are doing with AIX. Your results are incremental differences from some other "standard machine", which has also been carefully analyzed. By definition, the "standard machine" has been and is a Linux box, for the simple historical reason that this was my machine at the time that scrappy and I resurrected the regression tests several years ago. But it is a good choice for the standard, since that style of machine has a large installed base and the cost of entry for someone else wanting to participate is very low. If I understand the alternatives you are considering, the other choice is to distribute "expected" files which reflect what we think a machine should produce if it behaved the way we think it should. That doesn't really help anyone, since a tester would have to derive from first principles the correctness of the test results each time they run the test. Instead, we document the current behavior, and the regression tests can now catch *changes* in behavior, to be analyzed just as you are doing. If AIX fixes their mktime() and timezone behavior, you (or someone else running AIX) will see it, evaluate it, and adjust the regression "expected" files to reflect this. - Thomas
AW: AW: AW: [HACKERS] Re: tinterval - operator problems on AIX
> > > > On AIX mktime(3) leaves tm_isdst at -1 if it does not have timezone > > > > info for that particular year and returns -1. > > > > The following code then makes savings time out of the -1. > > > > tz = (tm->tm_isdst ? (timezone - 3600) : timezone); > > > Hmm. That description is consistant with what I see in the Linux man > > > page. So I should check for (tm->tm_isdst > 0) rather than > > > checking for non-zero? > > It is obviously not possible to determine tm_isdst with mktime for a > > negative time_t. Thus with above fix PST works, but PDT is then busted :-( > > Obvious to AIX only? Yes. The whole subject only concerns AIX (at least so far). > My conclusion is that the AIX timezone database is > damaged or missing for pre-1970 dates, but that other systems bothered > to get it at least somewhat right. Is there another issue here that I'm > missing? The tz db is neighter damaged nor missing anything (see below). Only mktime does not work for some (maybe even avoidable) reason for dates before 1970. > > localtime does convert a negative time_t correctly including dst. > > Is there another way to determine tm_isdst ? > > Yes. Replace AIX with Linux or something else, then recompile Postgres > ;) As I see it, the Linux results are also not 100 % correct in respect to dates before 1970. (based on assumption that Solaris is correct) e.g.: 1503c1503 <| Sat May 10 23:59:12 1947 PST --- >| Sat May 10 23:59:12 1947 PDT Was 1947 PDT or PST ? In eighter case one result is one hour off, Solaris or Linux. This raises another issue. Why do we distribute expected files with bogus results in them ? Imho it would be better to only have expected files for rounding issues and the like. Else the user feels that horology works fine on his machine, but as it looks it only works on a few. Andreas
