RE: AIX support
> Do you still need mkldexport.sh? Surely there's a better way to do that
> in year 2024. Some quick googling says there's a '-bexpall' option to
> 'ld', which kind of sounds like what we want. Will that work? How do
> other programs do this?
We have noticed couple of caveats with these flags -bexpall/-bexpfull in other
opensource tools on AIX. This option would export too many symbols causing
problems because a shared library may re-export symbols from another library
causing confused dependencies, duplicate symbols.
We have similar discussion wrt to these flag in Cmake
https://gitlab.kitware.com/cmake/cmake/-/issues/19163
Also, I tried some sample program to verify the same as below
>> cat foo.c
#include
#include
int func1()
{
char str1[] = "Hello ", str2[] = "world! ";
strcat(str1,str2);
puts(str1);
return 0;
}
>> gcc -c foo.c -o foo.o
>> gcc -shared -Wl,-bexpall -o foo.so foo.o
>> dump -Tov foo.so
foo.so:
***Object Module Header***
# Sections Symbol Ptr # Symbols Opt Hdr Len Flags
4 0x0d8812072 0x3002
Flags=( EXEC DYNLOAD SHROBJ DEP_SYSTEM )
Timestamp = "Sep 17 10:17:35 2024"
Magic = 0x1df (32-bit XCOFF)
***Optional Header***
Tsize Dsize Bsize Tstart Dstart
0x0548 0x010c 0x0004 0x1128 0x2670
SNloaderSNentry SNtext SNtoc SNdata
0x0004 0x 0x0001 0x0002 0x0002
TXTalignDATAalign TOC vstamp entry
0x0005 0x0004 0x2750 0x0001 0x
maxSTACKmaxDATA SNbss magic modtype
0x 0x 0x0003 0x010bRE
***Loader Section***
***Loader Symbol Table Information***
[Index] Value Scn IMEX Sclass Type IMPid Name
[0] 0x268c.data RW SECdef[noIMid] __rtinit
[1] 0xundef IMP DS EXTref libgcc_s.a(shr.o)
__cxa_finalize
[2] 0xundef IMP DS EXTref libgcc_s.a(shr.o)
_GLOBAL__AIXI_shr_o
[3] 0xundef IMP DS EXTref libgcc_s.a(shr.o)
_GLOBAL__AIXD_shr_o
[4] 0xundef IMP DS EXTref libc.a(shr.o)
__strtollmax
[5] 0xundef IMP DS EXTref libc.a(shr.o) puts
[6] 0x26f4.data EXP DS Ldef[noIMid]
__init_aix_libgcc_cxa_atexit
[7] 0x2724.data EXP DS Ldef[noIMid]
_GLOBAL__AIXI_foo_so
[8] 0x2730.data EXP DS Ldef[noIMid]
_GLOBAL__AIXD_foo_so
>>[9] 0x273c.data EXP DS SECdef[noIMid] strcat
[10]0x2744.data EXP DS Ldef[noIMid] func1
The code makes use of strcat from libc but re-exports the symbol (because of
-bexpall).
As of now due to the limitation with these flags (-bexpall / -bexpfull ? ), the
solution here is to continue to extract the symbols from the object files and
use that export file as part of building the shared library. (Continue to use
the mkldexport.sh script to generate the export symbols).
Thanks,
Sriram.
RE: AIX support
> The PPC asm code was originally written in 2002, and the first use of
> _ sync_lock_test_and_set(), for ARM, appeared in 2012. The commit that
> made __sync_lock_test_and_set() be chosen automatically for platforms
> that don't specify anything else was added in 2022.
Thanks for the info.
--
> Ok, if we don't need the assembler code at all, that's good. A patch to
> introduce AIX support should not change it for non-AIX powerpc systems
> though. That might be a good change, but would need to be justified
> separately, e.g. by some performance testing, and should be a separate
> patch.
> If you make no changes to s_lock.h at all, will it work? Why not?
With the existing asm code I see there are some syntax errors, being hit.
But after reverting the old changes the issues resolved. Below are diffs.
static __inline__ int
tas(volatile slock_t *lock)
{
@@ -424,17 +413,15 @@ tas(volatile slock_t *lock)
__asm__ __volatile__(
" lwarx %0,0,%3,1 \n"
" cmpwi %0,0\n"
-" bne 1f \n"
+" bne $+16\n" /* branch to li %1,1 */
" addi%0,%0,1 \n"
" stwcx. %0,0,%3 \n"
-" beq 2f \n"
-"1: \n"
+" beq $+12\n" /* branch to lwsync */
" li %1,1\n"
-" b 3f \n"
-"2: \n"
+" b $+12\n" /* branch to end of asm
sequence */
" lwsync \n"
" li %1,0\n"
-"3: \n"
+
: "=&b"(_t), "=r"(_res), "+m"(*lock)
: "r"(lock)
: "memory", "cc");
Let me know if I need to run any perf tools to check the performance of
the __sync_lock_test_and_set change.
---
> > I mean this part of the code is needed as this is specific to AIX
kernel memory
> > operation which is different from __sync_lock_test_and_set().
> >
> > I would like to mention that the changes made in
src/include/storage/s_lock.h
> > are pretty much required and need to be operated in assemble specific
to IBM
> > Power architecture.
> Was that earlier statement incorrect? Is the manual wrong or outdated or
> not applicable to us?
Here this change is specific to AIX, but since we are compiling with gcc, this
is not applicable. But I will try with __sync* routines and check.
---
> Do you still need mkldexport.sh? Surely there's a better way to do that
> in year 2024. Some quick googling says there's a '-bexpall' option to
> 'ld', which kind of sounds like what we want. Will that work? How do
> other programs do this?
Thanks for looking into this, I’m working on this, I will let you know.
Thanks,
Sriram.
Re: AIX support
On Thu, Sep 12, 2024 at 9:57 AM Heikki Linnakangas wrote: > If you make no changes to s_lock.h at all, will it work? Why not? It's good to keep the work independent and I don't want to hold up anything happening in this thread, but just for information: I have been poking around at the idea of entirely removing the old spinlock code and pointing spin.h's function-like-macros to the atomics code. We couldn't do that before, because atomics were sometimes implemented with spinlocks, but now that pg_atomic_flag is never implemented with spinlocks we can flip that around, and then have only one place where we know how to do this stuff. What is needed for that to progress is, I guess, to determine though assembler analysis or experimentation across a bunch of targets that it works out at least as good... https://www.postgresql.org/message-id/flat/CA%2BhUKGJ%2BoA%2B62iUZ-EQb5R2cAOW3Y942ZoOtzOD%3D1sQ05iNg6Q%40mail.gmail.com#23598cafac3dd08ca94fa9e2228a4764
Re: AIX support
On 11/09/2024 15:38, Srirama Kucherlapati wrote: I still don't understand. We have Linux powerpc systems running happily in the buildfarm. They are happy with the current spinlock implementation. Why is this needed? What happens without it? Not sure, by the time the below commits were made if there was a consideration to use the gcc routines. The PPC asm code was originally written in 2002, and the first use of __sync_lock_test_and_set(), for ARM, appeared in 2012. The commit that made __sync_lock_test_and_set() be chosen automatically for platforms that don't specify anything else was added in 2022. I tried to replace the AIX asm (under__ppc__ macro) with the gcc routine __sync_lock_test_and_set(), and all the buildfarm tests passed. Attached patch and the buildfarm output. Please let me know your feedback. Ok, if we don't need the assembler code at all, that's good. A patch to introduce AIX support should not change it for non-AIX powerpc systems though. That might be a good change, but would need to be justified separately, e.g. by some performance testing, and should be a separate patch. If you make no changes to s_lock.h at all, will it work? Why not? You said earlier: I mean this part of the code is needed as this is specific to AIX kernel memory operation which is different from __sync_lock_test_and_set(). I would like to mention that the changes made in src/include/storage/s_lock.h are pretty much required and need to be operated in assemble specific to IBM Power architecture. Was that earlier statement incorrect? Is the manual wrong or outdated or not applicable to us? Moving on.. Do you still need mkldexport.sh? Surely there's a better way to do that in year 2024. Some quick googling says there's a '-bexpall' option to 'ld', which kind of sounds like what we want. Will that work? How do other programs do this? -- Heikki Linnakangas Neon (https://neon.tech)
RE: AIX support
Hi Heikki and Team,
Thanks for your comments.
Here are some more details
> I still don't understand. We have Linux powerpc systems
> running happily in the buildfarm. They are happy with the
> current spinlock implementation. Why is this needed?
> What happens without it?
Not sure, by the time the below commits were made if there was a consideration
to use the gcc routines. My guess is that by using this PPC assembler code
would make the code independent of the compilers. Even the Linux ppc would use
the
same asm. Since gcc is available on AIX, I have replaced the asm changes with
the gcc routine __sync_lock_test_and_set() to set the lock.
We have the gcc(package) build on the AIX platform and as part of the testsuite
there are no issues wrt this routine. We tried to test with the sample test
program extracted from gcc testsuite. Also we discussed the same with our
compiler teams internally and they see no issues using this routine.
gcc-12.4.0/libgomp/testsuite/libgomp.c/sections-1.c
> > +#define TAS(lock)_check_lock((slock_t *) (lock), 0, 1)
> >
> > +#define S_UNLOCK(lock) _clear_lock((slock_t *) (lock), 0)
> >
> How is it different from __sync_lock_test_and_set()? Why is it needed?
> What is AIX kernel memory operation?
More Info: _check_lock routine description
https://www.ibm.com/docs/en/aix/7.1?topic=c-check-lock-subroutine
The _check_lock subroutine performs an atomic (uninterruptible) sequence of
operations. The compare_and_swap subroutine is similar, but does not issue
synchronization instructions and therefore is inappropriate for updating
lock
words.
This change need not be replaced with gcc routine as, these changes will be
compiled for the non-gcc platforms only. This piece of code would never be
compiled, as we are using only gcc to build.
I tried to replace the AIX asm (under__ppc__ macro) with the gcc routine
__sync_lock_test_and_set(), and all the buildfarm tests passed. Attached patch
and the buildfarm output. Please let me know your feedback.
> On a general note: it's your responsibility to explain all the changes
> in a way that others will understand and can verify. It is especially
> important for something critical and platform-specific like spinlocks,
> because others don't have easy access to the hardware to test these
> things independently. I also want to remind you that from the Postgres
> community point of view, you are introducing support for a new platform,
> AIX, not merely resurrecting the old stuff. Every change needs to be
> justified anew.
I do agree with you. To have a better understand on the previous changes,
I was going through the history of the file(s_lock.h) and see that multiple
changes that were made wrt the tas()routine specific to ppc/AIX. Below are
the commit levels.
I would kindly request, Tom Lane, to provide some insights on these changes.
commit e3b06a871b63b90d4a08560ce184bb33324410b8
commit 50938576d482cd36e52a60b5bb1b56026e63962a << added tas() for AIX
commit 7233aae50bea503369b0a4ef9a3b6a3864c96812
commit ceb4f5ea9c2c6c2bd44d4799ff4a62c40a038894 << added tas() for PPC
commit f9ba0a7fe56398e89fe349476d9e437c3197ea28
commit eb5e4c58d137c9258eff5e41b09cb5fe4ed6d64c
commit cd35d601b859d3a56632696b8d5293cbe547764b
commit 109867748259d286dd01fce17d5f895ce59c68d5
commit 5cfa8dd3007d7e953c6a03b0fa2215d97c581b0c
commit 631beeac3598a73dee2c2afa38fa2e734148031b
commit bc2a050d40976441cdb963ad829316c23e8df0aa
commit c41a1215f04912108068b909569551f42059db29
commit 50938576d482cd36e52a60b5bb1b56026e63962a
Please let me know if would like to try on the hardware, we have recently
setup a node in the OSU lab to try out.
Thanks,
Sriram.
0001-AIX-support-revert-the-changes-from-0b16bb8776bb8.v4.patch
Description: 0001-AIX-support-revert-the-changes-from-0b16bb8776bb8.v4.patch
buildfarm17-summary.log
Description: buildfarm17-summary.log
/* Test that all sections are touched. */
/* { dg-require-effective-target sync_int_long } */
#include
#include
#include
#include "libgomp_g.h"
#define N 100
static int data[N];
static int NTHR;
static void clean_data (void)
{
memset (data, -1, sizeof (data));
}
static void test_data (void)
{
int i;
for (i = 0; i < N; ++i)
assert (data[i] != -1);
}
static void set_data (unsigned i, int val)
{
int old;
assert (i >= 1 && i <= N);
old = __sync_lock_test_and_set (data+i-1, val);
assert (old == -1);
}
static void f_1 (void *dummy)
{
int iam = omp_get_thread_num ();
unsigned long s;
for (s = GOMP_sections_start (N); s ; s = GOMP_sections_next ())
set_data (s, iam);
GOMP_sections_end ();
}
static void test_1 (void)
{
clean_data ();
GOMP_parallel_sta
Re: AIX support
On 14/08/2024 18:22, Srirama Kucherlapati wrote: Hi Heikki, I have attached the merged patch with all the changes, the earlier patch was just only the changes specific to older review comments. > I'm sorry, I don't understand what you're saying here. Do you mean that > we don't need to do anything here, and the code we have in s_lock.h in > 'master' now will work fine on AIX? Or do we need to (re-)do some > changes to support AIX again? If we only support GCC, can we use the > __sync_lock_test_and_set() builtin instead? Here we need these changes for ppc. These changes are not for enabling the AIX support, but this is implementing “Enhanced PowerPC Architecture”. This routine is more of compare_and_increment, which is different from GCC __sync_lock_test_and_set(). Also I tried to write a sample function to check the assemble generated by __sync_lock_test_and_set(), which turned out to be different set of assemble code. I still don't understand. We have Linux powerpc systems running happily in the buildfarm. They are happy with the current spinlock implementation. Why is this needed? What happens without it? How is this different from __sync_lock_test_and_set()? Is the __sync_lock_test_and_set() on that platform broken, less efficient, or just different but equally good? > > +#define TAS(lock) _check_lock((slock_t *) (lock), > > 0, 1) > > >> +#define S_UNLOCK(lock) _clear_lock((slock_t *) (lock), 0) >> > > The above changes are specific to AIX kernel and it operates on fixed > > kernel memory. This is more like a compare_and_swap functionality with > > sync capability. For all the assemble code I think it would be better to > > use the IBM Power specific asm code to gain additional performance. > You mean we don't need the above? Ok, good. I mean this part of the code is needed as this is specific to AIX kernel memory operation which is different from __sync_lock_test_and_set(). How is it different from __sync_lock_test_and_set()? Why is it needed? What is AIX kernel memory operation? I would like to mention that the changes made in src/include/storage/s_lock.h are pretty much required and need to be operated in assemble specific to IBM Power architecture. Note that your patch both modifies the existing powerpc implementation, and introduces a new AIX-specific one. They cannot *both* be required, because only one of them will ever be compiled on a given platform. Which is it? Or are you trying to make this work on multiple different CPUs on AIX, so that different implementation gets chosen on different CPUs? Is the mkldexport stuff still needed on modern AIX? Or was it specific to XLC and never needed on GCC? How do other products do that? On a general note: it's your responsibility to explain all the changes in a way that others will understand and can verify. It is especially important for something critical and platform-specific like spinlocks, because others don't have easy access to the hardware to test these things independently. I also want to remind you that from the Postgres community point of view, you are introducing support for a new platform, AIX, not merely resurrecting the old stuff. Every change needs to be justified anew. -- Heikki Linnakangas Neon (https://neon.tech)
RE: AIX support
Hi Heikki, I have attached the merged patch with all the changes, the earlier patch was just only the changes specific to older review comments. > I'm sorry, I don't understand what you're saying here. Do you mean that > we don't need to do anything here, and the code we have in s_lock.h in > 'master' now will work fine on AIX? Or do we need to (re-)do some > changes to support AIX again? If we only support GCC, can we use the > __sync_lock_test_and_set() builtin instead? Here we need these changes for ppc. These changes are not for enabling the AIX support, but this is implementing “Enhanced PowerPC Architecture”. This routine is more of compare_and_increment, which is different from GCC __sync_lock_test_and_set(). Also I tried to write a sample function to check the assemble generated by __sync_lock_test_and_set(), which turned out to be different set of assemble code. > > +#define TAS(lock) _check_lock((slock_t *) (lock), > > 0, 1) > > >> +#define S_UNLOCK(lock) _clear_lock((slock_t *) (lock), 0) >> > > The above changes are specific to AIX kernel and it operates on fixed > > kernel memory. This is more like a compare_and_swap functionality with > > sync capability. For all the assemble code I think it would be better to > > use the IBM Power specific asm code to gain additional performance. > You mean we don't need the above? Ok, good. I mean this part of the code is needed as this is specific to AIX kernel memory operation which is different from __sync_lock_test_and_set(). I would like to mention that the changes made in src/include/storage/s_lock.h are pretty much required and need to be operated in assemble specific to IBM Power architecture. Warm regards, Sriram. 0001-AIX-support-revert-the-changes-from-0b16bb8776bb8.v3.patch Description: 0001-AIX-support-revert-the-changes-from-0b16bb8776bb8.v3.patch
Re: AIX support
On 14/08/2024 06:31, Srirama Kucherlapati wrote: Hi Heikki & Team, I tried to look at the assembly code changes with our team, in the below file. diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h index 29ac6cdcd9..69582f4ae7 100644 --- a/src/include/storage/s_lock.h +++ b/src/include/storage/s_lock.h static __inline__ int tas(volatile slock_t *lock) @@ -424,17 +430,15 @@ tas(volatile slock_t *lock) __asm__ __volatile__( "lwarx %0,0,%3,1\n" "cmpwi %0,0\n" "bne $+16\n"/* branch to li %1,1 */ "addi %0,%0,1\n" "stwcx. %0,0,%3\n" "beq $+12\n"/* branch to lwsync */ "li %1,1\n" "b $+12\n"/* branch to end of asm sequence */ "lwsync\n" "li %1,0\n" :"=&b"(_t), "=r"(_res), "+m"(*lock) :"r"(lock) :"memory", "cc"); For the changes in the above file, this code is very specific to power architecture we need to use the IBM Power specific asm code only, rather than using the GNU assembler. Also, all these asm specific code is under the macro __ppc__, which should not impact any other platforms. I see there is a GCC specific implementation (under this macro #if defined(HAVE_GCC__SYNC_INT32_TAS)) in the same file as well. I'm sorry, I don't understand what you're saying here. Do you mean that we don't need to do anything here, and the code we have in s_lock.h in 'master' now will work fine on AIX? Or do we need to (re-)do some changes to support AIX again? If we only support GCC, can we use the __sync_lock_test_and_set() builtin instead? If any changes are required, please include them in the patch. That'll make it clear what exactly you're proposing. +#define TAS(lock) _check_lock((slock_t *) (lock), 0, 1) +#define S_UNLOCK(lock) _clear_lock((slock_t *) (lock), 0) The above changes are specific to AIX kernel and it operates on fixed kernel memory. This is more like a compare_and_swap functionality with sync capability. For all the assemble code I think it would be better to use the IBM Power specific asm code to gain additional performance. You mean we don't need the above? Ok, good. I was trying to understand here wrt to both the assemble changes if you are looking for anything specific to the architecture. I don't know. You tell me what makes most sense on AIX / powerpc. Attached is the patch for the previous comments, kindly please let me know your comments. Is this all that's needed to resurrect AIX support? -- Heikki Linnakangas Neon (https://neon.tech)
RE: AIX support
Hi Heikki & Team, I tried to look at the assembly code changes with our team, in the below file. diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h index 29ac6cdcd9..69582f4ae7 100644 --- a/src/include/storage/s_lock.h +++ b/src/include/storage/s_lock.h static __inline__ int tas(volatile slock_t *lock) @@ -424,17 +430,15 @@ tas(volatile slock_t *lock) __asm__ __volatile__( "lwarx %0,0,%3,1\n" "cmpwi %0,0\n" "bne $+16\n"/* branch to li %1,1 */ "addi%0,%0,1\n" "stwcx. %0,0,%3\n" "beq $+12\n"/* branch to lwsync */ "li %1,1\n" "b $+12\n"/* branch to end of asm sequence */ "lwsync\n" "li %1,0\n" :"=&b"(_t), "=r"(_res), "+m"(*lock) :"r"(lock) :"memory", "cc"); For the changes in the above file, this code is very specific to power architecture we need to use the IBM Power specific asm code only, rather than using the GNU assembler. Also, all these asm specific code is under the macro __ppc__, which should not impact any other platforms. I see there is a GCC specific implementation (under this macro #if defined(HAVE_GCC__SYNC_INT32_TAS)) in the same file as well. +#define TAS(lock) _check_lock((slock_t *) (lock), 0, 1) +#define S_UNLOCK(lock) _clear_lock((slock_t *) (lock), 0) The above changes are specific to AIX kernel and it operates on fixed kernel memory. This is more like a compare_and_swap functionality with sync capability. For all the assemble code I think it would be better to use the IBM Power specific asm code to gain additional performance. I was trying to understand here wrt to both the assemble changes if you are looking for anything specific to the architecture. Attached is the patch for the previous comments, kindly please let me know your comments. Warm regards, Sriram. 0001-AIX-support-revert-changes-from-0b16bb8776bb.v3.patch Description: 0001-AIX-support-revert-changes-from-0b16bb8776bb.v3.patch
RE: AIX support
We are continuing to work on the changes… > Do you still care about 32-bit binaries on AIX? If not, let's make that > the default in configure or a check for it, and remove the instructions > on building 32-bit binaries from the docs. As most of the products are moving towards 64bit, we will try to remove this 32bit support. Warm regards, Sriram.
Re: AIX support
On 19/06/2024 17:55, Srirama Kucherlapati wrote: +/* Commenting for XLC + * "IBM XL C/C++ for AIX, V12.1" miscompiles, for 32-bit, some inline + * expansions of ginCompareItemPointers() "long long" arithmetic. To take + * advantage of inlining, build a 64-bit PostgreSQL. +#if defined(__ILP32__) && defined(__IBMC__) +#define PG_FORCE_DISABLE_INLINE +#endif + */ This seems irrelevant. + * Ordinarily, we'd code the branches here using GNU-style local symbols, that + * is "1f" referencing "1:" and so on. But some people run gcc on AIX with + * IBM's assembler as backend, and IBM's assembler doesn't do local symbols. + * So hand-code the branch offsets; fortunately, all PPC instructions are + * exactly 4 bytes each, so it's not too hard to count. Could you use GCC assembler to avoid this? @@ -662,6 +666,21 @@ tas(volatile slock_t *lock) #if !defined(HAS_TEST_AND_SET) /* We didn't trigger above, let's try here */ +#if defined(_AIX) /* AIX */ +/* + * AIX (POWER) + */ +#define HAS_TEST_AND_SET + +#include + +typedef int slock_t; + +#define TAS(lock) _check_lock((slock_t *) (lock), 0, 1) +#define S_UNLOCK(lock) _clear_lock((slock_t *) (lock), 0) +#endif /* _AIX */ + + /* These are in sunstudio_(sparc|x86).s */ #if defined(__SUNPRO_C) && (defined(__i386) || defined(__x86_64__) || defined(__sparc__) || defined(__sparc)) What CPI/compiler/OS configuration is this for, exactly? Could we rely on GCC-provided __sync_lock_test_and_set() builtin function instead? +# Allow platforms with buggy compilers to force restrict to not be +# used by setting $FORCE_DISABLE_RESTRICT=yes in the relevant +# template. Surely we don't need that anymore? Or is the compiler still buggy? Do you still care about 32-bit binaries on AIX? If not, let's make that the default in configure or a check for it, and remove the instructions on building 32-bit binaries from the docs. Please try hard to remove any changes from the diff that are not absolutely necessary. - Heikki
RE: AIX support
Thanks Hikki, for going through the changes. > +/* Commenting for XLC > + * "IBM XL C/C++ for AIX, V12.1" miscompiles, for 32-bit, some inline > + * expansions of ginCompareItemPointers() "long long" arithmetic. To take > + * advantage of inlining, build a 64-bit PostgreSQL. > +#if defined(__ILP32__) && defined(__IBMC__) > +#define PG_FORCE_DISABLE_INLINE > +#endif > + */ I can remove these unwanted comments. I have to analyze the changes for the rest of your comment and will get back to you. Warm regards, Sriram. From: Heikki Linnakangas Date: Wednesday, 19 June 2024 at 8:45 PM To: Srirama Kucherlapati , Laurenz Albe , Bruce Momjian , Heikki Linnakangas Cc: Peter Eisentraut , Alvaro Herrera , pgsql-hack...@postgresql.org , Noah Misch , Michael Paquier , Andres Freund , Tom Lane , Thomas Munro , tvk1...@gmail.com , postgres-ibm-...@wwpdl.vnet.ibm.com Subject: [EXTERNAL] Re: AIX support On 19/06/2024 17:55, Srirama Kucherlapati wrote: > +/* Commenting for XLC > + * "IBM XL C/C++ for AIX, V12.1" miscompiles, for 32-bit, some inline > + * expansions of ginCompareItemPointers() "long long" arithmetic. To take > + * advantage of inlining, build a 64-bit PostgreSQL. > +#if defined(__ILP32__) && defined(__IBMC__) > +#define PG_FORCE_DISABLE_INLINE > +#endif > + */ This seems irrelevant. > + * Ordinarily, we'd code the branches here using GNU-style local symbols, > that > + * is "1f" referencing "1:" and so on. But some people run gcc on AIX with > + * IBM's assembler as backend, and IBM's assembler doesn't do local symbols. > + * So hand-code the branch offsets; fortunately, all PPC instructions are > + * exactly 4 bytes each, so it's not too hard to count. Could you use GCC assembler to avoid this? > @@ -662,6 +666,21 @@ tas(volatile slock_t *lock) > > #if !defined(HAS_TEST_AND_SET) /* We didn't trigger above, let's try > here */ > > +#if defined(_AIX)/* AIX */ > +/* > + * AIX (POWER) > + */ > +#define HAS_TEST_AND_SET > + > +#include > + > +typedef int slock_t; > + > +#define TAS(lock)_check_lock((slock_t *) (lock), 0, 1) > +#define S_UNLOCK(lock) _clear_lock((slock_t *) (lock), 0) > +#endif/* _AIX */ > + > + > /* These are in sunstudio_(sparc|x86).s */ > > #if defined(__SUNPRO_C) && (defined(__i386) || defined(__x86_64__) || > defined(__sparc__) || defined(__sparc)) What CPI/compiler/OS configuration is this for, exactly? Could we rely on GCC-provided __sync_lock_test_and_set() builtin function instead? > +# Allow platforms with buggy compilers to force restrict to not be > +# used by setting $FORCE_DISABLE_RESTRICT=yes in the relevant > +# template. Surely we don't need that anymore? Or is the compiler still buggy? Do you still care about 32-bit binaries on AIX? If not, let's make that the default in configure or a check for it, and remove the instructions on building 32-bit binaries from the docs. Please try hard to remove any changes from the diff that are not absolutely necessary. - Heikki
RE: AIX support
Hi Team, Please find the attached patch, which resumes the AIX support with gcc alone. We have removed changes wrt to XLC on AIX. We are also continuing to work on the XLC and IBM-clang(openXLC) specific patch as well. Once we get an approval for the above patch we can submit a subsequent patch to support XLC/IBM-clang changes. Kindly let us know your inputs/feedback. Warm regards, Sriram. 0001-AIX-support-revert-the-changes-from-0b16bb8776bb8-v2.patch Description: 0001-AIX-support-revert-the-changes-from-0b16bb8776bb8-v2.patch
RE: AIX support
Hi Laurenz, we are working on the other file changes, we shall post you the updates soon. Warm regards, Sriram.
Re: AIX support
On Fri, 2024-06-07 at 16:30 +, Srirama Kucherlapati wrote: > Hi Team, We are pursuing to trim the changes wrt AIX. As of now we trimmed > the changes with respect to XLC and currently with trimmed changes the > buildfarm script passed (build and all the regression tests) > The XLC changes were trimmed only in the below file > modified: configure > modified: configure.ac Did you forget an attachment? Yours, Laurenz Albe
RE: AIX support
Hi Team, We are pursuing to trim the changes wrt AIX. As of now we trimmed the changes with respect to XLC and currently with trimmed changes the buildfarm script passed (build and all the regression tests) The XLC changes were trimmed only in the below file modified: configure modified: configure.ac We are looking further into the other file changes as well. Warm regards, Sriram.
Re: AIX support
On Thu, May 23, 2024 at 07:03:20PM +0300, Heikki Linnakangas wrote: > > Can you provide some more details on the expectations here? > > Smallest possible patch that makes Postgres work on AIX again. > > Perhaps start with the patch you posted yesterday, but remove hunks from it > one by one, to see which ones are still needed. Yes, bingo, that is exactly what needs to be done, and for the minimal compiler, gcc, and the most recently supported versions of AIX. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
Re: AIX support
On 23/05/2024 18:36, Srirama Kucherlapati wrote: Hi Peter, thanks for your feedback. We are eager to extend our support in resolving the issues specific to AIX or corresponding compilers (XLC and cLang). But as there are no issues with the patch after reverting the changes(with the latest compilers gcc12 and xlc-16.0.1.18), we were wondering if this patch can be merged with the current release 17?? Having said that, we are committed to resolve all the hacks and caveats that got accumulated specific to AIX over the period by picking and resolving one after the other, rather than waiting for all the hacks to be fixed. I'm not eager to put back those hacks just to have them be removed again. So I'd like to see a minimal patch, with the *minimal* changes required for AIX support. And perhaps split that into two patches: First add back AIX support with GCC, and second patch to add XLC support. I'd like to to see how much of the changes are because of the different compiler and how much from the OS. No promises for v17, but if the patch is small and non-intrusive, I would consider it at least. But let's see what it looks like first. It's the same work that needs to be done whether it goes into v17 or v18 anyway. One of the reasons that the AIX port ultimately became unmaintainable was that so many hacks and caveats were accumulated over the years. A new port should set a more recent baseline and trim all those hacks. Please help me understand this, with respect to the AIX specific hacks, is it just we can find all the location where _AIX macros are involved OR can we just look at the patch changes only, as all the changes that were made were specific to AIX. If not, is there any other location where we could find all the hacks to be resolved. Can you provide some more details on the expectations here? Smallest possible patch that makes Postgres work on AIX again. Perhaps start with the patch you posted yesterday, but remove hunks from it one by one, to see which ones are still needed. -- Heikki Linnakangas Neon (https://neon.tech)
RE: AIX support
Hi Peter, thanks for your feedback. We are eager to extend our support in resolving the issues specific to AIX or corresponding compilers (XLC and cLang). But as there are no issues with the patch after reverting the changes(with the latest compilers gcc12 and xlc-16.0.1.18), we were wondering if this patch can be merged with the current release 17?? Having said that, we are committed to resolve all the hacks and caveats that got accumulated specific to AIX over the period by picking and resolving one after the other, rather than waiting for all the hacks to be fixed. > One of the reasons that the AIX port ultimately became unmaintainable > was that so many hacks and caveats were accumulated over the years. A > new port should set a more recent baseline and trim all those hacks. Please help me understand this, with respect to the AIX specific hacks, is it just we can find all the location where _AIX macros are involved OR can we just look at the patch changes only, as all the changes that were made were specific to AIX. If not, is there any other location where we could find all the hacks to be resolved. Can you provide some more details on the expectations here? Warm regards, Sriram.
Re: AIX support
On 22.05.24 18:15, Sriram RK wrote: Please find the attached patch. Apart from the AIX specific changes, there is a minor change in this file wrt to XLC, below is the error for which we removed inline. Later, the build and tests passed for both XLC(16.1.0.18) and gcc(12) as well. I think what you should do next is aggressively trim anything that does not apply to current versions of AIX or the current compiler. For example, + # Old xlc versions (<13.1) don't have support for -qvisibility. Use expfull to force + +AIX versions before 7.1 are no longer +tested nor supported by the PostgreSQL +community. + (Probably most of that section needs to be retested and rewritten.) + # Native memset() is faster, tested on: + # - AIX 5.1 and 5.2, XLC 6.0 (IBM's cc) + # - AIX 5.3 ML3, gcc 4.0.1 + memset_loop_limit = 0 + # for the base executable (AIX 4.2 and up) + * "IBM XL C/C++ for AIX, V12.1" miscompiles, for 32-bit, some inline One of the reasons that the AIX port ultimately became unmaintainable was that so many hacks and caveats were accumulated over the years. A new port should set a more recent baseline and trim all those hacks.
Re: AIX support
Thanks Alvaro, for the info… Hi Team, We referred to the below links to build this patch … https://wiki.postgresql.org/wiki/Submitting_a_Patch https://peter.eisentraut.org/blog/2023/05/09/how-to-submit-a-patch-by-email-2023-edition Please find the attached patch. Apart from the AIX specific changes, there is a minor change in this file wrt to XLC, below is the error for which we removed inline. Later, the build and tests passed for both XLC(16.1.0.18) and gcc(12) as well. src/backend/storage/buffer/bufmgr.c "bufmgr.c", line 811.39: 1506-780 (S) Reference to "RelationGetSmgr" with internal linkage is not allowed within inline definition of "ReadBufferExtended". "bufmgr.c", line 811.15: 1506-780 (S) Reference to "ReadBuffer_common" with internal linkage is not allowed within inline definition of "ReadBufferExtended". gmake[4]: *** [: bufmgr.o] Error 1 Please let us know your feedback. Thanks, Sriram. 0001-AIX-support-revert-the-changes-from-0b16bb8776bb8.patch Description: 0001-AIX-support-revert-the-changes-from-0b16bb8776bb8.patch
Re: AIX support
On 2024-May-16, Sriram RK wrote: > Hi Team, > > We have an update wrt to the PG17 AIX port. > > We have reverted the changes specific to AIX (that were removed in > 0b16bb8776bb8) to the latest PG17 (head). > > The Buildfarm succeeded for these changes. All the tests passed. Excellent. > Can you please let us know, the process to post the changes for review? Here's some very good advice https://postgr.es/m/20240405172649...@rfd.leadboat.com Regards -- Álvaro Herrera
Re: AIX support
Hi Team, We have an update wrt to the PG17 AIX port. We have reverted the changes specific to AIX (that were removed in 0b16bb8776bb8) to the latest PG17 (head). The Buildfarm succeeded for these changes. All the tests passed. System config OS level : AIX-73D Compiler : gcc-12 & xlc(16.1.0.18) Wed May 15 21:26:00 2024: buildfarm run for AIXnode01:HEAD starting AIXnode01:HEAD [21:26:00] running configure ... AIXnode01:HEAD [21:27:03] running build ... AIXnode01:HEAD [21:27:27] running basic regression tests ... AIXnode01:HEAD [21:34:41] running make contrib ... AIXnode01:HEAD [21:34:43] running make testmodules ... AIXnode01:HEAD [21:34:44] running install ... AIXnode01:HEAD [21:34:58] running make contrib install ... AIXnode01:HEAD [21:35:05] running testmodules install ... AIXnode01:HEAD [21:35:08] checking pg_upgrade AIXnode01:HEAD [21:35:08] checking test-decoding AIXnode01:HEAD [21:35:29] running make check miscellaneous modules ... AIXnode01:HEAD [21:36:16] setting up db cluster (C)... AIXnode01:HEAD [21:36:19] starting db (C)... AIXnode01:HEAD [21:36:19] running installcheck (C)... AIXnode01:HEAD [21:46:27] restarting db (C)... AIXnode01:HEAD [21:46:29] running make isolation check ... AIXnode01:HEAD [21:49:57] restarting db (C)... AIXnode01:HEAD [21:50:02] running make PL installcheck (C)... AIXnode01:HEAD [21:50:09] restarting db (C)... AIXnode01:HEAD [21:50:12] running make contrib installcheck (C)... AIXnode01:HEAD [21:53:53] restarting db (C)... AIXnode01:HEAD [21:53:56] running make test-modules installcheck (C)... AIXnode01:HEAD [21:54:28] stopping db (C)... AIXnode01:HEAD [21:54:29] running make ecpg check ... AIXnode01:HEAD [21:54:45] OK Branch: HEAD All stages succeeded The below changes are applied on this commit level commit 54b69f1bd730a228a666441592a12d2a0cbe2a06 (HEAD -> pgAIX, origin/master, origin/HEAD, master) On branch pgAIX Changes to be committed: (use "git restore --staged ..." to unstage) new file: src/backend/port/aix/mkldexport.sh new file: src/include/port/aix.h new file: src/makefiles/Makefile.aix new file: src/template/aix Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git restore ..." to discard changes in working directory) modified: Makefile modified: config/c-compiler.m4 modified: configure modified: configure.ac modified: doc/src/sgml/dfunc.sgml modified: doc/src/sgml/installation.sgml modified: doc/src/sgml/runtime.sgml modified: meson.build modified: src/Makefile.shlib modified: src/backend/Makefile modified: src/backend/meson.build modified: src/backend/storage/buffer/bufmgr.c modified: src/backend/utils/error/elog.c modified: src/backend/utils/misc/ps_status.c modified: src/bin/pg_basebackup/t/010_pg_basebackup.pl modified: src/bin/pg_verifybackup/t/008_untar.pl modified: src/bin/pg_verifybackup/t/010_client_untar.pl modified: src/include/c.h modified: src/include/port/atomics.h modified: src/include/storage/s_lock.h modified: src/interfaces/libpq/Makefile modified: src/interfaces/libpq/meson.build modified: src/port/README modified: src/port/strerror.c modified: src/test/regress/Makefile modified: src/test/regress/expected/sanity_check.out modified: src/test/regress/expected/test_setup.out modified: src/test/regress/regress.c modified: src/test/regress/sql/sanity_check.sql modified: src/test/regress/sql/test_setup.sql modified: src/tools/gen_export.pl modified: src/tools/pginclude/headerscheck Can you please let us know, the process to post the changes for review? Regards, Sriram.
Re: AIX support
> > Also would like to know some info related to the request raised for > > buildfarm access, to register the > > node in OSU lab. Where can I get the status of the request? Whom can I > > contact to get the request > > approved? So that we can add the node to the buildfarm. > I assume you filled out the form at > https://buildfarm.postgresql.org/cgi-bin/register-form.pl? It can take a few > weeks, so I wouldn't worry yet. Thanks Noha, I had already submitted a form a week back, hope it might take another couple of weeks to get it approved.
Re: AIX support
On Wed, May 15, 2024 at 03:33:25PM +, Sriram RK wrote: > Hi Team, we have any updated from the XLC team, the issue specific to the > alignment is fixed > and XLC had released it as part of 16.1.0.18. The PTF is available at the > below location, > > You can also find a link here: > https://www.ibm.com/support/pages/fix-list-xl-cc-aix. > > >>/opt/IBM/xlC/16.1.0/bin/xlC align.c -o align.xl > > >>./align.xl > al4096 4096 @ 0x20008000 (mod 0) > al4096_initialized 4096 @ 0x20004000 (mod 0) > al4096_const 4096 @ 0x2000b000 (mod 0) > al4096_const_initialized 4096 @ 0x10008000 (mod 0) > al4096_static4096 @ 0x2000e000 (mod 0) > al4096_static_initialized4096 @ 0x20001000 (mod 0) > al4096_static_const 4096 @ 0x20011000 (mod 0) > al4096_static_const_initialized 4096 @ 0x10001000 (mod 0) That is good news. PGIOAlignedBlock is now in the IBM publication, https://www.ibm.com/support/pages/apar/IJ51032 > Also would like to know some info related to the request raised for buildfarm > access, to register the node in OSU lab. Where can I get the status of the > request? Whom can I contact to get the request approved? So that we can add > the node to the buildfarm. I assume you filled out the form at https://buildfarm.postgresql.org/cgi-bin/register-form.pl? It can take a few weeks, so I wouldn't worry yet.
Re: AIX support
Hi Team, we have any updated from the XLC team, the issue specific to the alignment is fixed and XLC had released it as part of 16.1.0.18. The PTF is available at the below location, You can also find a link here: https://www.ibm.com/support/pages/fix-list-xl-cc-aix. >>/opt/IBM/xlC/16.1.0/bin/xlC align.c -o align.xl >>./align.xl al4096 4096 @ 0x20008000 (mod 0) al4096_initialized 4096 @ 0x20004000 (mod 0) al4096_const 4096 @ 0x2000b000 (mod 0) al4096_const_initialized 4096 @ 0x10008000 (mod 0) al4096_static4096 @ 0x2000e000 (mod 0) al4096_static_initialized4096 @ 0x20001000 (mod 0) al4096_static_const 4096 @ 0x20011000 (mod 0) al4096_static_const_initialized 4096 @ 0x10001000 (mod 0) Also would like to know some info related to the request raised for buildfarm access, to register the node in OSU lab. Where can I get the status of the request? Whom can I contact to get the request approved? So that we can add the node to the buildfarm. Regards, Sriram.
Re: AIX support
On Wed, May 8, 2024 at 03:44:12PM +0200, Peter Eisentraut wrote: > On 08.05.24 13:39, Sriram RK wrote: > > We would like to understand your inputs/plans on reverting the changes > > for AIX. > > I think the ship has sailed for PG17. The way forward would be that you > submit a patch for new, modernized AIX support for PG18. Yes, I think we were clear that someone needs to review the reverted patch and figure out which parts are still needed, and why. We have no "plans" to restore support. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
Re: AIX support
On 08.05.24 13:39, Sriram RK wrote: We would like to understand your inputs/plans on reverting the changes for AIX. I think the ship has sailed for PG17. The way forward would be that you submit a patch for new, modernized AIX support for PG18.
Re: AIX support
Hi Team, We have the AIX node ready in OSU lab, and the branches 15 and 16 got build on the node. We had raised a request to register this node as buildfarm member. Yet to receive the approval. We would like to understand your inputs/plans on reverting the changes for AIX. Thanks, Sriram.
Re: AIX support
Hi Team, on further investigation we were able to resolve the perl issue by setting the right PERL env location. Earlier it was pointing to the 32bit perl, as a result the perl lib mismatch seems to be happening. Now we have successfully built release 15 and 16 stable branches on the OSU lab node. p9aix (OSU) OS: AIX 72Z RELEASE 16 p9aix:REL_16_STABLE [08:31:26] OK log passed to send_result === Branch: REL_16_STABLE All stages succeeded RELEASE 15 p9aix:REL_15_STABLE [08:55:37] OK log passed to send_result === Branch: REL_15_STABLE All stages succeeded Also, we had successfully built release 16 branch on our local nodes as well OS: AIX 71C pgsql-aix71C:REL_16_STABLE [02:25:32] OK log passed to send_result === Branch: REL_16_STABLE All stages succeeded OS: AIX72Z pgsql-aix72Z:REL_16_STABLE [02:35:03] OK log passed to send_result === Branch: REL_16_STABLE All stages succeeded OS: AIX73D pgsql-aix73D:REL_16_STABLE [05:32:29] OK log passed to send_result === Branch: REL_16_STABLE All stages succeeded Regards, Sriram.
Re: AIX support
Hi Team, There are couple of updates, firstly we got an AIX node on the OSU lab. Please feel free to reach me, so that we can provide access to the node. We have started working on setting up the buildfarm on that node. Secondly, as part of the buildfarm setup on our local nodes, we are hitting an issue with the plperl extension. In the logs we noticed that when the plperl extension is being created, it is failing to load the perl library. - CREATE EXTENSION plperlu; + server closed the connection unexpectedly + This probably means the server terminated abnormally + before or while processing the request. + connection to server was lost In the logfile we could see these 2024-05-04 05:05:17.537 CDT [3997786:17] pg_regress/plperl_setup LOG: statement: CREATE EXTENSION plperl; Util.c: loadable library and perl binaries are mismatched (got first handshake key 9b80080, needed 9a80080) We tried to resolve some of the suggestions mentioned here, but things couldn’t resolve. https://www.postgresql.org/message-id/CALDaNm15qaRFb3WiPFAdFqoB9pj1E5SCPPUGB%2BnJ4iF4gXO6Rw%40mail.gmail.com Any inputs here would be greatly appreciated. Regards, Sriram.
Re: AIX support
> > It would definitely make sense for a new port to start by getting > > things going with gcc only, and then look at resurrecting xlc > > support. > Sriram mentioned upthread that he was looking at both of them. I'd be > ready to assume that most of the interest is in xlc, not gcc. But I > may be wrong. Just a heads-up, we got a node in the OSU lab for the buildfarm. Will let you know once we have the buildfarm setup on that node. Also, we are making progress on setting up the buildfarm on a local node as well. But currently there are some tests failing, seems some issue with plperl. aix01::build-farm-17# ./run_build.pl --keepall --nosend --nostatus --verbose=5 --force REL_16_STABLE Fri Apr 26 00:53:50 2024: buildfarm run for AIXnode01:REL_16_STABLE starting AIXnode01:REL_16_STABLE [00:53:50] checking out source ... AIXnode01:REL_16_STABLE [00:53:56] checking if build run needed ... AIXnode01:REL_16_STABLE [00:53:56] copying source to pgsql.build ... AIXnode01:REL_16_STABLE [00:54:08] running configure ... AIXnode01:REL_16_STABLE [00:55:01] running build ... AIXnode01:REL_16_STABLE [01:08:09] running basic regression tests ... AIXnode01:REL_16_STABLE [01:09:51] running make contrib ... AIXnode01:REL_16_STABLE [01:11:08] running make testmodules ... AIXnode01:REL_16_STABLE [01:11:19] running install ... AIXnode01:REL_16_STABLE [01:11:48] running make contrib install ... AIXnode01:REL_16_STABLE [01:12:01] running testmodules install ... AIXnode01:REL_16_STABLE [01:12:06] checking test-decoding gmake: gcc: A file or directory in the path name does not exist. AIXnode01:REL_16_STABLE [01:12:28] running make check miscellaneous modules ... gmake: gcc: A file or directory in the path name does not exist. AIXnode01:REL_16_STABLE [01:13:50] setting up db cluster (C)... AIXnode01:REL_16_STABLE [01:13:53] starting db (C)... AIXnode01:REL_16_STABLE [01:13:53] running installcheck (C)... AIXnode01:REL_16_STABLE [01:15:05] restarting db (C)... AIXnode01:REL_16_STABLE [01:15:07] running make isolation check ... AIXnode01:REL_16_STABLE [01:15:51] restarting db (C)... AIXnode01:REL_16_STABLE [01:15:56] running make PL installcheck (C)... Branch: REL_16_STABLE Stage PLCheck-C failed with status 2
Re: AIX support
On Thu, Apr 25, 2024 at 01:06:24PM +0900, Michael Paquier wrote: > Anyway, getting an access to such compilers to be able to debug issues > on hosts that take less than 12h to just compile the code would > certainly help its adoption. So seeing commitment in the form of > patches and access to environments would help a lot. Overall, > studying that afresh with v18 looks like a good idea, assuming that > anybody who commits such patches has access to hosts to evaluate them, > with buildfarm members running on top, of course. Agreed. They can't even have buildfarm member for PG 17 since it doesn't compile anymore, so someone has to go over the reverted patch, figure out which ones are still valid, and report back. Trying to add a port, with possible breakage, during beta seems too risky compared to the value. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
Re: AIX support
On Thu, Apr 25, 2024 at 10:16:34AM +0200, Álvaro Herrera wrote: > On 2024-Apr-24, Bruce Momjian wrote: > > > I agree that targeting PG 18 for a new-er AIX port is the reasonable > > approach. If there is huge demand, someone can create an AIX fork for > > PG 17 using the reverted patches --- yeah, lots of pain there, but we > > have carried the AIX pain for too long with too little support. > > I'm not sure how large the demand would be for an AIX port specifically > of 17, though. I mean, people using older versions can continue to use > 16 until 18 is released. Upgrading past one major version is hardly > unheard of. Agreed. They seem to have packages for 11/12, and only 15 recently. I don't see how PG 17 would be missed, unless there are many people compiling from source. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
Re: AIX support
Hi, On 2024-04-25 00:20:05 -0400, Tom Lane wrote: > Something I've been mulling over is whether to suggest that the > proposed "new port" should only target building with gcc. Yes. I also wonder if such a port should only support building with sysv style shared library support, rather than the AIX (and windows) style. That'd make it considerably less impactful on the buildsystem level. I don't know what the performance difference is these days. Greetings, Andres Freund
Re: AIX support
Peter Eisentraut writes: > On 25.04.24 06:20, Tom Lane wrote: >> Something I've been mulling over is whether to suggest that the >> proposed "new port" should only target building with gcc. > My understanding is that the old xlc is dead and has been replaced by > "xlclang", which is presumably an xlc-compatible frontend on top of > clang/llvm. Hopefully, that will have fewer issues. [ googles... ] Actually it seems to be the other way around: per [1] xlclang is a clang-based front end to IBM's existing codegen+optimization logic, and the xlc front end is still there too. It's not at all clear that they have any intention of killing off xlc. Not sure where that leaves us in terms of either one being an interesting target to support. xlclang is presumably an easier lift to get working, but that also makes it much less interesting from the preserve-our-portability standpoint. regards, tom lane [1] https://www.ibm.com/docs/en/xl-c-and-cpp-aix/16.1?topic=new-clang-based-front-end
Re: AIX support
On 2024-Apr-24, Bruce Momjian wrote: > I agree that targeting PG 18 for a new-er AIX port is the reasonable > approach. If there is huge demand, someone can create an AIX fork for > PG 17 using the reverted patches --- yeah, lots of pain there, but we > have carried the AIX pain for too long with too little support. I'm not sure how large the demand would be for an AIX port specifically of 17, though. I mean, people using older versions can continue to use 16 until 18 is released. Upgrading past one major version is hardly unheard of. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ "If you want to have good ideas, you must have many ideas. Most of them will be wrong, and what you have to learn is which ones to throw away." (Linus Pauling)
Re: AIX support
On 25.04.24 06:20, Tom Lane wrote: Something I've been mulling over is whether to suggest that the proposed "new port" should only target building with gcc. On the one hand, that would (I think) remove a number of annoying issues, and the average end user is unlikely to care which compiler their database server was built with. On the other hand, I'm a strong proponent of avoiding software monocultures, and xlc is one of the few C compilers still standing that aren't gcc or clang. My understanding is that the old xlc is dead and has been replaced by "xlclang", which is presumably an xlc-compatible frontend on top of clang/llvm. Hopefully, that will have fewer issues.
Re: AIX support
On Thu, Apr 25, 2024 at 12:20:05AM -0400, Tom Lane wrote: > It would definitely make sense for a new port to start by getting > things going with gcc only, and then look at resurrecting xlc > support. Sriram mentioned upthread that he was looking at both of them. I'd be ready to assume that most of the interest is in xlc, not gcc. But I may be wrong. Saying that, dividing the effort into more successive steps is sensible here (didn't consider that previously, you have a good point). -- Michael signature.asc Description: PGP signature
Re: AIX support
Michael Paquier writes: > Some of the portability changes removed in 0b16bb877 feel indeed > obsolete, so it may not hurt to start an analysis from scratch to see > the minimum amount of work that would be really required with the > latest versions of xlc, using the newest compilers as a supported > base. Something I've been mulling over is whether to suggest that the proposed "new port" should only target building with gcc. On the one hand, that would (I think) remove a number of annoying issues, and the average end user is unlikely to care which compiler their database server was built with. On the other hand, I'm a strong proponent of avoiding software monocultures, and xlc is one of the few C compilers still standing that aren't gcc or clang. It would definitely make sense for a new port to start by getting things going with gcc only, and then look at resurrecting xlc support. > I'd like to think backporting these to stable branches should > be OK at some point, once the new port is proving baked enough. If things go as I expect, the "new port" would effectively drop support for older AIX and/or older compiler versions. So back- porting seems like an unlikely decision. > Anyway, getting an access to such compilers to be able to debug issues > on hosts that take less than 12h to just compile the code would > certainly help its adoption. +many regards, tom lane
Re: AIX support
On Wed, Apr 24, 2024 at 11:39:37PM -0400, Bruce Momjian wrote: > On Sat, Apr 20, 2024 at 12:25:47PM -0400, Tom Lane wrote: >> So I'm totally not in favor of #1, at least not without some hard >> commitments and follow-through on really cleaning up the mess >> (which maybe looks more like your #2). What's needed here, as >> you said, is for someone with a decent amount of expertise in >> modern AIX to review all the issues. Maybe framing that as a >> "new port" per #3 would be a good way to think about it. But >> I don't want to just revert the AIX-ectomy and continue drifting. >> >> On the whole, it wouldn't be the worst thing in the world if PG 17 >> lacks AIX support but that comes back in PG 18. That approach would >> solve the schedule-crunch aspect and give time for considered review >> of how many of the hacks removed in 0b16bb877 really need to be put >> back, versus being obsolete or amenable to a nicer solution in >> late-model AIX. If we take a "new port" mindset then it would be >> totally reasonable to say that it only supports very recent AIX >> releases, so I'd hope at least some of the cruft could be removed. > > I agree that targeting PG 18 for a new-er AIX port is the reasonable > approach. If there is huge demand, someone can create an AIX fork for > PG 17 using the reverted patches --- yeah, lots of pain there, but we > have carried the AIX pain for too long with too little support. Some of the portability changes removed in 0b16bb877 feel indeed obsolete, so it may not hurt to start an analysis from scratch to see the minimum amount of work that would be really required with the latest versions of xlc, using the newest compilers as a supported base. I'd like to think backporting these to stable branches should be OK at some point, once the new port is proving baked enough. Anyway, getting an access to such compilers to be able to debug issues on hosts that take less than 12h to just compile the code would certainly help its adoption. So seeing commitment in the form of patches and access to environments would help a lot. Overall, studying that afresh with v18 looks like a good idea, assuming that anybody who commits such patches has access to hosts to evaluate them, with buildfarm members running on top, of course. My 2c. -- Michael signature.asc Description: PGP signature
Re: AIX support
On Sat, Apr 20, 2024 at 12:25:47PM -0400, Tom Lane wrote: > > I can see several ways going forward: > > 1. We revert the removal of AIX support and carry on with the status quo > > ante. (The removal of AIX is a regression; it is timely and in scope > > now to revert the change.) > > 2. Like (1), but we consider that notice has been given, and we will > > remove it early in PG18 (like August) unless the situation improves. > > 3. We leave it out of PG17 and consider a new AIX port for PG18 on its > > own merits. > > Andres has ably summarized the reasons why the status quo ante was > getting untenable. The direct-I/O problem could have been tolerable > on its own, but in reality it was the straw that broke the camel's > back so far as our willingness to maintain AIX support went. There > were just too many hacks and workarounds for too many problems, > with too few people interested in looking for better answers. > > So I'm totally not in favor of #1, at least not without some hard > commitments and follow-through on really cleaning up the mess > (which maybe looks more like your #2). What's needed here, as > you said, is for someone with a decent amount of expertise in > modern AIX to review all the issues. Maybe framing that as a > "new port" per #3 would be a good way to think about it. But > I don't want to just revert the AIX-ectomy and continue drifting. > > On the whole, it wouldn't be the worst thing in the world if PG 17 > lacks AIX support but that comes back in PG 18. That approach would > solve the schedule-crunch aspect and give time for considered review > of how many of the hacks removed in 0b16bb877 really need to be put > back, versus being obsolete or amenable to a nicer solution in > late-model AIX. If we take a "new port" mindset then it would be > totally reasonable to say that it only supports very recent AIX > releases, so I'd hope at least some of the cruft could be removed. I agree that targeting PG 18 for a new-er AIX port is the reasonable approach. If there is huge demand, someone can create an AIX fork for PG 17 using the reverted patches --- yeah, lots of pain there, but we have carried the AIX pain for too long with too little support. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
Re: AIX support
On Sat Apr 20, 2024 at 10:42 AM CDT, Peter Eisentraut wrote: 3. We leave it out of PG17 and consider a new AIX port for PG18 on its own merits. Note that such a "new" port would probably require quite a bit of development and research work, to clean up all the cruft that had accumulated over the years in the old port. Another looming issue is that the meson build system only supported AIX with gcc before the removal. I don't know what it would take to expand that to support xclang, but if it requires meson upstream work, you have that to do, too. Happy to help advocate for any PRs from AIX folks on the Meson side. You can find me as @tristan957 on github. -- Tristan Partin Neon (https://neon.tech)
Re: AIX support
Hi Team, > I have some sympathy for this. The discussion about removing AIX > support had a very short turnaround and happened in an unrelated thread, > without any sort of public announcement or consultation. So this report > of "hey, we were still using that" is timely and fair. We would really like to thank you & the team for considering our request, and really appreciate for providing all the possible options to support AIX. > But the underlying issue that led to the removal (something to do with > direct I/O support and alignment) would still need to be addressed. As we already validated that these alignment specific issues are resolved with the latest versions of the compilers (gcc/ibm-clang). We would request you to use the latest versions for the build. > If we are making the reinstatement of AIX > support contingent on new buildfarm support, those machines need to be > available, at least initially, at least for back branches, like in a > week. Which seems tight. We are already working with the internal team in procuring the nodes for the buildfarm, which can be accessible by the community. > I can see several ways going forward: > 1. We revert the removal of AIX support and carry on with the status quo > ante. (The removal of AIX is a regression; it is timely and in scope > now to revert the change.) > 2. Like (1), but we consider that notice has been given, and we will > remove it early in PG18 (like August) unless the situation improves. We would really appreciate you for providing the possible options and we are very much inclined to these above approaches. Regards, Sriram.
Re: AIX support
Peter Eisentraut writes: > I have some sympathy for this. The discussion about removing AIX > support had a very short turnaround and happened in an unrelated thread, > without any sort of public announcement or consultation. So this report > of "hey, we were still using that" is timely and fair. Yup, that's a totally fair complaint. Still ... > I can see several ways going forward: > 1. We revert the removal of AIX support and carry on with the status quo > ante. (The removal of AIX is a regression; it is timely and in scope > now to revert the change.) > 2. Like (1), but we consider that notice has been given, and we will > remove it early in PG18 (like August) unless the situation improves. > 3. We leave it out of PG17 and consider a new AIX port for PG18 on its > own merits. Andres has ably summarized the reasons why the status quo ante was getting untenable. The direct-I/O problem could have been tolerable on its own, but in reality it was the straw that broke the camel's back so far as our willingness to maintain AIX support went. There were just too many hacks and workarounds for too many problems, with too few people interested in looking for better answers. So I'm totally not in favor of #1, at least not without some hard commitments and follow-through on really cleaning up the mess (which maybe looks more like your #2). What's needed here, as you said, is for someone with a decent amount of expertise in modern AIX to review all the issues. Maybe framing that as a "new port" per #3 would be a good way to think about it. But I don't want to just revert the AIX-ectomy and continue drifting. On the whole, it wouldn't be the worst thing in the world if PG 17 lacks AIX support but that comes back in PG 18. That approach would solve the schedule-crunch aspect and give time for considered review of how many of the hacks removed in 0b16bb877 really need to be put back, versus being obsolete or amenable to a nicer solution in late-model AIX. If we take a "new port" mindset then it would be totally reasonable to say that it only supports very recent AIX releases, so I'd hope at least some of the cruft could be removed. regards, tom lane
Re: AIX support
On 19.04.24 13:04, Sriram RK wrote: For any complier/hardware related issue we should able to provide support. We are in the process of identifying the AIX systems that can be added to the CI/buildfarm environment. I think we should manage expectations here, if there is any hope of getting AIX support back into PG17. I have some sympathy for this. The discussion about removing AIX support had a very short turnaround and happened in an unrelated thread, without any sort of public announcement or consultation. So this report of "hey, we were still using that" is timely and fair. But the underlying issue that led to the removal (something to do with direct I/O support and alignment) would still need to be addressed. And this probably wouldn't just need some infrastructure support; it would require contributions from someone who actively knows how to develop on this platform. Now, direct I/O is currently sort of an experimental feature, so disabling it on AIX, as was initially suggested in that discussion, might be okay for now, but the issue will come up again. Even if this new buildfarm support is forthcoming, there has to be some sort of deadline in any resurrection attempts for PG17. The first beta date has been set for 23 May. If we are making the reinstatement of AIX support contingent on new buildfarm support, those machines need to be available, at least initially, at least for backbranches, like in a week. Which seems tight. I can see several ways going forward: 1. We revert the removal of AIX support and carry on with the status quo ante. (The removal of AIX is a regression; it is timely and in scope now to revert the change.) 2. Like (1), but we consider that notice has been given, and we will remove it early in PG18 (like August) unless the situation improves. 3. We leave it out of PG17 and consider a new AIX port for PG18 on its own merits. Note that such a "new" port would probably require quite a bit of development and research work, to clean up all the cruft that had accumulated over the years in the old port. Another looming issue is that the meson build system only supported AIX with gcc before the removal. I don't know what it would take to expand that to support xclang, but if it requires meson upstream work, you have that to do, too.
Re: AIX support
For any complier/hardware related issue we should able to provide support. We are in the process of identifying the AIX systems that can be added to the CI/buildfarm environment. Regards, Sriram.
Re: AIX support
On Fri, Apr 19, 2024 at 6:01 AM Andres Freund wrote: > On 2024-04-18 11:15:43 +, Sriram RK wrote: > > We (IBM-AIX team) looked into this issue > > > > https://www.postgresql.org/message-id/20240225194322...@rfd.leadboat.com > > > > This is related to the compiler issue. The compilers xlc(13.1) and gcc(8.0) > > have issues. But we verified that this issue is resolved with the newer > > compiler versions openXL(xlc17.1) and gcc(12.0) onwards. > > The reason we used these compilers was that those were the only ones we had > kinda somewhat reasonable access to, via the gcc projects' > "compile farm" https://portal.cfarm.net/ > We have to rely on whatever the aix machines there provide. They're not > particularly plentiful resource-wise either. To be fair, those OSUOSL machines are donated by IBM: https://osuosl.org/services/powerdev/ It's just that they seem to be mostly focused on supporting Linux on POWER, with only a couple of AIX hosts (partitions/virtual machines?) made available via portal.cfarm.net, and they only very recently added a modern AIX 7.3 host. That's cfarm119, upgraded in September-ish, long after many threads on this list that between-the-lines threatened to drop support. > This is generally one of the big issues with AIX support. There are other > niche-y OSs that don't have a lot of users, e.g. openbsd, but in contrast to > AIX I can just start an openbsd VM within a few minutes and iron out whatever > portability issue I'm dealing with. Yeah. It is a known secret that you can run AIX inside Qemu/kvm (it appears that IBM has made changes to it to make that possible, because earlier AIX versions didn't like Qemu's POWER emulation or virtualisation, there are blog posts about it), but IBM doesn't actually make the images available to non-POWER-hardware owners (you need a serial number). If I were an OS vendor and wanted developers to target my OS for free, at the very top of my TODO list I would have: provide an easy to use image for developers to be able to spin something up in minutes and possibly even use in CI systems. That's the reason I can fix any minor portability issue on Linux, illumos, *BSD quickly and Windows with only moderate extra pain. Even Oracle knows this, see Solaris CBE. > > We want to make a note that postgres is used extensively in our IBM product > > and is being exploited by multiple customers. > > To be blunt: Then it'd have been nice to see some investment in that before > now. Both on the code level and the infrastructure level (i.e. access to > machines etc). In the AIX space generally, there were even clues that funding had been completely cut even for packaging PostgreSQL. I was aware of two packaging projects (not sure how they were related): 1. The ATOS packaging group, who used to show up on our mailing lists and discuss code changes, which announced it was shutting down: https://github.com/power-devops/bullfreeware 2. And last time I looked a few months back, the IBM AIX Toolbox packaging project only had PostgreSQL 10 or 11 packages, already out of support by us, meaning that their maintainer had given up, too: https://www.ibm.com/support/pages/aix-toolbox-open-source-software-downloads-alpha However I see that recently (last month?) someone has added PostgreSQL 15, so something has only just reawoken there? There are quite a lot of threads about AIX problems, but they are almost all just us non-AIX-users trying to unbreak stupid stuff on the build farm, which at some points began to seem distinctly quixotic: chivalric hackers valiantly trying to keep the entire Unix family tree working even though we don't remember why and th versions involved are out of support even by the vendor. Of the three old giant commercial Unixes, HP-UX was dropped without another mention (it really was a windmill after all), Solaris is somehow easier to deal with (I could guess it's because it influenced Linux and BSD so much, ELF and linker details spring to mind), while AIX fails on every dimension: unrepresented by users, lacking in build farm, unavailable to non-customers, and unusual among Unixen.
Re: AIX support
Hi, On 2024-04-18 11:15:43 +, Sriram RK wrote: > We (IBM-AIX team) looked into this issue > > https://www.postgresql.org/message-id/20240225194322...@rfd.leadboat.com > > This is related to the compiler issue. The compilers xlc(13.1) and gcc(8.0) > have issues. But we verified that this issue is resolved with the newer > compiler versions openXL(xlc17.1) and gcc(12.0) onwards. The reason we used these compilers was that those were the only ones we had kinda somewhat reasonable access to, via the gcc projects' "compile farm" https://portal.cfarm.net/ We have to rely on whatever the aix machines there provide. They're not particularly plentiful resource-wise either. This is generally one of the big issues with AIX support. There are other niche-y OSs that don't have a lot of users, e.g. openbsd, but in contrast to AIX I can just start an openbsd VM within a few minutes and iron out whatever portability issue I'm dealing with. Not being AIX customers we also can't raise bugs about compiler bugs, so we're stuck doing bad workarounds. > Also as part of the support, we will help in fixing all the issues related > to AIX and continue to support AIX for Postgres. If we need another CI > environment we can work to make one available. But for time being can we > start reverting the patch that has removed AIX support. The state when was removed was not in a state that I am OK with adding back. > We want to make a note that postgres is used extensively in our IBM product > and is being exploited by multiple customers. To be blunt: Then it'd have been nice to see some investment in that before now. Both on the code level and the infrastructure level (i.e. access to machines etc). Greetings, Andres Freund
Re: AIX support
> Let's start by setting up a new AIX buildfarm member. Regardless of what we > do with v17, we continue to support AIX on the stable branches, and we really > need a buildfarm member to keep the stable branches working anyway. Thanks Heikki. We had already build the source code(v17+ bcdfa5f2e2f) on our local nodes. We will try to setup the buildfarm and let you know. Is there any specific configuration we are looking for? Regards, Sriram.
Re: AIX support
On 18 April 2024 14:15:43 GMT+03:00, Sriram RK wrote: >Thanks Noah and Team, > >We (IBM-AIX team) looked into this issue > >https://www.postgresql.org/message-id/20240225194322...@rfd.leadboat.com > >This is related to the compiler issue. The compilers xlc(13.1) and gcc(8.0) >have issues. But we verified that this issue is resolved with the newer >compiler versions openXL(xlc17.1) and gcc(12.0) onwards. > >We reported this issue to the xlc team and they have noted this issue. A fix >might be possible in May for this issue in xlc v16. We would like to >understand if the community can start using the latest compilers to build the >source. > >Also as part of the support, we will help in fixing all the issues related to >AIX and continue to support AIX for Postgres. If we need another CI >environment we can work to make one available. But for time being can we start >reverting the patch that has removed AIX support. Let's start by setting up a new AIX buildfarm member. Regardless of what we do with v17, we continue to support AIX on the stable branches, and we really need a buildfarm member to keep the stable branches working anyway. >We want to make a note that postgres is used extensively in our IBM product >and is being exploited by multiple customers. Noted. I'm glad to hear you are interested to put in some effort for this. The situation from the current maintainers is that none of us have much interest, or resources or knowledge to keep the AIX build working, so we'll definitely need the help. No promises on v17, but let's at least make sure the stable branches keep working. And with the patches and buildfarm support from you, maybe v17 is feasible too. - Heikki
Re: AIX support
Thanks Noah and Team, We (IBM-AIX team) looked into this issue https://www.postgresql.org/message-id/20240225194322...@rfd.leadboat.com This is related to the compiler issue. The compilers xlc(13.1) and gcc(8.0) have issues. But we verified that this issue is resolved with the newer compiler versions openXL(xlc17.1) and gcc(12.0) onwards. We reported this issue to the xlc team and they have noted this issue. A fix might be possible in May for this issue in xlc v16. We would like to understand if the community can start using the latest compilers to build the source. Also as part of the support, we will help in fixing all the issues related to AIX and continue to support AIX for Postgres. If we need another CI environment we can work to make one available. But for time being can we start reverting the patch that has removed AIX support. We want to make a note that postgres is used extensively in our IBM product and is being exploited by multiple customers. Please let us know if there are any specific details you'd like to discuss further. Regards, Sriram.
Re: AIX support
On Fri, Apr 05, 2024 at 04:12:06PM +, Sriram RK wrote: > > > What you do need to do to reproduce the described problems is > > check out the Postgres git tree and rewind to just before > > commit 0b16bb877, where we deleted AIX support. Any attempt > > to restore AIX support would have to start with reverting that > > commit (and perhaps the followup f0827b443). > Going ahead, we want to build the changes that were removed as part of > “0b16bb8776bb8”, with latest Xlc and gcc version. > > We were building the source from the postgres ftp > server(https://ftp.postgresql.org/pub/source/), would like to understand if > there are any source level changes between the ftp server and the source on > github? To succeed in this endeavor, you'll need to develop fluency in the tools to answer questions like that, or bring in someone who is fluent. In this case, GNU diff is the standard tool for answering your question. These resources cover other topics you would need to learn: https://wiki.postgresql.org/wiki/Developer_FAQ https://wiki.postgresql.org/wiki/So,_you_want_to_be_a_developer%3F
Re: AIX support
> What you do need to do to reproduce the described problems is > check out the Postgres git tree and rewind to just before > commit 0b16bb877, where we deleted AIX support. Any attempt > to restore AIX support would have to start with reverting that > commit (and perhaps the followup f0827b443). > regards, tom lane Hi Team, thank you for all the info. We progressed to build the source on our nodes and the build was successful with the below configuration. Postgres - github-bcdfa5f2e2f AIX - 71c Xlc - 13.1.0 Bison- 3.0.5 Going ahead, we want to build the changes that were removed as part of “0b16bb8776bb8”, with latest Xlc and gcc version. We were building the source from the postgres ftp server(https://ftp.postgresql.org/pub/source/), would like to understand if there are any source level changes between the ftp server and the source on github? Regards, Sriram. From: Tom Lane Date: Friday, 29 March 2024 at 9:03 AM To: Thomas Munro Cc: Noah Misch , Sriram RK , Alvaro Herrera , pgsql-hack...@postgresql.org Subject: Re: AIX support Thomas Munro writes: > Oh, sorry, I had missed the part where newer compilers fix the issue > too. Old out-of-support versions of AIX running old compilers, what > fun. Indeed. One of the topics that needs investigation if you want to pursue this is which AIX system and compiler versions still deserve support, and which of the AIX hacks we had been carrying still need to be there based on that analysis. For context, we've been pruning support for extinct-in-the-wild OS versions pretty aggressively over the past couple of years, and I'd expect to apply the same standard to AIX. regards, tom lane
Re: AIX support
Thomas Munro writes: > Oh, sorry, I had missed the part where newer compilers fix the issue > too. Old out-of-support versions of AIX running old compilers, what > fun. Indeed. One of the topics that needs investigation if you want to pursue this is which AIX system and compiler versions still deserve support, and which of the AIX hacks we had been carrying still need to be there based on that analysis. For context, we've been pruning support for extinct-in-the-wild OS versions pretty aggressively over the past couple of years, and I'd expect to apply the same standard to AIX. regards, tom lane
Re: AIX support
On Fri, Mar 29, 2024 at 4:00 PM Thomas Munro wrote: > On Fri, Mar 29, 2024 at 3:48 PM Noah Misch wrote: > > The thread Alvaro and Tom cited contains an analysis. It's a compiler bug. > > You can get past the compiler bug by upgrading your compiler; both ibm-clang > > 17.1.1.2 and gcc 13.2.0 are free from the bug. > > For the specific issue that triggered that, I strongly suspect that it > would go away if we just used smgrzeroextend() instead of smgrextend() > using that variable with the alignment requirement, since, as far as I > can tell from build farm clues, the otherwise similar function-local > static variable used by the former (ie one that the linker must still > control the location of AFAIK?) seems to work fine. Oh, sorry, I had missed the part where newer compilers fix the issue too. Old out-of-support versions of AIX running old compilers, what fun.
Re: AIX support
Noah Misch writes: > On Thu, Mar 28, 2024 at 11:09:43AM +, Sriram RK wrote: >> Also, would like to know if we can access the buildfarm(power machines) to >> run any of the specific tests to hit this assert. > https://portal.cfarm.net/users/new/ is the form to request access. It lists > the eligibility criteria. There might be some confusion here about what system we are talking about. The Postgres buildfarm is described at https://buildfarm.postgresql.org/index.html but it consists of a large number of individual machines run by individual owners. There would not be a lot of point in adding a new AIX machine to the Postgres buildfarm right now, because it would surely fail to build HEAD. What Noah is referencing is the GCC compile farm, which happens to include some AIX machines. The existing AIX entries in the Postgres buildfarm are run (by Noah) on those GCC compile farm machines, which really the GCC crowd have been *very* forgiving about letting us abuse like that. If you have your own AIX hardware there's not a lot of reason that you should need to access the GCC farm. What you do need to do to reproduce the described problems is check out the Postgres git tree and rewind to just before commit 0b16bb877, where we deleted AIX support. Any attempt to restore AIX support would have to start with reverting that commit (and perhaps the followup f0827b443). regards, tom lane
Re: AIX support
On Fri, Mar 29, 2024 at 3:48 PM Noah Misch wrote: > On Thu, Mar 28, 2024 at 11:09:43AM +, Sriram RK wrote: > > We are setting up the build environment and trying to build the source and > > also trying to analyze the assert from the Aix point of view. > > The thread Alvaro and Tom cited contains an analysis. It's a compiler bug. > You can get past the compiler bug by upgrading your compiler; both ibm-clang > 17.1.1.2 and gcc 13.2.0 are free from the bug. For the specific issue that triggered that, I strongly suspect that it would go away if we just used smgrzeroextend() instead of smgrextend() using that variable with the alignment requirement, since, as far as I can tell from build farm clues, the otherwise similar function-local static variable used by the former (ie one that the linker must still control the location of AFAIK?) seems to work fine. But we didn't remove AIX because of that, it was just the straw that broke the camel's back.
Re: AIX support
On Thu, Mar 28, 2024 at 11:09:43AM +, Sriram RK wrote: > We are setting up the build environment and trying to build the source and > also trying to analyze the assert from the Aix point of view. The thread Alvaro and Tom cited contains an analysis. It's a compiler bug. You can get past the compiler bug by upgrading your compiler; both ibm-clang 17.1.1.2 and gcc 13.2.0 are free from the bug. > Also, would like to know if we can access the buildfarm(power machines) to > run any of the specific tests to hit this assert. https://portal.cfarm.net/users/new/ is the form to request access. It lists the eligibility criteria.
Re: AIX support
Hi Team, We are setting up the build environment and trying to build the source and also trying to analyze the assert from the Aix point of view. Also, would like to know if we can access the buildfarm(power machines) to run any of the specific tests to hit this assert. Thanks & regards, Sriram. > From: Sriram RK > Date: Thursday, 21 March 2024 at 10:05 PM > To: Tom Lane t...@sss.pgh.pa.us<mailto:t...@sss.pgh.pa.us>, Alvaro Herrera > Cc: pgsql-hack...@postgresql.org<mailto:pgsql-hack...@postgresql.org> > Subject: Re: AIX support > Thanks, Tom and Alvaro, for the info. > We shall look into to details and get back.
Re: AIX support
Thanks, Tom and Alvaro, for the info. We shall look into to details and get back. From: Tom Lane Date: Thursday, 21 March 2024 at 7:27 PM To: Sriram RK Cc: pgsql-hack...@postgresql.org Subject: Re: AIX support Sriram RK writes: > We are working on AIX systems and noticed that the thread on removing AIX > support in Postgres going forward. > https://github.com/postgres/postgres/commit/0b16bb8776bb834eb1ef8204ca95dd7667ab948b > We would be glad to understand any outstanding issues hindering the > support on AIX. Did you read the linked thread? Starting say about here: https://www.postgresql.org/message-id/flat/20240224172345.32%40rfd.leadboat.com#8b41e50c2190c82c861d91644eed9c30 > Also we would like to provide any feasible support from our end for enabling > the support on AIX. Who is "we", and how much resources are you prepared to put into this? > We would request the community to extend the support on AIX .. The community, in the sense of the existing people doing significant work on Postgres, are absolutely not going to do that. If you can bring a bunch of work to fix all the problems noted in the discussion thread, and commit to providing ongoing development manpower and hardware to keep it working, maybe something could happen. But I suspect you will find it cheaper to start thinking about migrating off AIX. regards, tom lane
Re: AIX support
Sriram RK writes: > We are working on AIX systems and noticed that the thread on removing AIX > support in Postgres going forward. > https://github.com/postgres/postgres/commit/0b16bb8776bb834eb1ef8204ca95dd7667ab948b > We would be glad to understand any outstanding issues hindering the > support on AIX. Did you read the linked thread? Starting say about here: https://www.postgresql.org/message-id/flat/20240224172345.32%40rfd.leadboat.com#8b41e50c2190c82c861d91644eed9c30 > Also we would like to provide any feasible support from our end for enabling > the support on AIX. Who is "we", and how much resources are you prepared to put into this? > We would request the community to extend the support on AIX .. The community, in the sense of the existing people doing significant work on Postgres, are absolutely not going to do that. If you can bring a bunch of work to fix all the problems noted in the discussion thread, and commit to providing ongoing development manpower and hardware to keep it working, maybe something could happen. But I suspect you will find it cheaper to start thinking about migrating off AIX. regards, tom lane
Re: AIX support
On 2024-Mar-21, Sriram RK wrote: > Hello Team, > > We are working on AIX systems and noticed that the thread on removing AIX > support in Postgres going forward. > > https://github.com/postgres/postgres/commit/0b16bb8776bb834eb1ef8204ca95dd7667ab948b” > > We would be glad to understand any outstanding issues hindering the support > on AIX. There's a Discussion link at the bottom of that commit message. I suggest you read that discussion complete, and consider how much effort you or your company are willing to spend on doing the maintenance of the port yourselves for the community. Maybe ponder this question: would it be less onerous to migrate your Postgres servers to Linux, like Phil Florent described on the currently-last message of that thread? -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "Para tener más hay que desear menos"
AIX support
Hello Team, We are working on AIX systems and noticed that the thread on removing AIX support in Postgres going forward. https://github.com/postgres/postgres/commit/0b16bb8776bb834eb1ef8204ca95dd7667ab948b” We would be glad to understand any outstanding issues hindering the support on AIX. It is important for us to have Postgres to be supported on AIX. As we are using Postgres extensively on AIX. Also we would like to provide any feasible support from our end for enabling the support on AIX. We would request the community to extend the support on AIX .. Thanks & regards, Sriram.
RE: Remove AIX Support (was: Re: Relation bulk write facility)
Hi, Historically many public hospitals I work for had IBM Power hardware. The SMT8 (8 threads/cores) capabilities of Power CPU are useful to lower Oracle licence & support cost. We migrate to PostgreSQL and it runs very well on Power, especially since the (relatively) recent parallel executions features of the RDBMS match very well the CPU capabilities. We chose to run PostgreSQL on Debian/Power (Little Endian) since ppc64le is an official Debian port. No AIX then. Only problem is that we still need to access Oracle databases and it can be useful to read directly with oracle_fdw but this tool needs an instant client and it's not open source of course. Oracle provides a binary but they don't provide patches for Debian/Power Little Endian (strange situation...) Just to say that of course we chose Linux for PostgreSQL but sometimes things are not so easy... We could have chosen AIX and we still have a about interoperability. Best regards, Phil De : Andres Freund Envoyé : jeudi 29 février 2024 10:35 À : Michael Banck Cc : Noah Misch ; Thomas Munro ; Heikki Linnakangas ; Peter Smith ; Robert Haas ; vignesh C ; pgsql-hackers ; Melanie Plageman Objet : Re: Remove AIX Support (was: Re: Relation bulk write facility) Hi, On 2024-02-29 10:24:24 +0100, Michael Banck wrote: > On Thu, Feb 29, 2024 at 12:57:31AM -0800, Andres Freund wrote: > > On 2024-02-29 09:13:04 +0100, Michael Banck wrote: > > > The commit message says there is not a lot of user demand and that might > > > be right, but contrary to other fringe OSes that got removed like HPPA > > > or Irix, I believe Postgres on AIX is still used in production and if > > > so, probably in a mission-critical manner at some old-school > > > institutions (in fact, one of our customers does just that) and not as a > > > thought-experiment. It is probably well-known among Postgres hackers > > > that AIX support is problematic/a burden, but the current users might > > > not be aware of this. > > > > Then these users should have paid somebody to actually do maintenance work > > on > > the AIX support,o it doesn't regularly stand in the way of implementing > > various things. > > Right, absolutely. > > But: did we ever tell them to do that? I don't think it's reasonable for > them to expect to follow -hackers and jump in when somebody grumbles > about AIX being a burden somewhere deep down a thread... Well, the thing is that it's commonly going to be deep down some threads that portability problems cause pain. This is far from the only time. Just a few threads: https://postgr.es/m/CA+TgmoauCAv+p4Z57PqgVgNxsApxKs3Yh9mDLdUDB8fep-s=1...@mail.gmail.com https://postgr.es/m/CA+hUKGK=DOC+hE-62FKfZy=ybt5ulkrg3zczd-jfykm-ipn...@mail.gmail.com https://postgr.es/m/20230124165814.2njc7gnvubn2a...@awork3.anarazel.de https://postgr.es/m/2385119.1696354...@sss.pgh.pa.us https://postgr.es/m/20221005200710.luvw5evhwf6cl...@awork3.anarazel.de https://postgr.es/m/20220820204401.vrf5kejih6jofvqb%40awork3.anarazel.de https://postgr.es/m/E1oWpzF-002EG4-AG%40gemulon.postgresql.org This is far from all. The only platform rivalling AIX on the pain-caused metric is windows. And at least that can be tested via CI (or locally). We've been relying on the gcc buildfarm to be able to maintain AIX at all, and that's not a resource that scales to many users. Greetings, Andres Freund
Re: Remove AIX Support (was: Re: Relation bulk write facility)
> On 29 Feb 2024, at 10:24, Michael Banck wrote: > On Thu, Feb 29, 2024 at 12:57:31AM -0800, Andres Freund wrote: >> Then these users should have paid somebody to actually do maintenance work on >> the AIX support,o it doesn't regularly stand in the way of implementing >> various things. > > Right, absolutely. > > But: did we ever tell them to do that? I don't think it's reasonable for > them to expect to follow -hackers and jump in when somebody grumbles > about AIX being a burden somewhere deep down a thread... Having spent a fair bit of time within open source projects that companies rely on, my experience is that those companies who need to hear such news have zero interaction with the project and most of time don't even know the project community exist. That conversely also means that the project don't know they exist. If their consultants and suppliers, who have a higher probability of knowing this, hasn't told them then it's highly unlikely that anything we say will get across. -- Daniel Gustafsson
Re: Remove AIX Support (was: Re: Relation bulk write facility)
Hi, On 2024-02-29 10:24:24 +0100, Michael Banck wrote: > On Thu, Feb 29, 2024 at 12:57:31AM -0800, Andres Freund wrote: > > On 2024-02-29 09:13:04 +0100, Michael Banck wrote: > > > The commit message says there is not a lot of user demand and that might > > > be right, but contrary to other fringe OSes that got removed like HPPA > > > or Irix, I believe Postgres on AIX is still used in production and if > > > so, probably in a mission-critical manner at some old-school > > > institutions (in fact, one of our customers does just that) and not as a > > > thought-experiment. It is probably well-known among Postgres hackers > > > that AIX support is problematic/a burden, but the current users might > > > not be aware of this. > > > > Then these users should have paid somebody to actually do maintenance work > > on > > the AIX support,o it doesn't regularly stand in the way of implementing > > various things. > > Right, absolutely. > > But: did we ever tell them to do that? I don't think it's reasonable for > them to expect to follow -hackers and jump in when somebody grumbles > about AIX being a burden somewhere deep down a thread... Well, the thing is that it's commonly going to be deep down some threads that portability problems cause pain. This is far from the only time. Just a few threads: https://postgr.es/m/CA+TgmoauCAv+p4Z57PqgVgNxsApxKs3Yh9mDLdUDB8fep-s=1...@mail.gmail.com https://postgr.es/m/CA+hUKGK=DOC+hE-62FKfZy=ybt5ulkrg3zczd-jfykm-ipn...@mail.gmail.com https://postgr.es/m/20230124165814.2njc7gnvubn2a...@awork3.anarazel.de https://postgr.es/m/2385119.1696354...@sss.pgh.pa.us https://postgr.es/m/20221005200710.luvw5evhwf6cl...@awork3.anarazel.de https://postgr.es/m/20220820204401.vrf5kejih6jofvqb%40awork3.anarazel.de https://postgr.es/m/E1oWpzF-002EG4-AG%40gemulon.postgresql.org This is far from all. The only platform rivalling AIX on the pain-caused metric is windows. And at least that can be tested via CI (or locally). We've been relying on the gcc buildfarm to be able to maintain AIX at all, and that's not a resource that scales to many users. Greetings, Andres Freund
Re: Remove AIX Support (was: Re: Relation bulk write facility)
Hi, On Thu, Feb 29, 2024 at 12:57:31AM -0800, Andres Freund wrote: > On 2024-02-29 09:13:04 +0100, Michael Banck wrote: > > The commit message says there is not a lot of user demand and that might > > be right, but contrary to other fringe OSes that got removed like HPPA > > or Irix, I believe Postgres on AIX is still used in production and if > > so, probably in a mission-critical manner at some old-school > > institutions (in fact, one of our customers does just that) and not as a > > thought-experiment. It is probably well-known among Postgres hackers > > that AIX support is problematic/a burden, but the current users might > > not be aware of this. > > Then these users should have paid somebody to actually do maintenance work on > the AIX support,o it doesn't regularly stand in the way of implementing > various things. Right, absolutely. But: did we ever tell them to do that? I don't think it's reasonable for them to expect to follow -hackers and jump in when somebody grumbles about AIX being a burden somewhere deep down a thread... Michael
Re: Remove AIX Support (was: Re: Relation bulk write facility)
Hi, On 2024-02-29 09:13:04 +0100, Michael Banck wrote: > The commit message says there is not a lot of user demand and that might > be right, but contrary to other fringe OSes that got removed like HPPA > or Irix, I believe Postgres on AIX is still used in production and if > so, probably in a mission-critical manner at some old-school > institutions (in fact, one of our customers does just that) and not as a > thought-experiment. It is probably well-known among Postgres hackers > that AIX support is problematic/a burden, but the current users might > not be aware of this. Then these users should have paid somebody to actually do maintenance work on the AIX support,o it doesn't regularly stand in the way of implementing various things. Greetings, Andres Freund
Re: Remove AIX Support (was: Re: Relation bulk write facility)
> On 29 Feb 2024, at 09:13, Michael Banck wrote: > In any case, users will have a couple of years to migrate as usual if > they upgrade to v16. As you say, there are many years left of AIX being supported so there is plenty of runway for planning a migration. -- Daniel Gustafsson
Remove AIX Support (was: Re: Relation bulk write facility)
Hi, On Sat, Feb 24, 2024 at 01:29:36PM -0800, Andres Freund wrote: > Let's just drop AIX. This isn't the only alignment issue we've found and the > solution for those isn't so much a fix as forcing everyone to carefully only > look into one direction and not notice the cliffs to either side. While I am not against dropping AIX (and certainly won't step up to maintain it just for fun), I don't think burying this inside some "Relation bulk write facility" thread is helpful; I have changed the thread title as a first step. The commit message says there is not a lot of user demand and that might be right, but contrary to other fringe OSes that got removed like HPPA or Irix, I believe Postgres on AIX is still used in production and if so, probably in a mission-critical manner at some old-school institutions (in fact, one of our customers does just that) and not as a thought-experiment. It is probably well-known among Postgres hackers that AIX support is problematic/a burden, but the current users might not be aware of this. Not sure what to do about this (especially now that this has been committed), maybe there should have been be a public deprecation notice first for v17... On the other hand, that might not work if important features like direct-IO would have to be bumped from v17 just because of AIX. I posted about this on Twitter and Mastodon to see whether anybody complains and did not get a lot of feedback. In any case, users will have a couple of years to migrate as usual if they upgrade to v16. Michael
Re: AIX support - alignment issues
On Tue, Jul 12, 2022 at 10:30 AM Tom Lane wrote: > Thomas Munro writes: > > Here's a patch to remove all of these. > > Looks sane by eyeball --- I didn't grep for other references, though. Thanks, pushed. > > I didn't originally suggest that because of some kind of (mostly > > vicarious) nostalgia. I wonder if we should allow ourselves a > > paragraph where we remember these systems. I personally think it's > > one of the amazing things about this project. Here's what I came up > > with, but I'm sure there are more. > > PlayStation 2 [1]? Although I suppose that falls under MIPS, > which probably means we could still run on it, if you can find one. Yeah. PS had MIPS, then PowerPC (Cell), and currently AMD (interestingly they also run a modified FreeBSD kernel, but you can't really get at it...). Sega Dreamcast had SH4. I added one more: Tru64 (but I didn't bother to list Digital UNIX or OSF/1, not sure if software historians consider those different OSes or just rebrands...). Patches to improve this little paragraph welcome. Pushed.
Re: AIX support - alignment issues
Thomas Munro writes: > Here's a patch to remove all of these. Looks sane by eyeball --- I didn't grep for other references, though. > I didn't originally suggest that because of some kind of (mostly > vicarious) nostalgia. I wonder if we should allow ourselves a > paragraph where we remember these systems. I personally think it's > one of the amazing things about this project. Here's what I came up > with, but I'm sure there are more. PlayStation 2 [1]? Although I suppose that falls under MIPS, which probably means we could still run on it, if you can find one. regards, tom lane [1] https://www.postgresql.org/message-id/flat/05e101c1834a%24e398b920%24f90e10ac%40toronto.redhat.com
Re: AIX support - alignment issues
On Tue, Jul 12, 2022 at 7:24 AM Tom Lane wrote:
> Thomas Munro writes:
> > It's funny to think that you probably could run modern PostgreSQL on
> > the Sun 3 boxes the project started on in 1986 (based on clues from
> > the papers in our history section) if you put NetBSD on them, but
> > you'd probably need to cross compile due to lack of RAM.
>
> Yeah. I'm wondering if that sh3el package was cross-compiled,
> and if so whether it was just part of a mass package build rather
> than something somebody was specifically interested in. You'd
> have to be a glutton for pain to want to do actual work with PG
> on the kind of SH3 hardware that seems to be available.
/me pictures Stark wheeling a real Sun 3 into a conference room
Yeah, we can always consider putting SuperH back if someone showed up
to maintain/test it. That seems unlikely, but apparently there's an
open source silicon project based on this ISA, so maybe a fast one
isn't impossible...
Here's a patch to remove all of these.
I didn't originally suggest that because of some kind of (mostly
vicarious) nostalgia. I wonder if we should allow ourselves a
paragraph where we remember these systems. I personally think it's
one of the amazing things about this project. Here's what I came up
with, but I'm sure there are more.
From 9c20f8b70632a6a79333c835bcf3f4c7d427f1cf Mon Sep 17 00:00:00 2001
From: Thomas Munro
Date: Tue, 12 Jul 2022 08:04:46 +1200
Subject: [PATCH 1/2] Further tidy-up for supported CPUs.
Further to commit 92d70b77, let's drop the code we carry for the
following untested architectures: M68K, M88K, M32R, SuperH. We have no
idea if anything actually works there, and surely as vintage hardware
and microcontrollers they would be underpowered for any modern purpose.
We could always consider re-adding SuperH based on modern evidence of
usage and build farm support, if someone shows up to provide it.
While here, SPARC is usually written in all caps.
Discussion: https://postgr.es/m/959917.1657522169%40sss.pgh.pa.us
---
doc/src/sgml/installation.sgml| 5 +-
src/backend/storage/lmgr/s_lock.c | 65 ---
src/include/storage/s_lock.h | 102 --
3 files changed, 2 insertions(+), 170 deletions(-)
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index dec13eaa93..381b728e08 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -2125,11 +2125,10 @@ export MANPATH
In general, PostgreSQL can be expected to work on
- these CPU architectures: x86, PowerPC, S/390, Sparc, ARM, MIPS, RISC-V,
+ these CPU architectures: x86, PowerPC, S/390, SPARC, ARM, MIPS, RISC-V,
and PA-RISC, including
big-endian, little-endian, 32-bit, and 64-bit variants where applicable.
- Code support exists for M68K, M88K, M32R, and SuperH, but these
- architectures are not known to have been tested recently. It is often
+ It is often
possible to build on an unsupported CPU type by configuring with
--disable-spinlocks, but performance will be poor.
diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c
index baea773a02..2a658ff594 100644
--- a/src/backend/storage/lmgr/s_lock.c
+++ b/src/backend/storage/lmgr/s_lock.c
@@ -220,71 +220,6 @@ update_spins_per_delay(int shared_spins_per_delay)
}
-/*
- * Various TAS implementations that cannot live in s_lock.h as no inline
- * definition exists (yet).
- * In the future, get rid of tas.[cso] and fold it into this file.
- *
- * If you change something here, you will likely need to modify s_lock.h too,
- * because the definitions for these are split between this file and s_lock.h.
- */
-
-
-#ifdef HAVE_SPINLOCKS /* skip spinlocks if requested */
-
-
-#if defined(__GNUC__)
-
-/*
- * All the gcc flavors that are not inlined
- */
-
-
-/*
- * Note: all the if-tests here probably ought to be testing gcc version
- * rather than platform, but I don't have adequate info to know what to
- * write. Ideally we'd flush all this in favor of the inline version.
- */
-#if defined(__m68k__) && !defined(__linux__)
-/* really means: extern int tas(slock_t* **lock); */
-static void
-tas_dummy()
-{
- __asm__ __volatile__(
-#if (defined(__NetBSD__) || defined(__OpenBSD__)) && defined(__ELF__)
-/* no underscore for label and % for registers */
- "\
-.global tas \n\
-tas: \n\
- movel %sp@(0x4),%a0 \n\
- tas %a0@ \n\
- beq _success \n\
- moveq #-128,%d0 \n\
- rts \n\
-_success: \n\
- moveq #0,%d0 \n\
- rts \n"
-#else
- "\
-.global _tas\n\
-_tas: \n\
- movel sp@(0x4),a0 \n\
- tas a0@ \n\
- beq _success \n\
- moveq #-128,d0 \n\
- rts \n\
-_success: \n\
- moveq #0,d0 \n\
- rts \n"
-#endif /* (__NetBSD__ || __OpenBSD__) && __ELF__ */
- );
-}
-#endif /* __m68k__ && !__linux__ */
-#endif /* not __GNUC__ */
-#endif /* HAVE_SPINLOCKS */
-
-
-
/
Re: AIX support - alignment issues
Thomas Munro writes: > On Mon, Jul 11, 2022 at 6:49 PM Tom Lane wrote: >> SuperH might be twitching a bit less feebly than these three, >> but it seems to be a legacy architecture as well. Not much >> has happened there since the early 2000's AFAICS. > It looks like there's an sh3el package for PostgreSQL on NetBSD here, > so whoever maintains that might be in touch: > https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/databases/postgresql14-server/index.html Hm. For a moment there I was feeling bad about recommending cutting off a platform somebody still pays attention to ... but looking at the relevant NetBSD mailing list archives makes it look like that port is pretty darn moribund. > It's funny to think that you probably could run modern PostgreSQL on > the Sun 3 boxes the project started on in 1986 (based on clues from > the papers in our history section) if you put NetBSD on them, but > you'd probably need to cross compile due to lack of RAM. Yeah. I'm wondering if that sh3el package was cross-compiled, and if so whether it was just part of a mass package build rather than something somebody was specifically interested in. You'd have to be a glutton for pain to want to do actual work with PG on the kind of SH3 hardware that seems to be available. regards, tom lane
Re: AIX support - alignment issues
Robert Haas writes: > On Mon, Jul 11, 2022 at 2:49 AM Tom Lane wrote: >> I think it'd be pretty reasonable to disclaim support for >> any architecture that doesn't have a representative in our >> buildfarm, which would lead to dropping all four of these. >> If you don't like it, step up and run a buildfarm animal. > I strongly suspect that anyone who tried to use a modern PostgreSQL on > any of these platforms would find it quite an adventure, which is > fine, because if you're trying to use any of those platforms in 2022, > you are probably the sort of person who enjoys an adventure. But it > can't really be useful to list them in the documentation, and it's > unlikely that any of them "just work". It's possible that they "just work", but we have no way of knowing that, or knowing if we break them in future. Thus the importance of having a buildfarm animal to tell us that. More generally, I think the value of carrying support for niche architectures is that it helps keep us from falling into the software-monoculture trap, from which we'd be unable to escape when the hardware landscape inevitably changes. However, it only helps if somebody is testing such arches on a regular basis. The fact that there's some #ifdef'd code somewhere for M88K proves diddly-squat about whether we could actually run on M88K today. The situation for niche operating systems is precisely analogous. regards, tom lane
Re: AIX support - alignment issues
On Mon, Jul 11, 2022 at 2:49 AM Tom Lane wrote: > While we're here ... > > + Code support exists for M68K, M88K, M32R, and SuperH, but these > architectures are not known to have been tested recently. > > I think it'd be pretty reasonable to disclaim support for > any architecture that doesn't have a representative in our > buildfarm, which would lead to dropping all four of these. > If you don't like it, step up and run a buildfarm animal. +1. Keeping stuff like this in the documentation doesn't make those platforms supported. What it does do is make it look like we're bad at updating our documentation. I strongly suspect that anyone who tried to use a modern PostgreSQL on any of these platforms would find it quite an adventure, which is fine, because if you're trying to use any of those platforms in 2022, you are probably the sort of person who enjoys an adventure. But it can't really be useful to list them in the documentation, and it's unlikely that any of them "just work". -- Robert Haas EDB: http://www.enterprisedb.com
Re: AIX support - alignment issues
On Mon, Jul 11, 2022 at 6:49 PM Tom Lane wrote: > SuperH might be twitching a bit less feebly than these three, > but it seems to be a legacy architecture as well. Not much > has happened there since the early 2000's AFAICS. It looks like there's an sh3el package for PostgreSQL on NetBSD here, so whoever maintains that might be in touch: https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/databases/postgresql14-server/index.html > I think it'd be pretty reasonable to disclaim support for > any architecture that doesn't have a representative in our > buildfarm, which would lead to dropping all four of these. > If you don't like it, step up and run a buildfarm animal. +1 It's funny to think that you probably could run modern PostgreSQL on the Sun 3 boxes the project started on in 1986 (based on clues from the papers in our history section) if you put NetBSD on them, but you'd probably need to cross compile due to lack of RAM. The grammar in particular.
Re: AIX support - alignment issues
Thomas Munro writes: > Yeah. I wasn't too sure if that was mostly about "recent" or mostly > about "all distributions" but it wasn't doing much. Thanks, pushed. While we're here ... + Code support exists for M68K, M88K, M32R, and SuperH, but these architectures are not known to have been tested recently. I confess great fondness for M68K, having spent a goodly chunk of the eighties hacking M68K assembly code. However, of these four architectures, I fear only SuperH has anything resembling a detectable pulse. According to Wikipedia: * Motorola ended development of M68K in 1994. The last processors had clock rates around 75MHz (and this was a CISC architecture, so instruction rates were a good bit less). Considering how depressingly slow my late-90s 360MHz HPPA box is, it's impossible to believe that anyone wants to run PG on M68K today. * M88K was introduced in 1988 and discontinued in 1991. Max clock rate was apparently somewhere under 100MHz, and in any case it's hard to believe that any remain alive in the wild. * M32R ... hard to tell for sure, because Wikipedia's only concrete info is a link to a 404 page at renasas.com. But they do say that the Linux kernel dropped support for it some years ago. SuperH might be twitching a bit less feebly than these three, but it seems to be a legacy architecture as well. Not much has happened there since the early 2000's AFAICS. I think it'd be pretty reasonable to disclaim support for any architecture that doesn't have a representative in our buildfarm, which would lead to dropping all four of these. If you don't like it, step up and run a buildfarm animal. (The same policy could be applied to operating systems, but it looks like we're good on that side.) regards, tom lane
Re: AIX support - alignment issues
On Mon, Jul 11, 2022 at 11:38 AM Tom Lane wrote: > WFM. I also wonder if in > > + PostgreSQL can be expected to work on current > + versions of these operating systems: Linux (all recent distributions), > Windows, > + FreeBSD, OpenBSD, NetBSD, DragonFlyBSD, macOS, AIX, Solaris, and illumos. > > we could drop "(all recent distributions)", figuring that "current > versions" covers that already. Other than that niggle, this > looks good to me. Yeah. I wasn't too sure if that was mostly about "recent" or mostly about "all distributions" but it wasn't doing much. Thanks, pushed.
Re: AIX support - alignment issues
Thomas Munro writes: > OK, I word-smothe thusly: > + and PA-RISC, including > + big-endian, little-endian, 32-bit, and 64-bit variants where applicable. WFM. I also wonder if in + PostgreSQL can be expected to work on current + versions of these operating systems: Linux (all recent distributions), Windows, + FreeBSD, OpenBSD, NetBSD, DragonFlyBSD, macOS, AIX, Solaris, and illumos. we could drop "(all recent distributions)", figuring that "current versions" covers that already. Other than that niggle, this looks good to me. regards, tom lane
Re: AIX support - alignment issues
On Fri, Jul 8, 2022 at 4:24 PM Tom Lane wrote:
> Thomas Munro writes:
> > * The documented list mentions some in different endiannesses and word
> > sizes explicitly but not others; I think it'd be tidier to list the
> > main architecture names and then tack on a "big and little endian, 32
> > and 64 bit" sentence.
>
> As phrased, this seems to be saying that we can do both
> endiannesses on any of the supported arches, which is a little
> weird considering that most of them are single-endianness. It's
> not a big deal, but maybe a tad more word-smithing there would
> help?
OK, I word-smothe thusly:
+ and PA-RISC, including
+ big-endian, little-endian, 32-bit, and 64-bit variants where applicable.
I also realised that we should list a couple more OSes (we know they
work, they are automatically tested). Then I wondered why we bother
to state a Windows version here. For consistency, we could list the
minimum Linux kernel, and so on for every other OS, but that's silly
for such brief and general documentation. So I propose that we just
say "current versions of ..." and remove the bit about Windows 10.
From 395e31b5ddb142230b9ffaf60af1f56fea46383d Mon Sep 17 00:00:00 2001
From: Thomas Munro
Date: Fri, 8 Jul 2022 10:19:32 +1200
Subject: [PATCH v2] Tidy up claimed supported CPUs and OSes.
* Remove arbitrary mention of certain endianness and bitness variants;
it's enough to say that applicable variants are expected to work.
* List RISC-V (known to work, being tested).
* List SuperH and M88K (code exists, unknown status, like M68K).
* De-list VAX and remove code (known not to work).
* Remove stray traces of Alpha (support was removed years ago).
* List illumos, DragonFlyBSD (known to work, being tested).
* No need to single Windows out by listing a specific version, when we
don't do that for other OSes; it's enough to say that we support
current versions of the listed OSes (when PG16 ships, that'll be
Windows 10+).
Reviewed-by: Tom Lane
Reviewed-by: Greg Stark
Discussion: https://postgr.es/m/CA%2BhUKGKk7NZO1UnJM0PyixcZPpCGqjBXW_0bzFZpJBGAf84XKg%40mail.gmail.com
---
contrib/pgcrypto/crypt-blowfish.c | 2 +-
doc/src/sgml/installation.sgml| 13 +++--
src/include/storage/s_lock.h | 30 --
3 files changed, 8 insertions(+), 37 deletions(-)
diff --git a/contrib/pgcrypto/crypt-blowfish.c b/contrib/pgcrypto/crypt-blowfish.c
index a663852ccf..1264eccb3f 100644
--- a/contrib/pgcrypto/crypt-blowfish.c
+++ b/contrib/pgcrypto/crypt-blowfish.c
@@ -41,7 +41,7 @@
#ifdef __i386__
#define BF_ASM0 /* 1 */
#define BF_SCALE 1
-#elif defined(__x86_64__) || defined(__alpha__) || defined(__hppa__)
+#elif defined(__x86_64__) || defined(__hppa__)
#define BF_ASM0
#define BF_SCALE 1
#else
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 1a1343a008..09cd9d8b9c 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -2125,18 +2125,19 @@ export MANPATH
In general, PostgreSQL can be expected to work on
- these CPU architectures: x86, x86_64, PowerPC,
- PowerPC 64, S/390, S/390x, Sparc, Sparc 64, ARM, MIPS, MIPSEL,
- and PA-RISC. Code support exists for M68K, M32R, and VAX, but these
+ these CPU architectures: x86, PowerPC, S/390, Sparc, ARM, MIPS, RISC-V,
+ and PA-RISC, including
+ big-endian, little-endian, 32-bit, and 64-bit variants where applicable.
+ Code support exists for M68K, M88K, M32R, and SuperH, but these
architectures are not known to have been tested recently. It is often
possible to build on an unsupported CPU type by configuring with
--disable-spinlocks, but performance will be poor.
- PostgreSQL can be expected to work on these operating
- systems: Linux (all recent distributions), Windows (10 and later),
- FreeBSD, OpenBSD, NetBSD, macOS, AIX, and Solaris.
+ PostgreSQL can be expected to work on current
+ versions of these operating systems: Linux (all recent distributions), Windows,
+ FreeBSD, OpenBSD, NetBSD, DragonFlyBSD, macOS, AIX, Solaris, and illumos.
Other Unix-like systems may also work but are not currently
being tested. In most cases, all CPU architectures supported by
a given operating system will work. Look in
diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h
index c4a19b2f43..1f5394ef7f 100644
--- a/src/include/storage/s_lock.h
+++ b/src/include/storage/s_lock.h
@@ -548,36 +548,6 @@ tas(volatile slock_t *lock)
#endif /* __m88k__ */
-/*
- * VAXen -- even multiprocessor ones
- * (thanks to Tom Ivar Helbekkmo)
- */
-#if defined(__vax__)
-#define HAS_TEST_AND_SET
-
-typedef unsigned char slock_t;
-
-#define TAS(lock) tas(lock)
-
-static __inline__ int
-tas(volatile slock_t *lock)
-{
- register int _res;
-
- __asm__ __volatile__(
- " movl $1, %0 \n"
- " bbssi $0, (%2), 1f \n"
- " clrl %0\n"
- "1: \n"
-: "=&r"(_res), "+m"(*lock)
-: "r"(lock)
-:
Re: AIX support - alignment issues
On Tue, Jul 5, 2022 at 1:32 AM Andres Freund wrote:
> I just thought an easier way - why don't we introduce a 'catalog_double'
> that's defined to be pg_attribute_aligned(whatever-we-need) on AIX? Then we
> can get rid of the manually enforced alignedness and we don't need to contort
> catalog order.
I investigated this a little bit today. It seems that
att_align_nominal() thinks that typalign=='d' means ALIGNOF_DOUBLE,
which on AIX is 4. So I think what we would need to do first is
redefine typalign=='d' to mean alignment to MAXIMUM_ALIGNOF. If we
don't do that, then there's no automatic way to get uint64 fields to
be placed on 8-byte boundaries, which it requires. Such a change would
have no effect on many systems, but if as on AIX double requires less
alignment than either "long" or "long long int", it will break on-disk
compatibility and in particular pg_upgrade compatibility.
If we did that, then we could pursue your proposal above. Rather than
creating an altogether new typedef, we could just apply
pg_attribute_aligned(MAXIMUM_ALIGNOF) to the existing typedef for
float8, which is documented as being the name that should be used in
the catalogs, and is. Since pg_attribute_aligned() is not supported on
all platforms, we elsewhere apply it conditionally, so we would
presumably do the same thing here. That would mean that it might fail
to apply on some platform somewhere, but we could compensate for that
by adding a static assertion checking that if we do struct
float8_alignmment_test { char pad; float8 x; } then
alignof(float8_alignment_test, x) == MAXIMUM_ALIGNOF. That way, if
pg_attribute_aligned() isn't supported but the platform doesn't have
this issue in the first place, all is well. If pg_attribute_aligned()
isn't supported and the platform does have this issue, compilation
will fail.
In theory, we could have the same issue with int64 on some other
platform. On this hypothetical system, ALIGNOF_LONG_LONG_INT <
ALIGNOF_DOUBLE. The compile would then align int64 catalog columns on,
say, 4-byte boundaries, but our tuple deforming code would think that
they were aligned to 8 byte boundaries. We could fix that by forcing
the int64 type to have maximum alignment as well or introducing a new
typedef that does. However, such a fix could probably be postponed
until such time as a system of this kind turns up. It might never
happen.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: AIX support - alignment issues
On Thu, 7 Jul 2022 at 22:36, Thomas Munro wrote: > > * Since Greg Stark's magnificent Vax talk[1], we became even more > dependent on IEEE 754 via the Ryu algorithm. AFAICT, unless someone > produces a software IEEE math implementation for GCC/VAX... if I had > a pick one to bump off that list, that's the easiest to argue because > it definitely doesn't work. Yeah that's definitely true. I think you could possibly build with a software fp implementation but then you would have to recompile libc and any other libraries as well. If it was worth spending a lot of effort we could perhaps separate the Float4/Float8 data type from the core C code floating point and compile with just the former using soft floats but use native floats for core code. That's probably way more effort than it's worth for VAX but it would conceivably be worthwhile if it helped for running on some embedded platform but I don't think so since they would presumably be using soft floats everywhere anyways. -- greg
Re: AIX support - alignment issues
Thomas Munro writes: > * The documented list mentions some in different endiannesses and word > sizes explicitly but not others; I think it'd be tidier to list the > main architecture names and then tack on a "big and little endian, 32 > and 64 bit" sentence. As phrased, this seems to be saying that we can do both endiannesses on any of the supported arches, which is a little weird considering that most of them are single-endianness. It's not a big deal, but maybe a tad more word-smithing there would help? > * Since Greg Stark's magnificent Vax talk[1], we became even more > dependent on IEEE 754 via the Ryu algorithm. AFAICT, unless someone > produces a software IEEE math implementation for GCC/VAX... if I had > a pick one to bump off that list, that's the easiest to argue because > it definitely doesn't work. Agreed. In principle I'd wish that we were not tied to one floating-point format, but the benefits of Ryu are too hard to pass up; and reality on the ground is that IEEE 754 achieved total victory a couple decades ago. We should stop claiming that VAX is a realistic target platform. > What do you think about the attached? WFM. Also, that crypt-blowfish.c hunk offers an answer to your question about whether to worry about "__hppa". regards, tom lane
Re: AIX support - alignment issues
On Thu, Jul 7, 2022 at 1:02 AM Peter Eisentraut
wrote:
> On 06.07.22 04:21, Thomas Munro wrote:
> > /*
> >* Do not try to collapse these into one "w+" mode file. Doesn't work
> > on
> > - * some platforms (eg, HPUX 10.20).
> > + * some platforms.
> >*/
> > termin = fopen("/dev/tty", "r");
> > termout = fopen("/dev/tty", "w");
>
> I don't know how /dev/tty behaves in detail under stdio. I think
> removing this part of the comment might leave the impression that
> attempting to use "w+" will never work, whereas the existing comment
> appears to indicate that it was only very old platforms that had the
> issue. If we don't have an immediate answer to that, I'd leave the
> comment as is.
Thanks. I put that bit back, removed the stray mention of "itanium"
in Windows-specific stuff that Andres mentioned, and pushed these
patches.
While adjusting the docs, I noticed a few little inconsistencies here
and there for other ISAs.
* The documented list of ISAs should by now mention RISC-V. I'm sure
it needs some fine tuning but it's working fine and tested by the
build farm.
* The documented list mentions some in different endiannesses and word
sizes explicitly but not others; I think it'd be tidier to list the
main architecture names and then tack on a "big and little endian, 32
and 64 bit" sentence.
* Under "code exists, not tested" we mentioned M68K, M32R, VAX, but
M88K and SuperH are also in that category and have been added/tweaked
in the past decade with reports that imply that they were working on
retro-gear. AFAIK only SuperH-family stuff is still produced. I
don't know much about that and I'm not planning to change anything,
except one special mention...
* Since Greg Stark's magnificent Vax talk[1], we became even more
dependent on IEEE 754 via the Ryu algorithm. AFAICT, unless someone
produces a software IEEE math implementation for GCC/VAX... if I had
a pick one to bump off that list, that's the easiest to argue because
it definitely doesn't work.
* When we removed Alpha we left a couple of traces.
What do you think about the attached?
[1] https://archive.fosdem.org/2016/schedule/event/postgresql_on_vax/
From 3f8dc52fdec8a9a09017d9542d84d78715ec0223 Mon Sep 17 00:00:00 2001
From: Thomas Munro
Date: Fri, 8 Jul 2022 10:19:32 +1200
Subject: [PATCH] Tidy up claimed supported CPUs.
* Be more consistent about listing endianness and bitness variants.
* Mentioned RISC-V.
* SuperH and M88K might as well be mentioned, if M68K is.
* VAX definitely doesn't work anymore, so remove it from docs and code.
* Remaining traces of Alpha support can also be removed from code.
---
contrib/pgcrypto/crypt-blowfish.c | 2 +-
doc/src/sgml/installation.sgml| 7 ---
src/include/storage/s_lock.h | 30 --
3 files changed, 5 insertions(+), 34 deletions(-)
diff --git a/contrib/pgcrypto/crypt-blowfish.c b/contrib/pgcrypto/crypt-blowfish.c
index a663852ccf..1264eccb3f 100644
--- a/contrib/pgcrypto/crypt-blowfish.c
+++ b/contrib/pgcrypto/crypt-blowfish.c
@@ -41,7 +41,7 @@
#ifdef __i386__
#define BF_ASM0 /* 1 */
#define BF_SCALE 1
-#elif defined(__x86_64__) || defined(__alpha__) || defined(__hppa__)
+#elif defined(__x86_64__) || defined(__hppa__)
#define BF_ASM0
#define BF_SCALE 1
#else
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 1a1343a008..eff180b216 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -2125,9 +2125,10 @@ export MANPATH
In general, PostgreSQL can be expected to work on
- these CPU architectures: x86, x86_64, PowerPC,
- PowerPC 64, S/390, S/390x, Sparc, Sparc 64, ARM, MIPS, MIPSEL,
- and PA-RISC. Code support exists for M68K, M32R, and VAX, but these
+ these CPU architectures: x86, PowerPC, S/390, Sparc, ARM, MIPS, RISC-V,
+ and PA-RISC.
+ Big-endian, little-endian, 32-bit and 64-bit systems are supported.
+ Code support exists for M68K, M88K, M32R, and SuperH, but these
architectures are not known to have been tested recently. It is often
possible to build on an unsupported CPU type by configuring with
--disable-spinlocks, but performance will be poor.
diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h
index c4a19b2f43..1f5394ef7f 100644
--- a/src/include/storage/s_lock.h
+++ b/src/include/storage/s_lock.h
@@ -548,36 +548,6 @@ tas(volatile slock_t *lock)
#endif /* __m88k__ */
-/*
- * VAXen -- even multiprocessor ones
- * (thanks to Tom Ivar Helbekkmo)
- */
-#if defined(__vax__)
-#define HAS_TEST_AND_SET
-
-typedef unsigned char slock_t;
-
-#define TAS(lock) tas(lock)
-
-static __inline__ int
-tas(volatile slock_t *lock)
-{
- register int _res;
-
- __asm__ __volatile__(
- " movl $1, %0 \n"
- " bbssi $0, (%2), 1f \n"
- " clrl %0\n"
- "1: \n"
-: "=&r"(_res), "+m"(*lock)
-: "r"(lock)
-: "memory");
- return _res;
-}
-
-#endif /* __vax__ */
-
-
#if defi
Re: AIX support - alignment issues
On Wed, Jul 6, 2022 at 12:27 PM Andres Freund wrote: > I think my proposal of introducing a version of double that is marked to be 8 > byte aligned should do the trick as well, and doesn't have the problem of > changing the meaning of 'double' references in external headers. In fact, we > already have float8 as a type, so we could just add it there. Yeah, but how easily will it be to know whether we've used that in every relevant place? Could we insist on 8-byte alignment even on 32-bit platforms? I think we have a few of those in the buildfarm, so maybe that would help us spot problems. Although I'm not sure how, exactly. > The problem with having a lot more alignment values is that it adds a bunch of > overhead to very performance critical paths. We don't want to add more > branches to att_align_nominal() if we can avoid it. Fair. > I'm fairly certain that we're going to add a lot more 64bit ints to catalogs > in the next few years, so this will become a bigger issue over time... Absolutely. > Outside of the catalogs I still think that we should work towards not aligning > byval values (and instead memcpy-ing the values to deal with alignment > sensitive platforms), so we don't waste so much space. And for catalogs we've > been talking about giving up the struct mapping as well, in the thread about > variable length names. In which case we could the cost of handling more > alignment values wouldn't be incurred as frequently. +1. Aligning stuff on disk appears to have few redeeming properties for the amount of pain it causes. -- Robert Haas EDB: http://www.enterprisedb.com
Re: AIX support - alignment issues
Hi, On 2022-07-06 11:55:57 -0400, Robert Haas wrote: > On Sat, Jul 2, 2022 at 2:34 PM Andres Freund wrote: > > I strikes me as a remarkably bad idea to manually try to maintain the > > correct > > alignment. Even with the tests added it's still quite manual and requires > > contorted struct layouts (see e.g. [1]). > > > > I think we should either teach our system the correct alignment rules or we > > should drop AIX support. > > I raised this same issue at > http://postgr.es/m/CA+TgmoaK377MXCWJqEXM3VvKDDC-frNUMKb=7u07tja59wt...@mail.gmail.com > and discussion ensued from there. I agree that manually maintaining > alignment, even with a regression test to help, is a really bad plan. > > The rule about columns of type "name" can be relaxed easily enough, > just by insisting that NAMEDATALEN must be a multiple of 8. As Tom > also said on this thread, adding such a constraint seems to have no > real downside. But the problem has a second aspect not related to > NameData, which is that int64 and double have different alignment > requirements on that platform. To get out from under that part of it, > it seems we either need to de-support AIX and any other platforms that > have such a discrepancy, or else have separate typalign values for > int64-align vs. double-align. I think my proposal of introducing a version of double that is marked to be 8 byte aligned should do the trick as well, and doesn't have the problem of changing the meaning of 'double' references in external headers. In fact, we already have float8 as a type, so we could just add it there. We don't currently have a float8 in the catalogs afaics, but I think it'd be better to not rely on that. It's not pretty, but still seems a lot better than doing this stuff manually. > From a theoretical point of view, I think what we're doing now is > pretty unprincipled. I've always found it a bit surprising that we get > away with just assuming that a bunch of various different primitive > data types are all going to have the same alignment requirement. The > purist in me feels that it would be better to have separate typalign > values for things that aren't guaranteed to behave the same. However, > there's a practical difficulty with that approach: if the only > operating system where this issue occurs in practice is AIX, I feel > it's going to be pretty hard for us to keep the code that caters to > this unusual situation working properly. And I'd rather have no code > for it at all than have code which doesn't really work. The problem with having a lot more alignment values is that it adds a bunch of overhead to very performance critical paths. We don't want to add more branches to att_align_nominal() if we can avoid it. I guess we can try to introduce TYPALIGN_INT64 and then hide the relevant branch with an ifdef for the common case of TYPALIGN_INT64 == TYPALIGN_DOUBLE. I'm fairly certain that we're going to add a lot more 64bit ints to catalogs in the next few years, so this will become a bigger issue over time... Outside of the catalogs I still think that we should work towards not aligning byval values (and instead memcpy-ing the values to deal with alignment sensitive platforms), so we don't waste so much space. And for catalogs we've been talking about giving up the struct mapping as well, in the thread about variable length names. In which case we could the cost of handling more alignment values wouldn't be incurred as frequently. Greetings, Andres Freund
Re: AIX support - alignment issues
On Sat, Jul 2, 2022 at 2:34 PM Andres Freund wrote: > I strikes me as a remarkably bad idea to manually try to maintain the correct > alignment. Even with the tests added it's still quite manual and requires > contorted struct layouts (see e.g. [1]). > > I think we should either teach our system the correct alignment rules or we > should drop AIX support. I raised this same issue at http://postgr.es/m/CA+TgmoaK377MXCWJqEXM3VvKDDC-frNUMKb=7u07tja59wt...@mail.gmail.com and discussion ensued from there. I agree that manually maintaining alignment, even with a regression test to help, is a really bad plan. The rule about columns of type "name" can be relaxed easily enough, just by insisting that NAMEDATALEN must be a multiple of 8. As Tom also said on this thread, adding such a constraint seems to have no real downside. But the problem has a second aspect not related to NameData, which is that int64 and double have different alignment requirements on that platform. To get out from under that part of it, it seems we either need to de-support AIX and any other platforms that have such a discrepancy, or else have separate typalign values for int64-align vs. double-align. >From a theoretical point of view, I think what we're doing now is pretty unprincipled. I've always found it a bit surprising that we get away with just assuming that a bunch of various different primitive data types are all going to have the same alignment requirement. The purist in me feels that it would be better to have separate typalign values for things that aren't guaranteed to behave the same. However, there's a practical difficulty with that approach: if the only operating system where this issue occurs in practice is AIX, I feel it's going to be pretty hard for us to keep the code that caters to this unusual situation working properly. And I'd rather have no code for it at all than have code which doesn't really work. -- Robert Haas EDB: http://www.enterprisedb.com
Re: AIX support - alignment issues
I wrote: > Our HEAD does work on that NetBSD installation. I can try this > patch, but it'll take an hour or two to get results ... stay tuned. Indeed, I still get a clean build and "make check" passes with this patch. regards, tom lane
Re: AIX support - alignment issues
Peter Eisentraut writes:
> On 06.07.22 04:21, Thomas Munro wrote:
>> /*
>> * Do not try to collapse these into one "w+" mode file. Doesn't work on
>> - * some platforms (eg, HPUX 10.20).
>> + * some platforms.
>> */
>> termin = fopen("/dev/tty", "r");
>> termout = fopen("/dev/tty", "w");
> I don't know how /dev/tty behaves in detail under stdio. I think
> removing this part of the comment might leave the impression that
> attempting to use "w+" will never work, whereas the existing comment
> appears to indicate that it was only very old platforms that had the
> issue. If we don't have an immediate answer to that, I'd leave the
> comment as is.
Yeah, I was kind of wondering whether we should give w+ a try now.
IIRC, the code was like that at one point, but we had to change it
(ie the comment comes from bitter experience). On the other hand,
it's probably not worth the trouble and risk to change it again.
regards, tom lane
Re: AIX support - alignment issues
On 06.07.22 04:21, Thomas Munro wrote:
/*
* Do not try to collapse these into one "w+" mode file. Doesn't work on
-* some platforms (eg, HPUX 10.20).
+* some platforms.
*/
termin = fopen("/dev/tty", "r");
termout = fopen("/dev/tty", "w");
I don't know how /dev/tty behaves in detail under stdio. I think
removing this part of the comment might leave the impression that
attempting to use "w+" will never work, whereas the existing comment
appears to indicate that it was only very old platforms that had the
issue. If we don't have an immediate answer to that, I'd leave the
comment as is.
Re: AIX support - alignment issues
On 2022-07-06 01:33:58 -0400, Tom Lane wrote: > Andres Freund writes: > > There's also a bunch of #ifdef __ia64__ in > > src/backend/utils/misc/guc-file.c, > > contrib/seg/segscan.c and contrib/cube/cubescan.c > > And all our other flex output files --- AFAICS that's part of flex's > recipe and not under our control. Clearly I need to stop reviewing things for the rest of the day :)
Re: AIX support - alignment issues
Andres Freund writes: > There's also a bunch of #ifdef __ia64__ in src/backend/utils/misc/guc-file.c, > contrib/seg/segscan.c and contrib/cube/cubescan.c And all our other flex output files --- AFAICS that's part of flex's recipe and not under our control. regards, tom lane
Re: AIX support - alignment issues
Hi, 0001 looks good to me. There's a leftover itanium reference in a comment in src/include/port/atomics/generic-msvc.h There's also a bunch of #ifdef __ia64__ in src/backend/utils/misc/guc-file.c, contrib/seg/segscan.c and contrib/cube/cubescan.c Otherwise lgtm as well. Greetings, Andres Freund
Re: AIX support - alignment issues
On Wed, Jul 6, 2022 at 3:26 PM Andres Freund wrote:
> On 2022-07-06 14:21:50 +1200, Thomas Munro wrote:
> > - * Notice that this means that we actually clear the word to set
> > - * the lock and set the word to clear the lock. This is the
> > - * opposite behavior from the SPARC LDSTUB instruction. For some
> > - * reason everything that H-P does is rather baroque...
> Are these comments retained elsewhere? It's confusing enough that I think we
> should make sure they're somewhere until we remove hppa support...
OK, I moved them into s_lock.h where the remaining asm lives.
> > -#if defined(__ia64__) || defined(__ia64)
> > -/*
> > - * Intel Itanium, gcc or Intel's compiler.
>
> Hm. Personally I'd do HPUX removal separately from IA64 removal.
OK, split.
From 7df6a66d9dd5f5c0c390a6271b55b58168f79924 Mon Sep 17 00:00:00 2001
From: Thomas Munro
Date: Mon, 4 Jul 2022 16:24:16 +1200
Subject: [PATCH v3 1/2] Remove HP-UX support.
HP-UX hardware is no longer produced. It is time to remove support for:
* HP-UX, the operating system.
* HP aCC, the compiler used only on HP-UX.
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
---
configure | 99 +--
configure.ac | 8 --
doc/src/sgml/dfunc.sgml| 29 ---
doc/src/sgml/installation.sgml | 3 +-
doc/src/sgml/regress.sgml | 6 +-
doc/src/sgml/runtime.sgml | 19 -
src/Makefile.shlib | 33
src/backend/libpq/ifaddr.c | 12 +--
src/backend/port/hpux/tas.c.template | 40 --
src/backend/port/tas/hpux_hppa.s | 28 ---
src/backend/tcop/postgres.c| 8 +-
src/backend/utils/misc/ps_status.c | 17
src/common/sprompt.c | 2 +-
src/include/pg_config.h.in | 6 --
src/include/port/atomics.h | 2 -
src/include/port/atomics/arch-ia64.h | 2 -
src/include/port/atomics/fallback.h| 4 +-
src/include/port/atomics/generic-acc.h | 106 -
src/include/port/hpux.h| 3 -
src/include/storage/s_lock.h | 105 +---
src/makefiles/Makefile.hpux| 47 ---
src/pl/plperl/ppport.h | 2 +-
src/port/dlopen.c | 50 +---
src/port/getrusage.c | 1 -
src/template/hpux | 34
src/test/regress/resultmap | 1 -
src/tools/msvc/Solution.pm | 2 -
src/tools/pginclude/cpluspluscheck | 1 -
src/tools/pginclude/headerscheck | 1 -
29 files changed, 54 insertions(+), 617 deletions(-)
delete mode 100644 src/backend/port/hpux/tas.c.template
delete mode 100644 src/backend/port/tas/hpux_hppa.s
delete mode 100644 src/include/port/atomics/generic-acc.h
delete mode 100644 src/include/port/hpux.h
delete mode 100644 src/makefiles/Makefile.hpux
delete mode 100644 src/template/hpux
diff --git a/configure b/configure
index fb07cd27d9..91b7b185f9 100755
--- a/configure
+++ b/configure
@@ -2994,7 +2994,6 @@ case $host_os in
darwin*) template=darwin ;;
dragonfly*) template=netbsd ;;
freebsd*) template=freebsd ;;
-hpux*) template=hpux ;;
linux*|gnu*|k*bsd*-gnu)
template=linux ;;
mingw*) template=win32 ;;
@@ -6856,100 +6855,6 @@ if test x"$pgac_cv_prog_CXX_cxxflags__qlonglong" = x"yes"; then
fi
-elif test "$PORTNAME" = "hpux"; then
- # On some versions of HP-UX, libm functions do not set errno by default.
- # Fix that by using +Olibmerrno if the compiler recognizes it.
-
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CC} supports +Olibmerrno, for CFLAGS" >&5
-$as_echo_n "checking whether ${CC} supports +Olibmerrno, for CFLAGS... " >&6; }
-if ${pgac_cv_prog_CC_cflags_pOlibmerrno+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- pgac_save_CFLAGS=$CFLAGS
-pgac_save_CC=$CC
-CC=${CC}
-CFLAGS="${CFLAGS} +Olibmerrno"
-ac_save_c_werror_flag=$ac_c_werror_flag
-ac_c_werror_flag=yes
-cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-
-int
-main ()
-{
-
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_prog_CC_cflags_pOlibmerrno=yes
-else
- pgac_cv_prog_CC_cflags_pOlibmerrno=no
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-ac_c_werror_flag=$ac_save_c_werror_flag
-CFLAGS="$pgac_save_CFLAGS"
-CC="$pgac_save_CC"
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_prog_CC_cflags_pOlibmerrno" >&5
-$as_echo "$pgac_cv_prog_CC_cflags_pOlibmerrno" >&6; }
-if test x"$pgac_cv_prog_CC_cflags_pOlibmerrno" = x"yes"; then
- CFLAGS="${CFLAGS} +Olibmerrno"
-fi
-
-
- { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CXX} supports +Olibmerrno, for CXXFLAGS" >&5
-$as_echo_n "checking whether ${CXX} supports +Olibmerrno, for CXXFLAGS..
Re: AIX support - alignment issues
Thomas Munro writes: > OK, here's a new attempt, this time leaving the hppa bits in. The > main tricksy bit is where s_lock.h is simplified a bit by moving the > fully inline GCC-only hppa support up a bit (it was handled a bit > weirdly with some #undef jiggery-pokery before to share stuff between > aCC and GCC), making the diff a little hard to follow. Does this make > sense? It might also be possible to drop one of __hppa and __hppa__ > where they are both tested (not clear to me if that is an aCC/GCC > thing). I have no idea if this'll actually work (or ever worked) on > NetBSD/hppa... if it comes to it I could try to boot it under > qemu-system-hppa if that's what it takes, but it may be easy for you > to test... Our HEAD does work on that NetBSD installation. I can try this patch, but it'll take an hour or two to get results ... stay tuned. I'm not sure about the __hppa vs __hppa__ thing. If we're assuming that NetBSD is the only remaining hppa platform of interest, then clearly only one of those is needed, but I don't know which one should be preferred. It appears that both are defined on NetBSD. (FWIW, I know that OpenBSD works on this machine too, or did the last time I tried it. But it probably has the same opinions as NetBSD about predefined macros.) regards, tom lane
Re: AIX support - alignment issues
Hi,
On 2022-07-06 14:21:50 +1200, Thomas Munro wrote:
> --- a/src/backend/port/hpux/tas.c.template
> +++ /dev/null
> @@ -1,40 +0,0 @@
> -/*
> - * tas() for HPPA.
> - *
> - * To generate tas.s using this template:
> - * 1. cc +O2 -S -c tas.c
> - * 2. edit tas.s:
> - * - replace the LDW with LDCWX
> - * 3. install as src/backend/port/tas/hpux_hppa.s.
> - *
> - * For details about the LDCWX instruction, see the "Precision
> - * Architecture and Instruction Reference Manual" (09740-90014 of June
> - * 1987), p. 5-38.
> - */
> -
> -int
> -tas(lock)
> -int *lock; /* LDCWX is a word instruction */
> -{
> -/*
> - * LDCWX requires that we align the "semaphore" to a 16-byte
> - * boundary. The actual datum is a single word (4 bytes).
> - */
> -lock = ((uintptr_t) lock + 15) & ~15;
> -
> -/*
> - * The LDCWX instruction atomically clears the target word and
> - * returns the previous value. Hence, if the instruction returns
> - * 0, someone else has already acquired the lock before we tested
> - * it (i.e., we have failed).
> - *
> - * Notice that this means that we actually clear the word to set
> - * the lock and set the word to clear the lock. This is the
> - * opposite behavior from the SPARC LDSTUB instruction. For some
> - * reason everything that H-P does is rather baroque...
> - */
> -if (*lock) { /* this generates the LDW */
> - return(0); /* success */
> -}
> -return(1); /* failure */
> -}
Are these comments retained elsewhere? It's confusing enough that I think we
should make sure they're somewhere until we remove hppa support...
> -#if defined(__ia64__) || defined(__ia64)
> -/*
> - * Intel Itanium, gcc or Intel's compiler.
Hm. Personally I'd do HPUX removal separately from IA64 removal.
Greetings,
Andres Freund
Re: AIX support - alignment issues
On Tue, Jul 5, 2022 at 4:53 PM Tom Lane wrote:
> Thomas Munro writes:
> > On Mon, Jul 4, 2022 at 12:08 PM Tom Lane wrote:
> >> I would not stand in the way of dropping HP-UX and IA64 support as of
> >> v16. (I do still feel that HPPA is of interest, to keep us honest
> >> about spinlock support --- but that dual-stack arrangement that IA64
> >> uses is surely not part of anyone's future.)
>
> > I tried to find everything relating to HP-UX, aCC, ia64 and hppa. Or
> > do you still want to keep the hppa bits for NetBSD (I wasn't sure if
> > your threat to set up a NetBSD/hppa system was affected by the
> > hardware failure you mentioned)?
>
> No, the hardware failure is that the machine's SCSI controller seems
> to be fried, thus internal drives no longer accessible. I have a
> working NetBSD-current installation on an external USB drive, and plan
> to commission it as a buildfarm animal once NetBSD 10 is officially
> branched. It'll be a frankencritter of the first order, because
> USB didn't exist when the machine was built, but hey...
OK, here's a new attempt, this time leaving the hppa bits in. The
main tricksy bit is where s_lock.h is simplified a bit by moving the
fully inline GCC-only hppa support up a bit (it was handled a bit
weirdly with some #undef jiggery-pokery before to share stuff between
aCC and GCC), making the diff a little hard to follow. Does this make
sense? It might also be possible to drop one of __hppa and __hppa__
where they are both tested (not clear to me if that is an aCC/GCC
thing). I have no idea if this'll actually work (or ever worked) on
NetBSD/hppa... if it comes to it I could try to boot it under
qemu-system-hppa if that's what it takes, but it may be easy for you
to test...
From 58e3974b15ce5a6325630252f5306e77528d2b1f Mon Sep 17 00:00:00 2001
From: Thomas Munro
Date: Mon, 4 Jul 2022 16:24:16 +1200
Subject: [PATCH v2] Remove HP-UX, aCC and ia64 support.
HP-UX hardware is no longer produced. This removes support for:
* HP-UX, the operating system.
* HP aCC, the compiler.
* HP/Intel Itanium (ia64), the CPU architecture discontinued in 2021.
Support for HP PA-RISC (hppa), the CPU architecture discontinued in
2008, remains in the tree for now to support potential use on NetBSD.
---
configure | 99 +---
configure.ac | 8 --
doc/src/sgml/dfunc.sgml| 29 -
doc/src/sgml/installation.sgml | 3 +-
doc/src/sgml/regress.sgml | 6 +-
doc/src/sgml/runtime.sgml | 19 ---
src/Makefile.shlib | 33 --
src/backend/libpq/ifaddr.c | 12 +-
src/backend/port/hpux/tas.c.template | 40 ---
src/backend/port/tas/hpux_hppa.s | 28 -
src/backend/tcop/postgres.c| 74
src/backend/utils/misc/ps_status.c | 17 ---
src/common/sprompt.c | 2 +-
src/include/miscadmin.h| 8 --
src/include/pg_config.h.in | 6 -
src/include/port/atomics.h | 4 -
src/include/port/atomics/arch-ia64.h | 29 -
src/include/port/atomics/fallback.h| 4 +-
src/include/port/atomics/generic-acc.h | 106 -
src/include/port/hpux.h| 3 -
src/include/storage/s_lock.h | 155 -
src/makefiles/Makefile.hpux| 47
src/port/dlopen.c | 50 +---
src/port/getrusage.c | 1 -
src/template/hpux | 34 --
src/test/regress/resultmap | 1 -
src/tools/msvc/Solution.pm | 2 -
src/tools/pginclude/cpluspluscheck | 2 -
src/tools/pginclude/headerscheck | 2 -
29 files changed, 36 insertions(+), 788 deletions(-)
delete mode 100644 src/backend/port/hpux/tas.c.template
delete mode 100644 src/backend/port/tas/hpux_hppa.s
delete mode 100644 src/include/port/atomics/arch-ia64.h
delete mode 100644 src/include/port/atomics/generic-acc.h
delete mode 100644 src/include/port/hpux.h
delete mode 100644 src/makefiles/Makefile.hpux
delete mode 100644 src/template/hpux
diff --git a/configure b/configure
index fb07cd27d9..91b7b185f9 100755
--- a/configure
+++ b/configure
@@ -2994,7 +2994,6 @@ case $host_os in
darwin*) template=darwin ;;
dragonfly*) template=netbsd ;;
freebsd*) template=freebsd ;;
-hpux*) template=hpux ;;
linux*|gnu*|k*bsd*-gnu)
template=linux ;;
mingw*) template=win32 ;;
@@ -6856,100 +6855,6 @@ if test x"$pgac_cv_prog_CXX_cxxflags__qlonglong" = x"yes"; then
fi
-elif test "$PORTNAME" = "hpux"; then
- # On some versions of HP-UX, libm functions do not set errno by default.
- # Fix that by using +Olibmerrno if the compiler recognizes it.
-
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CC} supports +Olibmerrno, for CFLAGS" >&5
-$as_echo_n "checking whether ${CC} supports +Olibmerrno, for CFLAGS... "
Re: AIX support - alignment issues
Hi, On 2022-07-05 08:13:21 +0200, Peter Eisentraut wrote: > On 05.07.22 07:31, Andres Freund wrote: > > On 2022-07-02 11:33:54 -0700, Andres Freund wrote: > > > If we decide we want to continue supporting AIX we should bite the bullet > > > and > > > add a 64bit-int TYPALIGN_*. It might be worth to translate that to bytes > > > when > > > building tupledescs, so we don't need more branches (reducing them > > > compared to > > > today). > > > > I just thought an easier way - why don't we introduce a 'catalog_double' > > that's defined to be pg_attribute_aligned(whatever-we-need) on AIX? Then we > > can get rid of the manually enforced alignedness and we don't need to > > contort > > catalog order. > > Isn't the problem that on AIX, double and int64 have different alignment > requirements, and we just check the one for double and apply it to int64? > That ought to be fixable by two separate alignment checks in configure and a > new alignment letter for pg_type. Except that that's quite a bit of work to get right, particularly without regressing the performance on all platforms. The attalign switches during tuple deforming are already quite hot. Greetings, Andres Freund
