[issue18235] _sysconfigdata.py wrong on AIX installations

[Eric N. Vander Weele]

Eric N. Vander Weele added the comment:

> Hmm, the patch seems to be incomplete, as it just removes all the AIX support 
> scripts and mentions, without adding anything new to accommodate for the 
> removal.

The new changes to accommodate for the script removal are in 
https://github.com/ericvw/cpython/commit/e889f5fd04c1b73ef06e9f6e60108b2a7718b7d6#diff-e2d5a00791bce9a01f99bc6fd613a39dR9159
 and 
https://github.com/ericvw/cpython/commit/e889f5fd04c1b73ef06e9f6e60108b2a7718b7d6#diff-67e997bcfdac55191033d57a16d1408aR2403.

> Please note that building with both xlc_r and gcc needs to be supported.

I'll need to double check gcc, I have it working with xlc_r already.

> Please open a new issue for this to discuss this change there.

Absolutely, I'll clean up the patch and provide more details for the changes in 
another issue.  Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 24.01.2017 16:47, Eric N. Vander Weele wrote:
> 
> I have a (large) patch that completely eliminates the need for ld_so_aix and 
> makeexp_aix.  I've applied and been using this with for Python 2.7 and 3.5+, 
> but I still need to go back and validate the tests to ensure everything 
> passes as expected.
> 
> It's still a work in progress, but the patch can be found at 
> https://github.com/ericvw/cpython/commit/e889f5fd04c1b73ef06e9f6e60108b2a7718b7d6.

Hmm, the patch seems to be incomplete, as it just removes
all the AIX support scripts and mentions, without adding
anything new to accommodate for the removal.

Please note that building with both xlc_r and gcc needs to
be supported.

> If this is out of context for this particular issue, I'm happy to raise a new 
> issue to have a focus discussion about potentially landing this.  Ignore 
> changes for README.AIX since I am addressing that in issue28845.

Please open a new issue for this to discuss this change there.

> I believe if we can eliminate the wrapper scripts, it will simplify the build 
> and linking for AIX for the interpreter and C-extension modules.

Sure, if that's possible. The scripts are rather old and AIX
has moved on since those days.

FWIW: They still do work without any problems, provided the
Makefile and distutils can find them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Eric N. Vander Weele]

Eric N. Vander Weele added the comment:

I have a (large) patch that completely eliminates the need for ld_so_aix and 
makeexp_aix.  I've applied and been using this with for Python 2.7 and 3.5+, 
but I still need to go back and validate the tests to ensure everything passes 
as expected.

It's still a work in progress, but the patch can be found at 
https://github.com/ericvw/cpython/commit/e889f5fd04c1b73ef06e9f6e60108b2a7718b7d6.

If this is out of context for this particular issue, I'm happy to raise a new 
issue to have a focus discussion about potentially landing this.  Ignore 
changes for README.AIX since I am addressing that in issue28845.

I believe if we can eliminate the wrapper scripts, it will simplify the build 
and linking for AIX for the interpreter and C-extension modules.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

OK - so details on the failing tests - for the record only.

However, I would appreciate some 'expert' feedback on what is expected to be 
happening in this test:

==
FAIL: test_sysconfig_compiler_vars 
(distutils.tests.test_sysconfig.SysconfigTestCase)
--
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_sysconfig.py", 
line 102, in test_sysconfig_compiler_vars
self.assertEqual(global_sysconfig.get_config_var('LDSHARED'), 
sysconfig.get_config_var('LDSHARED'))
AssertionError: 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp' != 
'/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp'

--

BULK details...

==
ERROR: test_record_extensions (distutils.tests.test_install.InstallTestCase)
--
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_install.py", 
line 216, in test_record_extensions
cmd.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/install.py", line 
563, in run
self.run_command('build')
  File "/data/prj/python/Python-2.7.13/Lib/distutils/cmd.py", line 326, in 
run_command
self.distribution.run_command(command)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/dist.py", line 972, in 
run_command
cmd_obj.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build.py", line 
127, in run
self.run_command(cmd_name)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/cmd.py", line 326, in 
run_command
self.distribution.run_command(command)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/dist.py", line 972, in 
run_command
cmd_obj.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", 
line 340, in run
self.build_extensions()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", 
line 449, in build_extensions
self.build_extension(ext)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", 
line 531, in build_extension
target_lang=language)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/ccompiler.py", line 691, 
in link_shared_object
extra_preargs, extra_postargs, build_temp, target_lang)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/unixccompiler.py", line 
202, in link
raise LinkError, msg
LinkError: command '/opt/lib/python2.7/config/ld_so_aix' failed with exit 
status 1

==
ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase)
--
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_config_cmd.py", 
line 46, in test_search_cpp
match = cmd.search_cpp(pattern='xxx', body='/* xxx */')
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/config.py", line 
206, in search_cpp
src, out = self._preprocess(body, headers, include_dirs, lang)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/config.py", line 
129, in _preprocess
self.compiler.preprocess(src, out, include_dirs=include_dirs)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/unixccompiler.py", line 
113, in preprocess
raise CompileError, msg
CompileError: command 'xlc_r' failed with exit status 40

==
ERROR: test_build_ext (distutils.tests.test_build_ext.BuildExtTestCase)
--
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_build_ext.py", 
line 63, in test_build_ext
cmd.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", 
line 340, in run
self.build_extensions()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", 
line 449, in build_extensions
self.build_extension(ext)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", 
line 531, in build_extension
target_lang=language)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/ccompiler.py", line 691, 
in link_shared_object
extra_preargs, extra_postargs, build_temp, target_lang)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/unixccompiler.py", line 
202, in link
raise LinkError, msg
LinkError: command '/opt/lib/python2.7/config/ld_so_aix' failed with exit 
status 1

==
ERROR: test_get_outputs (distutils.tests.test_build_ext.BuildExtTestCase)

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

I am back at this: (note: short answer to msg277745 - No. does not work for me)

As ... expressed before, this needs to work in three contexts:

2) In-tree testing of build module feature (test_distutils).

Below are results of 'in tree' and 'out of tree' testing. in/out seems to have 
no (more) impact.

msg219888 also has a patch - that moves the code back to where it originally 
was: _init_posix

But - same issue - ld_so_aix is not found. (notice the "src" in string of 
'threading.py')
[112/401/2] test_distutils
xlc_r: 1501-218 (S) file _configtest.i contains an incorrect file suffix
unable to execute '/opt/lib/python2.7/config/ld_so_aix': No such file or 
directory
Warning -- os.environ was modified by test_distutils
test test_distutils failed -- multiple errors occurred; run in verbose mode for 
details
[113/401/3] test_dl
test_dl skipped -- No module named dl
[114/401/3] test_docxmlrpc
/data/prj/python/src/Python-2.7.13/Lib/threading.py:846: DeprecationWarning: 
sys.exc_clear() not supported in 3.x; use except clauses
  self.__exc_clear()
[115/401/3] test_dumbdbm
root@x064:[/data/prj/python/Python-2.7.13]


Now, back to 'in tree' with patch from msg219888 (no "src" in path)

root@x064:[/data/prj/python/Python-2.7.13]find . -name _sysconfigdata\*.py -ls 
-exec grep LDSHARED {} \;
48056876   16 -rw-r-  1 root  100015362 Jan 24 15:07 
./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp',
[112/401/2] test_distutils
xlc_r: 1501-218 (S) file _configtest.i contains an incorrect file suffix
unable to execute '/opt/lib/python2.7/config/ld_so_aix': No such file or 
directory
Warning -- os.environ was modified by test_distutils
test test_distutils failed -- multiple errors occurred; run in verbose mode for 
details
[113/401/3] test_dl
test_dl skipped -- No module named dl
[114/401/3] test_docxmlrpc
/data/prj/python/Python-2.7.13/Lib/threading.py:846: DeprecationWarning: 
sys.exc_clear() not supported in 3.x; use except clauses
  self.__exc_clear()

Conclusion:
** As the code is - that is removed means the environment variable in 
sysconfigdata.py is correct,
   I would suggest the patch is to remove the 6 lines there.
   Further, the addition of the code below (per msg219888) has no effect, 
including it is not needed.

The .dist file is from the source tarball from last december for 2.7.13

michael@x071:[/data/prj/python/src]diff -u  Python-2.7.13/Lib/sysconfig.py.dist>
--- Python-2.7.13/Lib/sysconfig.py.dist 2016-12-17 20:05:06 +
+++ Python-2.7.13/Lib/sysconfig.py  2017-01-24 14:56:20 +
@@ -307,12 +307,6 @@
 msg = msg + " (%s)" % e.strerror
 raise IOError(msg)

-# On AIX, there are wrong paths to the linker scripts in the Makefile
-# -- these paths are relative to the Python source, but when installed
-# the scripts are in another directory.
-if _PYTHON_BUILD:
-vars['LDSHARED'] = vars['BLDSHARED']
-
 # There's a chicken-and-egg situation on OS X with regards to the
 # _sysconfigdata module after the changes introduced by #15298:
 # get_config_vars() is called by get_platform() as part of the
@@ -355,6 +349,11 @@
 # _sysconfigdata is generated at build time, see _generate_posix_vars()
 from _sysconfigdata import build_time_vars
 vars.update(build_time_vars)
+# On AIX, there are wrong paths to the linker scripts in the Makefile
+# -- these paths are relative to the Python source, but when installed
+# the scripts are in another directory.
+if _PYTHON_BUILD:
+vars['LDSHARED'] = vars['BLDSHARED']

 def _init_non_posix(vars):
 """Initialize the module as appropriate for NT"""

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 24.01.2017 15:23, Michael Felt wrote:
> As far as issue10656 and this issue are concerned:
> 
> Python-3.4 is out of context (but 3.4.6 was just released) - and does not 
> work with 'out of tree' builds. 
> 
> The other versions: 2.7.13, 3.5.3 and 3.6.0 do build out-of-tree.
> Note 3.5.3 and 3.6.0 use a different name for sysconfigdata.py

I'm not sure what you mean with "out of tree". When building
Python, you are supposed to use the source code directory
as work dir.

> Notes:
> 
> FYI: regarding build and src in different directories
> 
> Not working...
> 
> unable to execute '../src/Python-3.4.6/Modules/ld_so_aix': No such file or 
> directory
> unable to execute '../src/Python-3.4.6/Modules/ld_so_aix': No such file or 
> directory
>  many many times
> 
> Concludes with:
> ImportError: No module named 'time'
> make: 1254-004 The error code from the last command is 1.
> make: 1254-005 Ignored error code 1 from last command.
> ../src/Python-3.4.6/install-sh: ../src/Python-3.4.6/Modules/ld_so_aix does 
> not exist
> make: 1254-004 The error code from the last command is 1.
> 
> root@x064:[/data/prj/python/Python-3.4.6]find . -name _sysconfigdata.py -ls 
> -exec grep LDSHARED {} \;
> 48049316   19 -rw-r-  1 root  felt 19209 Jan 24 13:55 
> ./build/lib.aix-5.3-3.4/_sysconfigdata.py
>  'BLDSHARED': '/opt/lib/python3.4/config/ld_so_aix xlc_r '
>  'LDSHARED': '/opt/lib/python3.4/config/ld_so_aix xlc_r '

This is strange: where does the "../src/Python-3.4.6/Modules/ld_so_aix"
originate if the paths in the sysconfig file are absolute ?

> +++ Working 
> 
> Working: 2.7.13 -- Note: the 'lines' in sysconfig.py are removed:
> root@x064:[/data/prj/python/Python-2.7.13.0]find . -name _sysconfigdata\*.py 
> -ls -exec grep LDSHARED {} \;
> 48038426   16 -rw-r-  1 root  felt 15807 Jan 24 13:36 
> ./build/lib.aix-5.3-2.7/_sysconfigdata.py
>  'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
>  'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r 
> -bI:/opt/lib/python2.7/config/python.exp',

BLDSHARED should be fixed to use an absolute path, I guess.

> Working: 3.5.3
> 48049196   21 -rw-r-  1 root  felt 20640 Jan 24 13:55 
> ./build/lib.aix-5.3-3.5/_sysconfigdata.py
>  'BLDSHARED': '/opt/lib/python3.5/config-3.5m/ld_so_aix xlc_r '
>  'LDSHARED': '/opt/lib/python3.5/config-3.5m/ld_so_aix xlc_r '

Here BLDSHARED is wrong, but shouldn't do any harm since
it's only used for building Python itself, not for
extension modules.

> Working: 3.6.0 -- Note new file name!!
> root@x064:[/data/prj/python/Python-3.6.0]find . -name _sysconfigdata\*.py -ls 
> -exec grep LDSHARED {} \;
> 47794146   20 -rw-r-  1 root  felt 20394 Jan 24 13:49 
> ./build/lib.aix-5.3-3.6/_sysconfigdata_m_aix5_.py
>  'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
>  'LDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '

This looks odd and like a regression. Again, the path
should be absolute and point to the build tree, not the
installation tree.

PS: The BLDSHARED setting in sysconfig is not really relevant.
It's only there because we have it in the Makefile.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

As far as issue10656 and this issue are concerned:

Python-3.4 is out of context (but 3.4.6 was just released) - and does not work 
with 'out of tree' builds. 

The other versions: 2.7.13, 3.5.3 and 3.6.0 do build out-of-tree.
Note 3.5.3 and 3.6.0 use a different name for sysconfigdata.py

Notes:

FYI: regarding build and src in different directories

Not working...

unable to execute '../src/Python-3.4.6/Modules/ld_so_aix': No such file or 
directory
unable to execute '../src/Python-3.4.6/Modules/ld_so_aix': No such file or 
directory
 many many times

Concludes with:
ImportError: No module named 'time'
make: 1254-004 The error code from the last command is 1.
make: 1254-005 Ignored error code 1 from last command.
../src/Python-3.4.6/install-sh: ../src/Python-3.4.6/Modules/ld_so_aix does not 
exist
make: 1254-004 The error code from the last command is 1.

root@x064:[/data/prj/python/Python-3.4.6]find . -name _sysconfigdata.py -ls 
-exec grep LDSHARED {} \;
48049316   19 -rw-r-  1 root  felt 19209 Jan 24 13:55 
./build/lib.aix-5.3-3.4/_sysconfigdata.py
 'BLDSHARED': '/opt/lib/python3.4/config/ld_so_aix xlc_r '
 'LDSHARED': '/opt/lib/python3.4/config/ld_so_aix xlc_r '

+++ Working 

Working: 2.7.13 -- Note: the 'lines' in sysconfig.py are removed:
root@x064:[/data/prj/python/Python-2.7.13.0]find . -name _sysconfigdata\*.py 
-ls -exec grep LDSHARED {} \;
48038426   16 -rw-r-  1 root  felt 15807 Jan 24 13:36 
./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp',

Working: 3.5.3
48049196   21 -rw-r-  1 root  felt 20640 Jan 24 13:55 
./build/lib.aix-5.3-3.5/_sysconfigdata.py
 'BLDSHARED': '/opt/lib/python3.5/config-3.5m/ld_so_aix xlc_r '
 'LDSHARED': '/opt/lib/python3.5/config-3.5m/ld_so_aix xlc_r '

Working: 3.6.0 -- Note new file name!!
root@x064:[/data/prj/python/Python-3.6.0]find . -name _sysconfigdata\*.py -ls 
-exec grep LDSHARED {} \;
47794146   20 -rw-r-  1 root  felt 20394 Jan 24 13:49 
./build/lib.aix-5.3-3.6/_sysconfigdata_m_aix5_.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

re: msg286115, 

1) Building modules in the tree during the build process.
** if "in the tree" means building outside of source - such as:
../src/python-X.Y.Z/configure ...
make

** the tarball for Python-2.7.13 is building from 
../src/Python-2.7.13/configure where as ../src/Python-2.7.12/configure and the 
earlier (ones I tried) were not.

** However, 'make distclean' is not working. 
   Further details in a later issue - this bit about make distclean id merely 
FYI. 

2) In-tree testing of build module feature (test_distutils).

** Is this as simple as (from the 'build directory':
make test 

** shall check this later, make test in general seems to be working - is 
test_distutils a specific test you want verified?
3) Building and installing modules with an installed version of Python.
** FYI: this will work with the 5 lines deleted - as I mentioned above.
   This is how I manually patched _sysconfigdata.py in Python-2.7.12 when I ran 
into this.

Now reading the other comments from last night...

re: 286172 and 286173

Short response:
it looks as if patches elsewhere in the 'auto' config files has taken care of 
this issue - at least for Python-2.7

I shall download and check the latest Python-3.{4|5|6} and report back.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

Sorry I meant Issue 10656. I recently committed a fix regarding out-of-tree 
builds and ld_so_aix, and it sounds like you were thinking of reverting that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

I don’t run AIX, but my understanding is there are three distinct branches 
(2.7, 3.5, and 3.6+). Some of the following is guessed, so please correct me if 
I am wrong:

Python 2.7:
_sysconfigdata.py incorrectly created with LDSHARED = Modules/ld_so_aix.
Result: Installed distutils fails (unless you’re in the build tree).
Fix: Apply David’s issue18235.patch + Michael H’s msg219888 fix.

Python 3.5:
David’s issue18235.patch already applied.
_sysconfigdata.py incorrectly created with BLDSHARED = $(LIBPL)/ld_so_aix, i.e. 
the installed location.
Distutils does not refer to _sysconfigdata.py.
Result: Distutils unaffected, but running from the build tree, 
sysconfig.get_config_var("LDSHARED") would incorrectly use the installed 
location.
Fix: Apply Michael H’s msg219888.

Python 3.6+:
Distutils does use _sysconfigdata*.py, thanks to revision 3fa8aebed636.
Result: Distutils shouldn’t work unless Python is already installed. Since the 
3.6 and 3.x buildbots pass test_distutils, does that mean that both those 
versions of Python are already installed?
Fix: Should also be fixed by merging msg219888 from 3.5.

Marc-Andre, regarding configure.ac and absolute vs relative paths, I suggest 
you open a separate bug. That is an orthogonal issue, and this bug is already 
too complicated. But also keep in mind Issue 18235. You want the build tree, 
not the source tree.

Also, if you mean to drop the assignment in sysconfig, I think that will break 
in-tree usage of sysconfig (3.5+) and distutils (3.6+), unless Python happens 
to also be installed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

BTW: Does the ticket still apply to 3.5+ ? From reading the ticket, it seems 
that the problem is already fixed for those versions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Hmm, looking at the patch again:

diff -r a089a8b1f93d Lib/sysconfig.py
--- a/Lib/sysconfig.py  Fri Jun 21 18:37:02 2013 -0400
+++ b/Lib/sysconfig.py  Fri Jun 21 22:33:15 2013 -0700
@@ -368,7 +368,7 @@
 # -- these paths are relative to the Python source, but when installed
 # the scripts are in another directory.
 if _PYTHON_BUILD:
-vars['LDSHARED'] = vars['BLDSHARED']
+vars['BLDSHARED'] = vars['LDSHARED']
 
 # There's a chicken-and-egg situation on OS X with regards to the
 # _sysconfigdata module after the changes introduced by #15298:

I think that with the configure fix, the special case for AIX can be dropped 
altogether. It just polishes over the bug in configure, turning a relative path 
into an absolute one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Reading the ticket, it seems that there is some confusion about what LDSHARED 
and BLDSHARED are used for. BLDSHARED is used to override the LDSHARED value 
when building libpython and the shared modules (via setup.py).

LDSHARED is what is meant to be used for building shared modules after 
installation.

This distinction is not being followed by all targets in the Makefile, but is 
needed in cases where special build tools are necessary, as is the case on AIX.

For the latter, BLDSHARED has to point to the source tree version of those 
build tools, while LDSHARED has to point to the installed version of these.

On AIX, BLDSHARED should therefore point to the ./Modules/ld_so_aix (or better: 
the absolute dir of the source tree), while LDSHARED needs to point to the 
installation target for ld_so_aix, e.g. 
/usr/local/lib/python2.7/config/ld_so_aix.

So the fix which was already applied to 3.4 is correct for 2.7 as well. The 
missing part is the fix to the configure script, since in Python 2.7, this 
still uses relative paths:

BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) 
-bI:\$(BINLIBDEST)/config/python.exp"

instead of the ones from 3.4:

BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) 
-bI:\$(srcdir)/Modules/python.exp"
LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) 
-bI:\$(BINLIBDEST)/config/python.exp"

--
nosy: +lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

You guys are the experts. I can only comment on what I see. IMHO: the file 
_sysconfigdata.py is more accurate with nothing in it.

I am clearly confused by whatever process this is. If you believe it is more 
accurate to have the BLD variable 'inaccurate' in this file - and that we just 
need to "believe" that it has the correct value during the build (which, 
starting with version 2.7.13 seems to be the case, as I am able to build in a 
different directory - and could not do this with version 2.7.10 (first time I 
tried using separate directories).

In short, I have no expert opinion on this. My naive view is that both 
variables being accurate in the file is more accurate. In any case, the patch 
is needed for a working _sysconfigdata.py and integration with something such 
as "pip build ...". Only after I learned about 'pip' did I ever run into this, 
i.e., only people who are trying to install something extra have issues here. I 
have been 'patching' _sysconfigdata.py manually so that things work normally.

I hope my feedback helps you move forward on this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

I completely agree with Martin's concern.  As I expressed before, this needs to 
work in three contexts:

1) Building modules in the tree during the build process.
2) In-tree testing of build module feature (test_distutils).
3) Building and installing modules with an installed version of Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

If the assignment is completely removed, won’t this break the test when run 
from the source or build tree (as opposed to when installed)? Or at least make 
the situation worse: the AIX buildbot is already failing test_distutils, but at 
least it is looking for Modules/ld_so_aix locally rather than in the 
yet-to-be-installed location.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

This is "only" for Python-2.7 (for now). The others will be tested as I am able.

Working with the patch submitted 2013-10-19 (aka 
https://bugs.python.org/file32229/issue18235.patch)

A) Without/before the patch:

root@x064:[/data/prj/python/Python-2.7.13.0]grep LDSHARED 
./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',

B) Apply the patch:
diff -r 0a26ef834a49 Lib/sysconfig.py
--- a/Lib/sysconfig.py  Sat Oct 19 14:24:44 2013 +0200
+++ b/Lib/sysconfig.py  Sat Oct 19 11:46:10 2013 -0700
@@ -368,7 +368,7 @@
 # -- these paths are relative to the Python source, but when installed
 # the scripts are in another directory.
 if _PYTHON_BUILD:
-vars['LDSHARED'] = vars['BLDSHARED']
+vars['BLDSHARED'] = vars['LDSHARED']

After the patch:

root@x064:[/data/prj/python/Python-2.7.13.0]grep LDSHARED 
./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp',

Which makes me think - maybe NOTHING is needed - as 'BLDSHARED' should be 
Modules/ld_so_aix,
while  'LDSHARED' should be '/opt/lib/python2.7/config/ld_so_aix

Rather than actual deletes: commented out to (after patch was applied)

C: if: statement and assignment - commented out
diff -ru Python-2.7.13/Lib/sysconfig.py Python-2.7.13.0/Lib/sysconfig.py
--- Python-2.7.13/Lib/sysconfig.py  2016-12-17 20:05:06 +
+++ Python-2.7.13.0/Lib/sysconfig.py2017-01-23 14:39:31 +
@@ -310,8 +310,8 @@
 # On AIX, there are wrong paths to the linker scripts in the Makefile
 # -- these paths are relative to the Python source, but when installed
 # the scripts are in another directory.
-if _PYTHON_BUILD:
-vars['LDSHARED'] = vars['BLDSHARED']
+#if _PYTHON_BUILD:
+#vars['BLDSHARED'] = vars['LDSHARED']

root@x064:[/data/prj/python/Python-2.7.13.0]grep LDSHARED 
./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp',
root@x064:[/data/prj/python/Python-2.7.13.0]

So, perhaps the best patch 'today', versus what may have been best in 2013 - is 
to remove the 5 lines starting with # On AIX, ...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

Ok. I shall rebuild from scratch.

When I do, I start from a "clean" system - only the compiler and my mkinstallp 
helper script.

I am adding the latest zlib for what I would package but I shall forgoe that 
for this test.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

Michael, you need to build from scratch. The values are set and tweaked in 
various phases of configure and then written to _sysconfigdata.py for 
installation.

The values in the file reflect the values used during the build, but many of 
them are not used in an installed version of Python.

Three important phases are:

1) Building modules in the tree during the build process.
2) In-tree testing of build module feature.
3) Building and installing modules with an installed version of Python.

The initial configuration scripts must match the location where the export 
files will be installed. And the _sysconfigdata.py definitions used to build 
external modules in an installed version of Python must refer to the proper 
location.

All of the pieces are interconnected and must be tested in a wholistic manner. 
A partial rebuild does not test the impact of the patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

I did not go through a whole build process.

Changing:


if _PYTHON_BUILD:
vars['BLDSHARED'] = vars['LDSHARED']
to

if _PYTHON_BUILD:
vars['LDSHARED'] = vars['BLDSHARED']

is not going to help if both variables are wrong in _sysconfigdata*.py

Also,

The patch is not applied in any version - yet the values in _sysconfigdata*.py 
are different, i.e., version dependent (without the patch).

If I errored in my assumption that the file being patched is reading 
_sysconfigdata*.py then I will need to patch and build from scratch to see the 
effect on _sysconfigdata.py

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

So are you saying you tried patching Python 2 and/or 3, but did not see any 
relevant change at all?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

The "key" bit of the patch is to move the code AND make this assignment:

if _PYTHON_BUILD:
vars['LDSHARED'] = vars['BLDSHARED']

Besides, imho, skipping around the problem that LDSHARED is not correctly 
assigned - it cannot work because BLDSHARED is not set to the 'installed' value.

And, frankly I am amazed (in Python3-3.6.0) both are equal AND both pointing at 
the 'installed' location.

OOPS: I just looked and the patch above is NOT in the Python3-3.6 branch, and 
as both variables are identical now in _sysconfigdata_m_aix5.py - neither 
assignment has any affect.

Again: comparing python2 and python3 it seems the latest Python3 now have the 
fullpath - and equal definitions - for both variables WHILE Python2 use 
'relative' pathnames - BUT both are equal.

As far as the patch goes - it is irrevalant and can be removed in Python3 while 
a quick hack for Python2 would be to use (in the new location!)

if _PYTHON_BUILD:
vars['LDSHARED'] = vars['LDCXXSHARED']

Details:
root@x064:[/data/prj/python]grep SHARED 
python*/X64/opt/lib/python*/_sysconfig*.py | grep LD
python-2.7.12.0/X64/opt/lib/python2.7/_sysconfigdata.py: 'BLDSHARED': 
'./Modules/ld_so_aix xlc_r -bI:./Modules/python.exp',
python-2.7.12.0/X64/opt/lib/python2.7/_sysconfigdata.py: 'LDCXXSHARED': 
'/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp',
python-2.7.12.0/X64/opt/lib/python2.7/_sysconfigdata.py: 'LDSHARED': 
'./Modules/ld_so_aix xlc_r -bI:./Modules/python.exp',
python-2.7.13/X64/opt/lib/python2.7/_sysconfigdata.py: 'BLDSHARED': 
'Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L/opt/lib',
python-2.7.13/X64/opt/lib/python2.7/_sysconfigdata.py: 'LDCXXSHARED': 
'/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp',
python-2.7.13/X64/opt/lib/python2.7/_sysconfigdata.py: 'LDSHARED': 
'Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L/opt/lib',
python3-3.5.2/X64/opt/lib/python3.5/_sysconfigdata.py: 'BLDSHARED': 
'/opt/lib/python3.5/config/ld_so_aix xlc_r '
python3-3.5.2/X64/opt/lib/python3.5/_sysconfigdata.py: 'LDCXXSHARED': 
'/opt/lib/python3.5/config/ld_so_aix xlc_r '
python3-3.5.2/X64/opt/lib/python3.5/_sysconfigdata.py: 'LDSHARED': 
'/opt/lib/python3.5/config/ld_so_aix xlc_r '
python3-3.6.0/X64/opt/lib/python3.6/_sysconfigdata_m_aix5_.py: 'BLDSHARED': 
'/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '
python3-3.6.0/X64/opt/lib/python3.6/_sysconfigdata_m_aix5_.py: 'LDCXXSHARED': 
'/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '
python3-3.6.0/X64/opt/lib/python3.6/_sysconfigdata_m_aix5_.py: 'LDSHARED': 
'/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

OK - just looked at what happens with Python3-3.6

new file name: _sysconfigdata_m_aix5_.py

values:
root@x064:[/data/prj/python/python3-3.6.0/X64/opt/lib/python3.6]grep SHARED 
_sysconf*.py | grep LD
 'BLDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '
 'LDCXXSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '
 'LDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '


The 'build' worked, so I am guessing that BLDSHARED had a different value that 
what it shows here. 

The full pathname for ld_so_aix is correct.

As I mentioned in https://bugs.python.org/issue28311#msg277692 - still broken 
for Python2-2.7(.13) - shall try your patch above (once I figure out how to 
grab it)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Eric N. Vander Weele]

Changes by Eric N. Vander Weele :


--
nosy: +ericvw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

FYI: after manually creating the .../libpython3.5/config directory and copying 
three files (see below) I was able to "pip3 install cython" as test.

 At the packing area:
michael@x071:[/data/prj/python3/python3-3.5.2]ls Xany/opt/lib/python3.5/config
ld_so_aix   makexp_aix  python.exp

in the package:
michael@x071:[/data/prj/python3/python3-3.5.2]restore -Tqf installp/ppc/*.I | 
grep lib/python3.5/config
New volume on installp/ppc/aixtools.python3.3.5.2.0.I:
Cluster size is 51200 bytes (100 blocks).
The volume number is 1.
The backup date is: Wed Nov 16 21:48:37 UTC 2016
Files are backed up by name.
The user is root.
./opt/lib/python3.5/config
./opt/lib/python3.5/config/ld_so_aix
./opt/lib/python3.5/config/python.exp
./opt/lib/python3.5/config/makexp_aix
./opt/lib/python3.5/configparser.py
./opt/lib/python3.5/config-3.5m
./opt/lib/python3.5/config-3.5m/libpython3.5m.a
./opt/lib/python3.5/config-3.5m/Setup
./opt/lib/python3.5/config-3.5m/python-config.py
./opt/lib/python3.5/config-3.5m/config.c
./opt/lib/python3.5/config-3.5m/Setup.local
./opt/lib/python3.5/config-3.5m/config.c.in
./opt/lib/python3.5/config-3.5m/makesetup
./opt/lib/python3.5/config-3.5m/Setup.config
./opt/lib/python3.5/config-3.5m/install-sh
./opt/lib/python3.5/config-3.5m/Makefile
./opt/lib/python3.5/config-3.5m/python.o
The number of archived files is 7390.

Results:

+-+
 Installing Software...
+-+

installp:  APPLYING software for:
aixtools.python3.rte 3.5.2.0

Restoring files, please wait.
4250 files restored.
Finished processing all filesets.  (Total time:  1 mins 1 secs).

+-+
Summaries:
+-+

Installation Summary

NameLevel   PartEvent   Result
---
aixtools.python3.rte3.5.2.0 USR APPLY   SUCCESS

root@x064:[/data/prj/python3/python3-3.5.2]export OBJECT_MODE=64

root@x064:[/data/prj/python3/python3-3.5.2]pip3 install cython
Collecting cython
  Using cached Cython-0.25.1.tar.gz
Installing collected packages: cython
  Running setup.py install for cython ... done
Successfully installed cython-0.25.1
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Question: if there is a patch since 3.5.2 was released - how do I pick this up 
in order to test (and maybe patch)? I am a bit lost in the hg/git discussion 
about where to go to get current status.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

As this is the issue still open with regard to issues with ld_so_aix...

The current release Python3-3.5.2 does not include ld_so_aix in the "make 
install DESTDIR=xxx output ("make install" also neglects to install ld_so_aix)

Installation Summary

NameLevel   PartEvent   Result
---
aixtools.python3.rte3.5.2.0 USR APPLY   SUCCESS
root@x064:[/data/prj/python3/python3-3.5.2]find /opt -name ld_so_aix
root@x064:[/data/prj/python3/python3-3.5.2]grep ld_so_aix /opt/lib/py*/_sys*.py
 'BLDSHARED': '/opt/lib/python3.5/config/ld_so_aix xlc_r '
 'LDCXXSHARED': '/opt/lib/python3.5/config/ld_so_aix xlc_r '
 'LDSHARED': '/opt/lib/python3.5/config/ld_so_aix xlc_r '

So, the last two variables are correct - BLDSHARED is not correct in 
_sysconfigdata.py because the file does not exist during the build process. 
However, the build succeeds, so apparently this value for BLDSHARED is not used 
during the build.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

Regarding out-of-tree builds (Problem 3), see Issue 10656, which already has a 
potential patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

Michael F.: It sounds like you have three separate but related problems:

1. Confusion between LDSHARED and BLDSHARED referring to the in-source build 
tree vs final installed files. I think this is what David and Michael H. were 
originally trying to fix here.

2. Only BLDSHARED has an extra -L flag. Where is the code that adds it? I can’t 
find it in configure.ac in either Python 3 or 2 versions.

3. BLDSHARED also fails to work for out-of-tree builds, because it refers to 
the build tree (.) rather than the source tree. This sounds like another bug to 
me; do you want to make a patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

correction:

On 23-Oct-16 20:54, Michael Felt wrote:
> except neither BLDSHARED nor LDCXXSHARED have taken the extra LDFLAG
except neither LDSHARED nor LDCXXSHARED have taken the extra LDFLAG
like BLDSHARED has.
> that was exported before configure was called (in this case -L/opt/lib)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

The value that works for LDSHARED is the value that LDCXXSHARED has. 
BLDSHARED is also broken (currently).

  'BLDSHARED': './Modules/ld_so_aix xlc_r -bI:./Modules/python.exp 
-L/opt/lib',

  'LDCXXSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r 
-bI:/opt/lib/python2.7/config/python.exp',

except neither BLDSHARED nor LDCXXSHARED have taken the extra LDFLAG 
that was exported before configure was called (in this case -L/opt/lib)

In short: LDSHARED needs to be a full path, not relative and should 
include -L flags used during the build, if any.

LDCXXSHARED should also add LDFLAGS -L entries (and perhaps, where 
relevant -R flags).

BLDSHARED does not work when "builddir" is . and source_dir is 
../src/python-X.W.Y because Modules/ld_so_aix is not in ".", but is at 
../src/python-X.X.Y/Modules/ld_so_aix

On 15-Oct-16 00:23, Martin Panter wrote:
> Martin Panter added the comment:
>
> This is my understanding:
>
> We are talking about the code at 
>  that 
> switches the values of LDSHARED and/or BLDSHARED.
>
> Yes, Michael H. was suggesting to both move and change (revert) back to 
> overwriting LDSHARED with the value of BLDSHARED. Since he is moving the code 
> into _init_posix(), which I think only gets called at run time, the state of 
> _PYTHON_BUILD won’t affect the value of LDSHARED saved in the installed 
> config file.
>
> --
>
> ___
> Python tracker 
> 
> ___

--
nosy: +aixto...@gmail.com

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

This is my understanding:

We are talking about the code at 
 that 
switches the values of LDSHARED and/or BLDSHARED.

Yes, Michael H. was suggesting to both move and change (revert) back to 
overwriting LDSHARED with the value of BLDSHARED. Since he is moving the code 
into _init_posix(), which I think only gets called at run time, the state of 
_PYTHON_BUILD won’t affect the value of LDSHARED saved in the installed config 
file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Haubenwallner]

Michael Haubenwallner added the comment:

...a long time since I've been in this area...

David, I'm not completely sure which code fragments you're talking about for 
"revert or change".

Anyway: If I remember correctly, the confusion here is about the idea behind 
LDSHARED and BLDSHARED.
As far as I understand, this idea originally was:

LDSHARED: The "command to create shared modules". Used as variable in the 
"Makefile (and similar) templates to build python modules" for both in-python 
and third party modules. Initialized to hold the value which works for third 
party modules to link against the _installed_ python.

BLDSHARED: Read as "Buildtime-LDSHARED". Holds the value to override LDSHARED 
with while building python itself (_PYTHON_BUILD=True), which works for 
building in-python modules to link against the _builddir_ python.

So there's no point in ever setting BLDSHARED to the value of LDSHARED.

Actually there is no point in having BLDSHARED visible at all in an installed 
python, nor using its value while building third party modules - there's just 
no code yet that prunes BLDSHARED from the installed _sysconfigdata.py.

HTH,
/haubi/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

Michael Felt,

The patch was from Michael Haubenwallner.  I was addressing Michael 
Haubenwallner.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

I am not suggesting anything. I am only trying to report what I experience.

The first observation was that I could not build python when src != build 
directory. I believe that was the original complaint as well.

The second observation came when I tried to use pip install, and also make of 
the mercurial "addition". That module consistently failed until I editted, 
manually, the _sysconf*.py file to make LDwhatever the same as LDCXXwhatever.

With study I  could figure out what they stand for - I would guess they are 
related, if not derived from, to some "autotool" conventions.

To be honest, I do not understand what is meant by "flipping", so I am neither 
for nor against.

For me to test a patch - which I am very willing to do - I would be grateful 
for the mercurial command(s) to test it normally (now that I finally have it 
working - I figured out how to make the SSL part work this evening).

In short, please do not shoot me, the messenger :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

Michael,

Are you suggesting to move the code fragment *AND* revert or change the 
reversal of LDSHARED?  The Python code seems to be setting and reversing the 
value in multiple places.

This also relates to Issue25825.

Repeatedly flipping this around is not making progress.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

Reopening this to fix the original bug in 2.7, and to improve things for Python 
3.

Michael Felt (or anyone else): Can you confirm if Michael Haubenwallner’s 
suggested patch from  is 
appropriate? It looks like that patch won’t apply directly to 2.7, but the 
spirit of that patch looks sensible to me for Python 2 and 3. I.e. don’t mess 
with the variables in _generate_posix_vars(), which writes them to a data file, 
but overwrite LDSHARED in _init_posix(), which happens at run time.

--
nosy: +Michael.Felt
stage: resolved -> patch review
status: closed -> open
superseder: AIX shared library extension modules installation broken - 
Python2.7 -> 
versions: +Python 2.7, Python 3.5, Python 3.6, Python 3.7 -Python 3.3, Python 
3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

It might have been good to get this reopened to attract more attention to it. 
Anyway, Issue 28311 has now been opened for 2.7, and it seems to be about the 
same problem.

--
nosy: +martin.panter
superseder:  -> AIX shared library extension modules installation broken - 
Python2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Phil Elson]

Phil Elson added the comment:

FWIW - this doesn't appear to have been backported to v2.7.x. As a result, the 
./Modules/ld_so_aix reference still exists in _sysconfigdata.py in v2.7.8 (and 
therefore I was unable to build numpy). The workaround is easy enough, it is 
just something that is worth noting.

--
nosy: +pelson

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Haubenwallner]

Michael Haubenwallner added the comment:

Hmm... instead of reversing the order while keeping in _generate_posix_vars(), 
feels like it would have been better to move the code from 2000 back to 
_init_posix() where it originally was, without changing the order - because now 
for sysconfig within Python build, the working B-value of LDSHARED is lost.

Something like (note the function names):

--- a/Lib/sysconfig.py  Fri Jun 06 01:23:53 2014 -0500
+++ b/Lib/sysconfig.py  Fri Jun 06 17:11:02 2014 +0200
@@ -364,11 +364,6 @@ def _generate_posix_vars():
 if hasattr(e, strerror):
 msg = msg +  (%s) % e.strerror
 raise OSError(msg)
-# On AIX, there are wrong paths to the linker scripts in the Makefile
-# -- these paths are relative to the Python source, but when installed
-# the scripts are in another directory.
-if _PYTHON_BUILD:
-vars['BLDSHARED'] = vars['LDSHARED']
 
 # There's a chicken-and-egg situation on OS X with regards to the
 # _sysconfigdata module after the changes introduced by #15298:
@@ -409,6 +404,11 @@ def _init_posix(vars):
 # _sysconfigdata is generated at build time, see _generate_posix_vars()
 from _sysconfigdata import build_time_vars
 vars.update(build_time_vars)
+# On AIX, we have different paths for building the Python modules
+# relative to the Python source, and building third party modules
+# after installing the Python dist.
+if _PYTHON_BUILD:
+vars['LDSHARED'] = vars['BLDSHARED']
 
 def _init_non_posix(vars):
 Initialize the module as appropriate for NT

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Haubenwallner]

Changes by Michael Haubenwallner michael.haubenwall...@salomon.at:


Removed file: http://bugs.python.org/file32542/python-tip-aix-absbuilddir.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Haubenwallner]

Michael Haubenwallner added the comment:

issue#10656 is the out-of-source part already.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Haubenwallner]

Michael Haubenwallner added the comment:

Kindly ping. Do you prefer another report for the aix-absbuilddir patch as this 
one is already closed?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

Do you want me to open a new issue or do you want to open a new issue?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Haubenwallner]

Michael Haubenwallner added the comment:

Erm, question target was the python committers. And I'd create the new issue if 
they prefer.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Haubenwallner]

Michael Haubenwallner added the comment:

Actually, ld_so_aix is autogenerated from ld_so_aix.in into builddir, while 
makexp_aix is not.

Attached patch eventually might fix the test too?

--
nosy: +haubi
Added file: http://bugs.python.org/file32542/python-tip-aix-absbuilddir.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

+1

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

I think I see the source of the confusion. test_distutils fails because it runs 
in the build tree and without files installed. The test does not use the 
installed version of sysconfig, so it looks for ./Modules/ld_so_aix, which 
fails. The kludge from 2000 was trying to make the test work. But after 
installation, the kludge defines variables incorrectly to build Modules after 
Python is installed.

Because the distutils test is testing in tree and not the installed version of 
Python, it is not testing what the end user will experience. I am unsure about 
the best way to make the test pass.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Larry Hastings]

Changes by Larry Hastings la...@hastings.org:


--
nosy: +larry

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

Patch against current trunk

--
keywords: +patch
Added file: http://bugs.python.org/file32229/issue18235.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

With my patch applied, _sysconfig.py looks like

 'BLDSHARED': '/usr/local/lib/python3.4/config/ld_so_aix gcc -pthread '
  '-bI:/usr/local/lib/python3.4/config/python.exp',
 'LDSHARED': '/usr/local/lib/python3.4/config/ld_so_aix gcc -pthread '
 '-bI:/usr/local/lib/python3.4/config/python.exp',

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ha, that code dates back to 2000 and 2001 (f1e40abbedb3 and c958678720fd, 
respectively).

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Roundup Robot]

Roundup Robot added the comment:

New changeset c554194240fc by Antoine Pitrou in branch '3.3':
Issue #18235: Fix the sysconfig variables LDSHARED and BLDSHARED under AIX.
http://hg.python.org/cpython/rev/c554194240fc

New changeset e3ac917fcf9c by Antoine Pitrou in branch 'default':
Issue #18235: Fix the sysconfig variables LDSHARED and BLDSHARED under AIX.
http://hg.python.org/cpython/rev/e3ac917fcf9c

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Antoine Pitrou]

Antoine Pitrou added the comment:

Committed the patch. I trust you that you got it right!

--
resolution:  - fixed
stage:  - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Éric Araujo]

Éric Araujo added the comment:

Can you suggest how to fix this?

--
components: +Build -Extension Modules
nosy: +eric.araujo
versions: +Python 3.3, Python 3.4 -Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

It looks like someone already tried to fix part of this, but reversed the 
assignment

diff -r a089a8b1f93d Lib/sysconfig.py
--- a/Lib/sysconfig.py  Fri Jun 21 18:37:02 2013 -0400
+++ b/Lib/sysconfig.py  Fri Jun 21 22:33:15 2013 -0700
@@ -368,7 +368,7 @@
 # -- these paths are relative to the Python source, but when installed
 # the scripts are in another directory.
 if _PYTHON_BUILD:
-vars['LDSHARED'] = vars['BLDSHARED']
+vars['BLDSHARED'] = vars['LDSHARED']
 
 # There's a chicken-and-egg situation on OS X with regards to the
 # _sysconfigdata module after the changes introduced by #15298:


BLDSHARED is relative to srcdir and LDSHARED is relative to the install 
directory BINLIBDEST.

LDSHARED=   $(BINLIBDEST)/config/ld_so_aix $(CC) 
-bI:$(BINLIBDEST)/config/python.exp $(PY_LDFLAGS)
BLDSHARED=  $(srcdir)/Modules/ld_so_aix $(CC) 
-bI:$(srcdir)/Modules/python.exp $(PY_LDFLAGS)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

David Edelsohn added the comment:

This is the cause of the failures in test_distutils.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[David Edelsohn]

New submission from David Edelsohn:

_sysconfigdata.py includes information about how to build extension modules. On 
AIX this requires a wrapper script to build shared libraries.  The file 
includes definitions like:

 'BLDSHARED': './Modules/ld_so_aix gcc -pthread -bI:./Modules/python.exp',
 'LDSHARED': './Modules/ld_so_aix gcc -pthread -bI:./Modules/python.exp',
 'LINKCC': './Modules/makexp_aix Modules/python.exp . libpython3.4dm.a;  gcc '
   '-pthread',
 'MAKESETUP': './Modules/makesetup',

which is correct in the build directory, but is not correct for the install 
directory.  The paths do not correspond to the installed location of ld_so_aix 
and makexp_aix in lib/pythonX.Y/config .

--
components: Extension Modules
messages: 191301
nosy: David.Edelsohn
priority: normal
severity: normal
status: open
title: _sysconfigdata.py wrong on AIX installations
type: behavior
versions: Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18235
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com