Comment #5 on issue 275 by zbe...@google.com: Incorrect installation with
setup.py breaks local override
http://code.google.com/p/protobuf/issues/detail?id=275
Thanks for the explanation.
I believe this needs a little clarification though:
The local chrome tests don't use protobuf installed with --root. They just
have the live in a third_party/protobuf/ directory and they are installed
using an embedded .gyp file.
(http://src.chromium.org/svn/trunk/src/third_party/protobuf/protobuf.gyp,
look for "'target_name': 'py_proto'," to see the self-explanatory build
rules.)
This installation seems "correct", as it has the __init__.py both in the
namespace (google) and the package (protobuf). It does not, however, use
setuptools or distutils, it just copies files, so there may be some issue
here. But:
The global system package (installed by the linux distribution) is
installed using setuptools with --root, which is correct according to what
P.J said, because it is, afterall, a distribution package (not necessarily
an rpm). It also contains the protobuf-$version-$pyversion-nspkg.pth file.
And in that enviroment, overriding the system package seems impossible. The
code overrides PYTHONPATH prior to running the .py files, and import
google.protobuf always pulls the wrong one, it is always the system package
that gets imported, no matter what. This is a summary that one of the
developers digging in this wrote about it in email discussion:
----snip----
To cut a long story short:
/usr/local/lib/python2.6/site-packages/protobuf-2.3.0-py2.6-nspkg.pth
exists and is processed by the 'site' module (which is automatically
imported by Python). That pth contains this wonderful code:
import sys,types,os; p =
os.path.join(sys._getframe(1).f_locals['sitedir'], *('google',)); ie =
os.path.exists(os.path.join(p,'__init__.py')); m = not ie and
sys.modules.setdefault('google',types.ModuleType('google')); mp = (m
or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and
mp.append(p)
Which basically tests for
/usr/local/lib/python2.6/site-packages/google/__init__.py (which
doesn't exist) and, because it doesn't exist, adds a built in 'google'
package to the modules list. Once the modules list already contains
'google', sys.path is ignored when trying to import submodules (like
the protobuf stuff) because the module is "already loaded".
I have no idea what that pth file is trying to achieve, but I touched
/usr/local/lib/python2.6/site-packages/google/__init__.py and now
things appear to be working, on that host at least.
----snip----
From the PEP:
"The import statement is extended so that it directly considers *.pth files
during import; a directory is considered a package if it either contains a
file named __init__.py, or a file whose name ends with ".pth". Unlike .pth
files on the top level, lines starting with "import" are not supported in
per-package .pth files."
This makes sense, but both of the places that we want to import from are
valid in this sense (one has a .pth, the other has an __init__.py), and one
of them always takes precedence (the one with .pth).
So, I'm not sure if this is an OutOfScope, but I'm certainly still confused
about what we (or someone else) did wrong.
--
You received this message because you are subscribed to the Google Groups "Protocol
Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.