Comment #5 on issue 275 by zbe...@google.com: Incorrect installation with setup.py breaks local override
http://code.google.com/p/protobuf/issues/detail?id=275

Thanks for the explanation.

I believe this needs a little clarification though:
The local chrome tests don't use protobuf installed with --root. They just have the live in a third_party/protobuf/ directory and they are installed using an embedded .gyp file. (http://src.chromium.org/svn/trunk/src/third_party/protobuf/protobuf.gyp, look for "'target_name': 'py_proto'," to see the self-explanatory build rules.)

This installation seems "correct", as it has the __init__.py both in the namespace (google) and the package (protobuf). It does not, however, use setuptools or distutils, it just copies files, so there may be some issue here. But:

The global system package (installed by the linux distribution) is installed using setuptools with --root, which is correct according to what P.J said, because it is, afterall, a distribution package (not necessarily an rpm). It also contains the protobuf-$version-$pyversion-nspkg.pth file.

And in that enviroment, overriding the system package seems impossible. The code overrides PYTHONPATH prior to running the .py files, and import google.protobuf always pulls the wrong one, it is always the system package that gets imported, no matter what. This is a summary that one of the developers digging in this wrote about it in email discussion:
----snip----
To cut a long story short:
/usr/local/lib/python2.6/site-packages/protobuf-2.3.0-py2.6-nspkg.pth
exists and is processed by the 'site' module (which is automatically
imported by Python). That pth contains this wonderful code:

import sys,types,os; p =
os.path.join(sys._getframe(1).f_locals['sitedir'], *('google',)); ie =
os.path.exists(os.path.join(p,'__init__.py')); m = not ie and
sys.modules.setdefault('google',types.ModuleType('google')); mp = (m
or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and
mp.append(p)

Which basically tests for
/usr/local/lib/python2.6/site-packages/google/__init__.py (which
doesn't exist) and, because it doesn't exist, adds a built in 'google'
package to the modules list. Once the modules list already contains
'google', sys.path is ignored when trying to import submodules (like
the protobuf stuff) because the module is "already loaded".

I have no idea what that pth file is trying to achieve, but I touched
/usr/local/lib/python2.6/site-packages/google/__init__.py and now
things appear to be working, on that host at least.
----snip----

From the PEP:
"The import statement is extended so that it directly considers *.pth files during import; a directory is considered a package if it either contains a file named __init__.py, or a file whose name ends with ".pth". Unlike .pth files on the top level, lines starting with "import" are not supported in per-package .pth files."

This makes sense, but both of the places that we want to import from are valid in this sense (one has a .pth, the other has an __init__.py), and one of them always takes precedence (the one with .pth).

So, I'm not sure if this is an OutOfScope, but I'm certainly still confused about what we (or someone else) did wrong.

--
You received this message because you are subscribed to the Google Groups "Protocol 
Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

Reply via email to