On Dec 23, 2011, at 4:55pm, Russell Jurney wrote: > Avro 1.53 doesn't have this issue? Does it use python-snappy?
No - this was (I think) right before Tom committed support for Snappy in Python. -- Ken > > Russell Jurney > twitter.com/rjurney > [email protected] > datasyndrome.com > > On Dec 23, 2011, at 7:05 PM, Ken Krugler <[email protected]> wrote: > >> I installed brew, then ran 'brew install snappy', which worked. >> >> But 'sudo python setup.py install' still fails, though with a different >> problem (I was getting the same compilation errors as below, now I get a >> broken pipe error). >> >>> running install >>> running bdist_egg >>> running egg_info >>> writing requirements to avro.egg-info/requires.txt >>> writing avro.egg-info/PKG-INFO >>> writing top-level names to avro.egg-info/top_level.txt >>> writing dependency_links to avro.egg-info/dependency_links.txt >>> reading manifest file 'avro.egg-info/SOURCES.txt' >>> writing manifest file 'avro.egg-info/SOURCES.txt' >>> installing library code to build/bdist.macosx-10.6-universal/egg >>> running install_lib >>> running build_py >>> creating build/bdist.macosx-10.6-universal/egg >>> creating build/bdist.macosx-10.6-universal/egg/avro >>> copying build/lib/avro/__init__.py -> >>> build/bdist.macosx-10.6-universal/egg/avro >>> copying build/lib/avro/datafile.py -> >>> build/bdist.macosx-10.6-universal/egg/avro >>> copying build/lib/avro/io.py -> build/bdist.macosx-10.6-universal/egg/avro >>> copying build/lib/avro/ipc.py -> build/bdist.macosx-10.6-universal/egg/avro >>> copying build/lib/avro/protocol.py -> >>> build/bdist.macosx-10.6-universal/egg/avro >>> copying build/lib/avro/schema.py -> >>> build/bdist.macosx-10.6-universal/egg/avro >>> copying build/lib/avro/tool.py -> build/bdist.macosx-10.6-universal/egg/avro >>> copying build/lib/avro/txipc.py -> >>> build/bdist.macosx-10.6-universal/egg/avro >>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/__init__.py to >>> __init__.pyc >>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/datafile.py to >>> datafile.pyc >>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/io.py to io.pyc >>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/ipc.py to ipc.pyc >>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/protocol.py to >>> protocol.pyc >>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/schema.py to >>> schema.pyc >>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/tool.py to >>> tool.pyc >>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/txipc.py to >>> txipc.pyc >>> creating build/bdist.macosx-10.6-universal/egg/EGG-INFO >>> installing scripts to build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts >>> running install_scripts >>> running build_scripts >>> creating build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts >>> copying build/scripts-2.6/avro -> >>> build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts >>> changing mode of >>> build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts/avro to 755 >>> copying avro.egg-info/PKG-INFO -> >>> build/bdist.macosx-10.6-universal/egg/EGG-INFO >>> copying avro.egg-info/SOURCES.txt -> >>> build/bdist.macosx-10.6-universal/egg/EGG-INFO >>> copying avro.egg-info/dependency_links.txt -> >>> build/bdist.macosx-10.6-universal/egg/EGG-INFO >>> copying avro.egg-info/requires.txt -> >>> build/bdist.macosx-10.6-universal/egg/EGG-INFO >>> copying avro.egg-info/top_level.txt -> >>> build/bdist.macosx-10.6-universal/egg/EGG-INFO >>> zip_safe flag not set; analyzing archive contents... >>> creating 'dist/avro-1.6.1-py2.6.egg' and adding >>> 'build/bdist.macosx-10.6-universal/egg' to it >>> removing 'build/bdist.macosx-10.6-universal/egg' (and everything under it) >>> Processing avro-1.6.1-py2.6.egg >>> Removing /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg >>> Copying avro-1.6.1-py2.6.egg to /Library/Python/2.6/site-packages >>> avro 1.6.1 is already the active version in easy-install.pth >>> Installing avro script to /usr/local/bin >>> >>> Installed /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg >>> Processing dependencies for avro==1.6.1 >>> Searching for python-snappy >>> Reading http://pypi.python.org/simple/python-snappy/ >>> Reading http://github.com/andrix/python-snappy >>> Best match: python-snappy 0.3.2 >>> Downloading >>> http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f >>> Processing python-snappy-0.3.2.tar.gz >>> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir >>> /tmp/easy_install-jARda8/python-snappy-0.3.2/egg-dist-tmp-DDcMTW >>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for >>> C/ObjC but not for C++ >>> snappymodule.cc:41: warning: ‘_state’ defined but not used >>> /usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler >>> (/usr/bin/../libexec/gcc/darwin/ppc/as or >>> /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not >>> installed >>> Installed assemblers are: >>> /usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64 >>> /usr/bin/../libexec/gcc/darwin/i386/as for architecture i386 >>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for >>> C/ObjC but not for C++ >>> snappymodule.cc:41: warning: ‘_state’ defined but not used >>> snappymodule.cc:246: fatal error: error writing to -: Broken pipe >>> compilation terminated. >>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for >>> C/ObjC but not for C++ >>> snappymodule.cc:41: warning: ‘_state’ defined but not used >>> lipo: can't open input file: /var/tmp//cc1RAU1Q.out (No such file or >>> directory) >>> error: Setup script exited with error: command 'gcc-4.2' failed with exit >>> status 1 >> >> >> Note that I have to run as sudo, otherwise I get: >> >> Installing avro script to /usr/local/bin >> error: /usr/local/bin/avro: Permission denied >> >> I've added my vote to https://issues.apache.org/jira/browse/AVRO-981, and a >> comment about why I think it's good to fix this issue. >> >> In the meantime, we're going to use Avro 1.5.3 >> >> Thanks, >> >> -- Ken >> >> On Dec 14, 2011, at 7:07pm, Russell Jurney wrote: >> >>> I installed the snappy macport, and I still can't get snappy-python to >>> build. I'll try switching to brew. >>> >>> On Wed, Dec 14, 2011 at 5:46 PM, Joe Crobak <[email protected]> wrote: >>> It appears that the python-snappy library assumes that the snappy library >>> has already been installed. I saw similar errors, but once I installed >>> snappy (I use homebrew, so 'brew install snappy') python-snappy installs >>> fine. >>> >>> HTH, >>> Joe >>> >>> >>> On Wed, Dec 14, 2011 at 8:23 PM, Russell Jurney <[email protected]> >>> wrote: >>> I am unable to build Avro for Python on OS X 10.6.8 because python-snappy >>> fails to build. Is there a way around this? >>> >>> I wrote a post here >>> http://datasyndrome.com/post/13707537045/booting-the-analytics-application-events-ruby >>> about how to use Avro with Pig and Ruby. The code is here: >>> https://github.com/rjurney/Booting-the-Analytics-Application I am trying >>> to create Python instructions that parallel the Ruby ones. >>> >>> I've tried multiple install methods, and I am unable to build >>> snappy-python, so the avro build fails. This is the error message: >>> >>> Installed >>> /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg >>> Processing dependencies for avro==-AVRO-VERSION- >>> Searching for python-snappy >>> Reading http://pypi.python.org/simple/python-snappy/ >>> Reading http://github.com/andrix/python-snappy >>> Best match: python-snappy 0.3.2 >>> Downloading >>> http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f >>> Processing python-snappy-0.3.2.tar.gz >>> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir >>> /tmp/easy_install-0KEFAv/python-snappy-0.3.2/egg-dist-tmp-1BPj3j >>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for >>> C/ObjC but not for C++ >>> snappymodule.cc:31:22: error: snappy-c.h: No such file or directory >>> snappymodule.cc: In function ‘PyObject* snappy__compress(PyObject*, >>> PyObject*)’: >>> snappymodule.cc:62: error: ‘snappy_status’ was not declared in this scope >>> snappymodule.cc:62: error: expected `;' before ‘status’ >>> snappymodule.cc:75: error: ‘snappy_max_compressed_length’ was not declared >>> in this scope >>> snappymodule.cc:79: error: ‘status’ was not declared in this scope >>> snappymodule.cc:79: error: ‘snappy_compress’ was not declared in this scope >>> snappymodule.cc:81: error: ‘SNAPPY_OK’ was not declared in this scope >>> snappymodule.cc: In function ‘PyObject* snappy__uncompress(PyObject*, >>> PyObject*)’: >>> snappymodule.cc:107: error: ‘snappy_status’ was not declared in this scope >>> snappymodule.cc:107: error: expected `;' before ‘status’ >>> snappymodule.cc:120: error: ‘status’ was not declared in this scope >>> snappymodule.cc:120: error: ‘snappy_uncompressed_length’ was not declared >>> in this scope >>> snappymodule.cc:121: error: ‘SNAPPY_OK’ was not declared in this scope >>> snappymodule.cc:128: error: ‘snappy_uncompress’ was not declared in this >>> scope >>> snappymodule.cc:129: error: ‘SNAPPY_OK’ was not declared in this scope >>> snappymodule.cc: In function ‘PyObject* >>> snappy__is_valid_compressed_buffer(PyObject*, PyObject*)’: >>> snappymodule.cc:151: error: ‘snappy_status’ was not declared in this scope >>> snappymodule.cc:151: error: expected `;' before ‘status’ >>> snappymodule.cc:156: error: ‘status’ was not declared in this scope >>> snappymodule.cc:156: error: ‘snappy_validate_compressed_buffer’ was not >>> declared in this scope >>> snappymodule.cc:157: error: ‘SNAPPY_OK’ was not declared in this scope >>> snappymodule.cc: At global scope: >>> snappymodule.cc:41: warning: ‘_state’ defined but not used >>> error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed >>> with exit status 1 >>> -- >> >> -------------------------- >> Ken Krugler >> http://www.scaleunlimited.com >> custom big data solutions & training >> Hadoop, Cascading, Mahout & Solr >> >> >> >> -------------------------- Ken Krugler http://www.scaleunlimited.com custom big data solutions & training Hadoop, Cascading, Mahout & Solr
