Yes, I think this is common. There are many Python packages that also compile C++ libraries. In those cases setup.py calls make, but in our case we can have it call cmake. This means all the current work can stay pretty much as-is.
--Subutai On Fri, Apr 11, 2014 at 3:30 PM, Chetan Surpur <[email protected]> wrote: > I think setup.py should call cmake, since Python is a layer on top of C++, > and C++ / cmake should be independent of Python. > > > On Fri, Apr 11, 2014 at 3:23 PM, David Ragazzi <[email protected]>wrote: > >> > In my ideal world, I think we start with cmake and make to build the >> c++ components, and then setup.py for the python installation in lieu of a >> typical `make install`, but both would be required: >> > (cmake $NUPIC && make) >> python setup.py (develop|install) OR easy_install . OR pip install . OR >> manual PYTHONPATH fixup. >> >> This is really a cruel question... >> >> As NuPIC is a mixture of Python and C++ code, I don't know surely if >> CMake should call setup.py or if setup.py should call CMake.. :-( >> >> >> >> >> >> >> On 11 April 2014 18:57, Austin Marshall <[email protected]> wrote: >> >>> The problem with standards is there are so many! The layout you've >>> drawn here is fairly typical and consistent with what I would expect. In >>> short, there's usually a root directory with a README and setup.py, and >>> maybe bin/ docs/ and any number of python packages, but usually just one. >>> The term "package" is meaningful within the python ecosystem -- it's any >>> directory with a file called __init__.py, which can be empty, and usually >>> contains additional python modules (files with a .py suffix) or packages. >>> >>> Where it gets difficult is that there are multiple competing packaging >>> implementations (setuptools, distutils, etc), multiple competing packaging >>> formats (egg vs wheel, binary vs source), multiple competing installers >>> (easy_install vs pip vs setup.py), and then of course the issue of c/c++ >>> extensions, all of which may be interchangeable at various different >>> stages. For many of the approaches there is some redundancy with the way >>> we're using cmake with NuPIC. >>> >>> In my ideal world, I think we start with cmake and make to build the c++ >>> components, and then setup.py for the python installation in lieu of a >>> typical `make install`, but both would be required: >>> >>> (cmake $NUPIC && make) >>> python setup.py (develop|install) OR easy_install . OR pip install . OR >>> manual PYTHONPATH fixup. >>> >>> The cmake and make commands build in-place, don't require env variables, >>> and doesn't affect anything outside of the $NUPIC source tree. Then, the >>> python setup.py ecosystem is used for using/installing the final python >>> package. I think this approach gives maximum control and flexibility. >>> >>> >>> >>> On Fri, Apr 11, 2014 at 2:42 PM, David Ragazzi >>> <[email protected]>wrote: >>> >>>> Ok.. I found this one and liked it.. It also seems being well >>>> recommended for Python projects.. >>>> >>>> I adapted to NuPIC.. >>>> >>>> *Nupic/* >>>> *|-- bin/* >>>> *| |-- nupic* >>>> *|* >>>> *|-- docs/* >>>> *|* >>>> *|-- nupic/* >>>> *| |-- test/* >>>> *| | |-- __init__.py* >>>> *| | |-- test_main.py* >>>> *| | * >>>> *| |-- __init__.py* >>>> *| |-- main.py* >>>> *|* >>>> *|-- setup.py* >>>> *|-- README.md* >>>> >>>> I got this model from here: >>>> >>>> http://stackoverflow.com/questions/193161/what-is-the-best-project-structure-for-a-python-application >>>> >>>> >>>> Which is inspired in best practices recommended here: >>>> >>>> http://as.ynchrono.us/2007/12/filesystem-structure-of-python-project_21.html >>>> >>>> Subutai, Chetan, Fergal, and others: what do you think? It has setup.py >>>> in root, the binaries and source are in separated locations, etc.. However, >>>> we could improve it... >>>> >>>> David >>>> >>>> >>>> >>>> >>>> On 11 April 2014 16:07, David Ragazzi <[email protected]> wrote: >>>> >>>>> Sorry for my ignorancy.. But which is default structure for Python >>>>> projects? Please,someone explain to me in order to we can align what we >>>>> want and the current process does.. >>>>> >>>>> >>>>> On 11 April 2014 15:53, Fergal Byrne <[email protected]>wrote: >>>>> >>>>>> +1 on Subutai's view. IMHO it is a *requirement* for NuPIC to be like >>>>>> any other OSS software. >>>>>> >>>>>> For nupic.core, we should have cd build; cmake ..; make; [make test;] >>>>>> make install (installs to std locations /usr/local/bin, lib, include) >>>>>> For nupic.py; setup.py or pip install (installs to same place as >>>>>> everything else) >>>>>> >>>>>> There should be no environment variables to remember or set. >>>>>> >>>>>> This will require serious reorganisation, but it is the price of >>>>>> getting NuPIC out in the wild. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Apr 11, 2014 at 7:27 PM, Matthew Taylor <[email protected]>wrote: >>>>>> >>>>>>> I can see Subutai's point here. While nupic.core is a C++ project, >>>>>>> nupic is a python project. We should try to maintain python standards >>>>>>> as much as possible, even though we have to deal with a C++ build as >>>>>>> well. >>>>>>> --------- >>>>>>> Matt Taylor >>>>>>> OS Community Flag-Bearer >>>>>>> Numenta >>>>>>> >>>>>>> >>>>>>> On Fri, Apr 11, 2014 at 11:18 AM, Subutai Ahmad <[email protected]> >>>>>>> wrote: >>>>>>> > >>>>>>> > I'm not an expert in this. I just know it is really really hard to >>>>>>> make >>>>>>> > these things robust in all situations. For example, below is my >>>>>>> sys.path >>>>>>> > - where would you install nupic? >>>>>>> > >>>>>>> > I strongly recommend we use the standard python way of doing this, >>>>>>> whatever >>>>>>> > that is. >>>>>>> > >>>>>>> > --Subutai >>>>>>> > >>>>>>> > ['', >>>>>>> > '/Users/Subutai/local/bin', >>>>>>> > '/Users/Subutai/nta/grok-projects/gef', >>>>>>> > >>>>>>> > >>>>>>> '/Users/Subutai/local/lib/python2.7/site-packages/pandas-0.13.1-py2.7-macosx-10.9-intel.egg', >>>>>>> > >>>>>>> >>>>>>> '/Users/Subutai/local/lib/python2.7/site-packages/plotly-0.5.7-py2.7.egg', >>>>>>> > >>>>>>> > >>>>>>> '/Users/Subutai/local/lib/python2.7/site-packages/simplejson-3.4.0-py2.7.egg', >>>>>>> > >>>>>>> > >>>>>>> '/Users/Subutai/nta/nupic-darwin64/lib/python2.7/site-packages/setuptools-1.0-py2.7.egg', >>>>>>> > >>>>>>> > >>>>>>> '/Users/Subutai/nta/nupic-darwin64/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg', >>>>>>> > >>>>>>> > >>>>>>> '/Users/Subutai/Library/Python/2.7/lib/python/site-packages/setuptools-2.0.1-py2.7.egg', >>>>>>> > >>>>>>> > >>>>>>> '/Library/Python/2.7/site-packages/readline-6.2.4.1-py2.7-macosx-10.7-intel.egg', >>>>>>> > >>>>>>> >>>>>>> '/Users/Subutai/nta/nupic-darwin64/nta/eng/lib/python2.7/site-packages', >>>>>>> > '/Users/Subutai/nta/grok-projects', >>>>>>> > '/Users/Subutai/local/lib/python2.7/site-packages', >>>>>>> > '/Users/Subutai/nta/grok', >>>>>>> > '/Users/Subutai/nta/nupic-darwin64/lib64/python2.7/site-packages', >>>>>>> > '/Users/Subutai/nta/nupic-darwin64/lib/python2.7/site-packages', >>>>>>> > '/Users/Subutai/local/src/ipython-1.1.0', >>>>>>> > >>>>>>> '/Users/Subutai/nta/nupic-darwin64/lib/python2.7/site-packages/PIL', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', >>>>>>> > >>>>>>> >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', >>>>>>> > '/Users/Subutai/Library/Python/2.7/lib/python/site-packages', >>>>>>> > >>>>>>> > >>>>>>> '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', >>>>>>> > '/Library/Python/2.7/site-packages', >>>>>>> > '/Users/Subutai/nta/nupic-darwin64/lib64/python2.7/site-packages', >>>>>>> > >>>>>>> '/Users/Subutai/local/lib/python2.7/site-packages/IPython/extensions'] >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > On Fri, Apr 11, 2014 at 11:12 AM, David Ragazzi < >>>>>>> [email protected]> >>>>>>> > wrote: >>>>>>> >> >>>>>>> >> We could use this in CMake: >>>>>>> >> >>>>>>> >> execute_process(COMMAND ${python} import sys; print sys.path; >>>>>>> >> OUTPUT_VARIABLE PYTHON_PATH) >>>>>>> >> >>>>>>> >> and then install python libraries to ${PYTHON_PATH}/nupic >>>>>>> >> >>>>>>> >> :-) >>>>>>> >> >>>>>>> >> >>>>>>> >> On 11 April 2014 15:06, David Ragazzi <[email protected]> >>>>>>> wrote: >>>>>>> >>> >>>>>>> >>> Subutai, >>>>>>> >>> >>>>>>> >>> Isn't.. sys.path py function return default location >>>>>>> (PYTHONPATH).. It's >>>>>>> >>> just we use this default location! >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> On 11 April 2014 15:01, Subutai Ahmad <[email protected]> >>>>>>> wrote: >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> Yes, but that *requires* you to set an environment variable. >>>>>>> >>>> >>>>>>> >>>> --Subutai >>>>>>> >>>> >>>>>>> >>>> On Fri, Apr 11, 2014 at 10:16 AM, Matthew Taylor < >>>>>>> [email protected]> >>>>>>> >>>> wrote: >>>>>>> >>>>> >>>>>>> >>>>> On Fri, Apr 11, 2014 at 9:58 AM, Subutai Ahmad < >>>>>>> [email protected]> >>>>>>> >>>>> wrote: >>>>>>> >>>>> > To see why it would not work consider this: I should be able >>>>>>> to login >>>>>>> >>>>> > to my >>>>>>> >>>>> > home directory, type python, and then do "import nupic". It >>>>>>> is >>>>>>> >>>>> > necessary to >>>>>>> >>>>> > support this. Unless the NuPIC release is *placed* in a >>>>>>> standard >>>>>>> >>>>> > location, >>>>>>> >>>>> > python will have no idea where nupic is. >>>>>>> >>>>> >>>>>>> >>>>> +1. I do this all the time. However, I can easily update my >>>>>>> PYTHONPATH >>>>>>> >>>>> after I've built NuPIC. >>>>>>> >>>>> >>>>>>> >>>>> >>>>>>> >>>>> >>>>>>> >>>>> --------- >>>>>>> >>>>> Matt Taylor >>>>>>> >>>>> OS Community Flag-Bearer >>>>>>> >>>>> Numenta >>>>>>> >>>>> >>>>>>> >>>>> _______________________________________________ >>>>>>> >>>>> nupic mailing list >>>>>>> >>>>> [email protected] >>>>>>> >>>>> >>>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> _______________________________________________ >>>>>>> >>>> nupic mailing list >>>>>>> >>>> [email protected] >>>>>>> >>>> >>>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>>> >>>> >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> -- >>>>>>> >>> David Ragazzi >>>>>>> >>> OS Community Commiter >>>>>>> >>> Numenta.org >>>>>>> >>> -- >>>>>>> >>> "I think James Connolly, the Irish revolutionary, is right when >>>>>>> he says >>>>>>> >>> that the only prophets are those who make their future. So we're >>>>>>> not >>>>>>> >>> anticipating, we're working for it." >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> -- >>>>>>> >> David Ragazzi >>>>>>> >> OS Community Commiter >>>>>>> >> Numenta.org >>>>>>> >> -- >>>>>>> >> "I think James Connolly, the Irish revolutionary, is right when >>>>>>> he says >>>>>>> >> that the only prophets are those who make their future. So we're >>>>>>> not >>>>>>> >> anticipating, we're working for it." >>>>>>> >> >>>>>>> >> _______________________________________________ >>>>>>> >> nupic mailing list >>>>>>> >> [email protected] >>>>>>> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>>> >> >>>>>>> > >>>>>>> > >>>>>>> > _______________________________________________ >>>>>>> > nupic mailing list >>>>>>> > [email protected] >>>>>>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>>> > >>>>>>> >>>>>>> _______________________________________________ >>>>>>> nupic mailing list >>>>>>> [email protected] >>>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Fergal Byrne, Brenter IT >>>>>> >>>>>> Author, Real Machine Intelligence with Clortex and NuPIC >>>>>> https://leanpub.com/realsmartmachines >>>>>> >>>>>> <http://www.examsupport.ie>http://inbits.com - Better Living through >>>>>> Thoughtful Technology >>>>>> http://ie.linkedin.com/in/fergbyrne/ >>>>>> https://github.com/fergalbyrne >>>>>> >>>>>> e:[email protected] t:+353 83 4214179 >>>>>> Formerly of Adnet [email protected] http://www.adnet.ie >>>>>> >>>>>> _______________________________________________ >>>>>> nupic mailing list >>>>>> [email protected] >>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> David Ragazzi >>>>> OS Community Commiter >>>>> Numenta.org >>>>> -- >>>>> "I think James Connolly, the Irish revolutionary, is right when he >>>>> says that the only prophets are those who make their future. So we're >>>>> not anticipating, we're working for it." >>>>> >>>> >>>> >>>> >>>> -- >>>> David Ragazzi >>>> OS Community Commiter >>>> Numenta.org >>>> -- >>>> "I think James Connolly, the Irish revolutionary, is right when he says >>>> that >>>> the only prophets are those who make their future. So we're not >>>> anticipating, we're working for it." >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> >> -- >> David Ragazzi >> OS Community Commiter >> Numenta.org >> -- >> "I think James Connolly, the Irish revolutionary, is right when he says that >> the only prophets are those who make their future. So we're not >> anticipating, we're working for it." >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
