Matt, Thanks for the reply so I can join this discussion because I wasn't subscribe to [nupic-discuss] previoiusly.
> Once we've made some decisions, we can take the implementation discussion to nupic-hackers (anyone is welcome to do this). I've already summarized a proposal regarding this on [nupic-hackers] asProposal to move building of nupic and nupic.core towards standard workflow: http://lists.numenta.org/pipermail/nupic-hackers_lists.numenta.org/2014-April/000520.html. For those who's not subscribed to [nupic-hackers], you are welcome to join the discussion at https://github.com/numenta/nupic/issues/805 on Github . 2014-04-14 4:30 GMT+08:00 Matthew Taylor <[email protected]>: > Rik, > > We decided to have this conversation on the nupic-discuss list first, > because it will affect how users build and install NuPIC. Once we've > made some decisions, we can take the implementation discussion to > nupic-hackers (anyone is welcome to do this). > --------- > Matt Taylor > OS Community Flag-Bearer > Numenta > > > On Fri, Apr 11, 2014 at 9:38 PM, Rik <[email protected]> wrote: > > Probably too late at this point but this thread go to nupic-hackers? > > > > On Sat, Apr 12, 2014, at 08:36 AM, Subutai Ahmad wrote: > >> Yes, I think this is common. There are many Python packages that also > >> compile C++ libraries. In those cases setup.py calls make, but in our > >> case > >> we can have it call cmake. This means all the current work can stay > >> pretty > >> much as-is. > >> > >> --Subutai > >> > >> > >> On Fri, Apr 11, 2014 at 3:30 PM, Chetan Surpur <[email protected]> > >> wrote: > >> > >> > I think setup.py should call cmake, since Python is a layer on top of > C++, > >> > and C++ / cmake should be independent of Python. > >> > > >> > > >> > On Fri, Apr 11, 2014 at 3:23 PM, David Ragazzi < > [email protected]>wrote: > >> > > >> >> > In my ideal world, I think we start with cmake and make to build > the > >> >> c++ components, and then setup.py for the python installation in > lieu of a > >> >> typical `make install`, but both would be required: > >> >> > (cmake $NUPIC && make) > >> >> python setup.py (develop|install) OR easy_install . OR pip install . > OR > >> >> manual PYTHONPATH fixup. > >> >> > >> >> This is really a cruel question... > >> >> > >> >> As NuPIC is a mixture of Python and C++ code, I don't know surely if > >> >> CMake should call setup.py or if setup.py should call CMake.. :-( > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> On 11 April 2014 18:57, Austin Marshall <[email protected]> wrote: > >> >> > >> >>> The problem with standards is there are so many! The layout you've > >> >>> drawn here is fairly typical and consistent with what I would > expect. In > >> >>> short, there's usually a root directory with a README and setup.py, > and > >> >>> maybe bin/ docs/ and any number of python packages, but usually > just one. > >> >>> The term "package" is meaningful within the python ecosystem -- > it's any > >> >>> directory with a file called __init__.py, which can be empty, and > usually > >> >>> contains additional python modules (files with a .py suffix) or > packages. > >> >>> > >> >>> Where it gets difficult is that there are multiple competing > packaging > >> >>> implementations (setuptools, distutils, etc), multiple competing > packaging > >> >>> formats (egg vs wheel, binary vs source), multiple competing > installers > >> >>> (easy_install vs pip vs setup.py), and then of course the issue of > c/c++ > >> >>> extensions, all of which may be interchangeable at various different > >> >>> stages. For many of the approaches there is some redundancy with > the way > >> >>> we're using cmake with NuPIC. > >> >>> > >> >>> In my ideal world, I think we start with cmake and make to build > the c++ > >> >>> components, and then setup.py for the python installation in lieu > of a > >> >>> typical `make install`, but both would be required: > >> >>> > >> >>> (cmake $NUPIC && make) > >> >>> python setup.py (develop|install) OR easy_install . OR pip install > . OR > >> >>> manual PYTHONPATH fixup. > >> >>> > >> >>> The cmake and make commands build in-place, don't require env > variables, > >> >>> and doesn't affect anything outside of the $NUPIC source tree. > Then, the > >> >>> python setup.py ecosystem is used for using/installing the final > python > >> >>> package. I think this approach gives maximum control and > flexibility. > >> >>> > >> >>> > >> >>> > >> >>> On Fri, Apr 11, 2014 at 2:42 PM, David Ragazzi < > [email protected]>wrote: > >> >>> > >> >>>> Ok.. I found this one and liked it.. It also seems being well > >> >>>> recommended for Python projects.. > >> >>>> > >> >>>> I adapted to NuPIC.. > >> >>>> > >> >>>> *Nupic/* > >> >>>> *|-- bin/* > >> >>>> *| |-- nupic* > >> >>>> *|* > >> >>>> *|-- docs/* > >> >>>> *|* > >> >>>> *|-- nupic/* > >> >>>> *| |-- test/* > >> >>>> *| | |-- __init__.py* > >> >>>> *| | |-- test_main.py* > >> >>>> *| | * > >> >>>> *| |-- __init__.py* > >> >>>> *| |-- main.py* > >> >>>> *|* > >> >>>> *|-- setup.py* > >> >>>> *|-- README.md* > >> >>>> > >> >>>> I got this model from here: > >> >>>> > >> >>>> > http://stackoverflow.com/questions/193161/what-is-the-best-project-structure-for-a-python-application > >> >>>> > >> >>>> > >> >>>> Which is inspired in best practices recommended here: > >> >>>> > >> >>>> > http://as.ynchrono.us/2007/12/filesystem-structure-of-python-project_21.html > >> >>>> > >> >>>> Subutai, Chetan, Fergal, and others: what do you think? It has > setup.py > >> >>>> in root, the binaries and source are in separated locations, etc.. > However, > >> >>>> we could improve it... > >> >>>> > >> >>>> David > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> On 11 April 2014 16:07, David Ragazzi <[email protected]> > wrote: > >> >>>> > >> >>>>> Sorry for my ignorancy.. But which is default structure for Python > >> >>>>> projects? Please,someone explain to me in order to we can align > what we > >> >>>>> want and the current process does.. > >> >>>>> > >> >>>>> > >> >>>>> On 11 April 2014 15:53, Fergal Byrne <[email protected] > >wrote: > >> >>>>> > >> >>>>>> +1 on Subutai's view. IMHO it is a *requirement* for NuPIC to be > like > >> >>>>>> any other OSS software. > >> >>>>>> > >> >>>>>> For nupic.core, we should have cd build; cmake ..; make; [make > test;] > >> >>>>>> make install (installs to std locations /usr/local/bin, lib, > include) > >> >>>>>> For nupic.py; setup.py or pip install (installs to same place as > >> >>>>>> everything else) > >> >>>>>> > >> >>>>>> There should be no environment variables to remember or set. > >> >>>>>> > >> >>>>>> This will require serious reorganisation, but it is the price of > >> >>>>>> getting NuPIC out in the wild. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> On Fri, Apr 11, 2014 at 7:27 PM, Matthew Taylor < > [email protected]>wrote: > >> >>>>>> > >> >>>>>>> I can see Subutai's point here. While nupic.core is a C++ > project, > >> >>>>>>> nupic is a python project. We should try to maintain python > standards > >> >>>>>>> as much as possible, even though we have to deal with a C++ > build as > >> >>>>>>> well. > >> >>>>>>> --------- > >> >>>>>>> Matt Taylor > >> >>>>>>> OS Community Flag-Bearer > >> >>>>>>> Numenta > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> On Fri, Apr 11, 2014 at 11:18 AM, Subutai Ahmad < > [email protected]> > >> >>>>>>> wrote: > >> >>>>>>> > > >> >>>>>>> > I'm not an expert in this. I just know it is really really > hard to > >> >>>>>>> make > >> >>>>>>> > these things robust in all situations. For example, below > is my > >> >>>>>>> sys.path > >> >>>>>>> > - where would you install nupic? > >> >>>>>>> > > >> >>>>>>> > I strongly recommend we use the standard python way of doing > this, > >> >>>>>>> whatever > >> >>>>>>> > that is. > >> >>>>>>> > > >> >>>>>>> > --Subutai > >> >>>>>>> > > >> >>>>>>> > ['', > >> >>>>>>> > '/Users/Subutai/local/bin', > >> >>>>>>> > '/Users/Subutai/nta/grok-projects/gef', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/local/lib/python2.7/site-packages/pandas-0.13.1-py2.7-macosx-10.9-intel.egg', > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/local/lib/python2.7/site-packages/plotly-0.5.7-py2.7.egg', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/local/lib/python2.7/site-packages/simplejson-3.4.0-py2.7.egg', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/nta/nupic-darwin64/lib/python2.7/site-packages/setuptools-1.0-py2.7.egg', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/nta/nupic-darwin64/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/Library/Python/2.7/lib/python/site-packages/setuptools-2.0.1-py2.7.egg', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/Library/Python/2.7/site-packages/readline-6.2.4.1-py2.7-macosx-10.7-intel.egg', > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/nta/nupic-darwin64/nta/eng/lib/python2.7/site-packages', > >> >>>>>>> > '/Users/Subutai/nta/grok-projects', > >> >>>>>>> > '/Users/Subutai/local/lib/python2.7/site-packages', > >> >>>>>>> > '/Users/Subutai/nta/grok', > >> >>>>>>> > > '/Users/Subutai/nta/nupic-darwin64/lib64/python2.7/site-packages', > >> >>>>>>> > > '/Users/Subutai/nta/nupic-darwin64/lib/python2.7/site-packages', > >> >>>>>>> > '/Users/Subutai/local/src/ipython-1.1.0', > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/nta/nupic-darwin64/lib/python2.7/site-packages/PIL', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', > >> >>>>>>> > '/Users/Subutai/Library/Python/2.7/lib/python/site-packages', > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', > >> >>>>>>> > '/Library/Python/2.7/site-packages', > >> >>>>>>> > > '/Users/Subutai/nta/nupic-darwin64/lib64/python2.7/site-packages', > >> >>>>>>> > > >> >>>>>>> > '/Users/Subutai/local/lib/python2.7/site-packages/IPython/extensions'] > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > On Fri, Apr 11, 2014 at 11:12 AM, David Ragazzi < > >> >>>>>>> [email protected]> > >> >>>>>>> > wrote: > >> >>>>>>> >> > >> >>>>>>> >> We could use this in CMake: > >> >>>>>>> >> > >> >>>>>>> >> execute_process(COMMAND ${python} import sys; print sys.path; > >> >>>>>>> >> OUTPUT_VARIABLE PYTHON_PATH) > >> >>>>>>> >> > >> >>>>>>> >> and then install python libraries to ${PYTHON_PATH}/nupic > >> >>>>>>> >> > >> >>>>>>> >> :-) > >> >>>>>>> >> > >> >>>>>>> >> > >> >>>>>>> >> On 11 April 2014 15:06, David Ragazzi < > [email protected]> > >> >>>>>>> wrote: > >> >>>>>>> >>> > >> >>>>>>> >>> Subutai, > >> >>>>>>> >>> > >> >>>>>>> >>> Isn't.. sys.path py function return default location > >> >>>>>>> (PYTHONPATH).. It's > >> >>>>>>> >>> just we use this default location! > >> >>>>>>> >>> > >> >>>>>>> >>> > >> >>>>>>> >>> On 11 April 2014 15:01, Subutai Ahmad <[email protected]> > >> >>>>>>> wrote: > >> >>>>>>> >>>> > >> >>>>>>> >>>> > >> >>>>>>> >>>> Yes, but that *requires* you to set an environment > variable. > >> >>>>>>> >>>> > >> >>>>>>> >>>> --Subutai > >> >>>>>>> >>>> > >> >>>>>>> >>>> On Fri, Apr 11, 2014 at 10:16 AM, Matthew Taylor < > >> >>>>>>> [email protected]> > >> >>>>>>> >>>> wrote: > >> >>>>>>> >>>>> > >> >>>>>>> >>>>> On Fri, Apr 11, 2014 at 9:58 AM, Subutai Ahmad < > >> >>>>>>> [email protected]> > >> >>>>>>> >>>>> wrote: > >> >>>>>>> >>>>> > To see why it would not work consider this: I should be > able > >> >>>>>>> to login > >> >>>>>>> >>>>> > to my > >> >>>>>>> >>>>> > home directory, type python, and then do "import > nupic". It > >> >>>>>>> is > >> >>>>>>> >>>>> > necessary to > >> >>>>>>> >>>>> > support this. Unless the NuPIC release is *placed* in a > >> >>>>>>> standard > >> >>>>>>> >>>>> > location, > >> >>>>>>> >>>>> > python will have no idea where nupic is. > >> >>>>>>> >>>>> > >> >>>>>>> >>>>> +1. I do this all the time. However, I can easily update > my > >> >>>>>>> PYTHONPATH > >> >>>>>>> >>>>> after I've built NuPIC. > >> >>>>>>> >>>>> > >> >>>>>>> >>>>> > >> >>>>>>> >>>>> > >> >>>>>>> >>>>> --------- > >> >>>>>>> >>>>> Matt Taylor > >> >>>>>>> >>>>> OS Community Flag-Bearer > >> >>>>>>> >>>>> Numenta > >> >>>>>>> >>>>> > >> >>>>>>> >>>>> _______________________________________________ > >> >>>>>>> >>>>> nupic mailing list > >> >>>>>>> >>>>> [email protected] > >> >>>>>>> >>>>> > >> >>>>>>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>>>>>> >>>> > >> >>>>>>> >>>> > >> >>>>>>> >>>> > >> >>>>>>> >>>> _______________________________________________ > >> >>>>>>> >>>> nupic mailing list > >> >>>>>>> >>>> [email protected] > >> >>>>>>> >>>> > >> >>>>>>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>>>>>> >>>> > >> >>>>>>> >>> > >> >>>>>>> >>> > >> >>>>>>> >>> > >> >>>>>>> >>> -- > >> >>>>>>> >>> David Ragazzi > >> >>>>>>> >>> OS Community Commiter > >> >>>>>>> >>> Numenta.org > >> >>>>>>> >>> -- > >> >>>>>>> >>> "I think James Connolly, the Irish revolutionary, is right > when > >> >>>>>>> he says > >> >>>>>>> >>> that the only prophets are those who make their future. So > we're > >> >>>>>>> not > >> >>>>>>> >>> anticipating, we're working for it." > >> >>>>>>> >> > >> >>>>>>> >> > >> >>>>>>> >> > >> >>>>>>> >> > >> >>>>>>> >> -- > >> >>>>>>> >> David Ragazzi > >> >>>>>>> >> OS Community Commiter > >> >>>>>>> >> Numenta.org > >> >>>>>>> >> -- > >> >>>>>>> >> "I think James Connolly, the Irish revolutionary, is right > when > >> >>>>>>> he says > >> >>>>>>> >> that the only prophets are those who make their future. So > we're > >> >>>>>>> not > >> >>>>>>> >> anticipating, we're working for it." > >> >>>>>>> >> > >> >>>>>>> >> _______________________________________________ > >> >>>>>>> >> nupic mailing list > >> >>>>>>> >> [email protected] > >> >>>>>>> >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>>>>>> >> > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > _______________________________________________ > >> >>>>>>> > nupic mailing list > >> >>>>>>> > [email protected] > >> >>>>>>> > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>>>>>> > > >> >>>>>>> > >> >>>>>>> _______________________________________________ > >> >>>>>>> nupic mailing list > >> >>>>>>> [email protected] > >> >>>>>>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> -- > >> >>>>>> > >> >>>>>> Fergal Byrne, Brenter IT > >> >>>>>> > >> >>>>>> Author, Real Machine Intelligence with Clortex and NuPIC > >> >>>>>> https://leanpub.com/realsmartmachines > >> >>>>>> > >> >>>>>> <http://www.examsupport.ie>http://inbits.com - Better Living > through > >> >>>>>> Thoughtful Technology > >> >>>>>> http://ie.linkedin.com/in/fergbyrne/ > >> >>>>>> https://github.com/fergalbyrne > >> >>>>>> > >> >>>>>> e:[email protected] t:+353 83 4214179 > >> >>>>>> Formerly of Adnet [email protected] http://www.adnet.ie > >> >>>>>> > >> >>>>>> _______________________________________________ > >> >>>>>> nupic mailing list > >> >>>>>> [email protected] > >> >>>>>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>>>>> > >> >>>>>> > >> >>>>> > >> >>>>> > >> >>>>> -- > >> >>>>> David Ragazzi > >> >>>>> OS Community Commiter > >> >>>>> Numenta.org > >> >>>>> -- > >> >>>>> "I think James Connolly, the Irish revolutionary, is right when he > >> >>>>> says that the only prophets are those who make their future. So > we're > >> >>>>> not anticipating, we're working for it." > >> >>>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> David Ragazzi > >> >>>> OS Community Commiter > >> >>>> Numenta.org > >> >>>> -- > >> >>>> "I think James Connolly, the Irish revolutionary, is right when he > says that > >> >>>> the only prophets are those who make their future. So we're not > >> >>>> anticipating, we're working for it." > >> >>>> > >> >>>> _______________________________________________ > >> >>>> nupic mailing list > >> >>>> [email protected] > >> >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>>> > >> >>>> > >> >>> > >> >>> _______________________________________________ > >> >>> nupic mailing list > >> >>> [email protected] > >> >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> > >> >>> > >> >> > >> >> > >> >> -- > >> >> David Ragazzi > >> >> OS Community Commiter > >> >> Numenta.org > >> >> -- > >> >> "I think James Connolly, the Irish revolutionary, is right when he > says that > >> >> the only prophets are those who make their future. So we're not > >> >> anticipating, we're working for it." > >> >> > >> >> _______________________________________________ > >> >> nupic mailing list > >> >> [email protected] > >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >> > >> >> > >> > > >> > _______________________________________________ > >> > nupic mailing list > >> > [email protected] > >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> > > >> > > >> _______________________________________________ > >> nupic mailing list > >> [email protected] > >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > _______________________________________________ > > nupic mailing list > > [email protected] > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > -- In fascination of creating worlds by words,and in pursuit of words behind the world.
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
