Hi David, Agreed. That's really the essence of my proposal: we add an install step to put things in standard locations. Everything we've done so far will flow naturally - I wasn't suggesting redoing the current process. I really like the simplicity of the current flow that you have been the leader of. We would still use cmake for everything (except a wrapper setup.py script for python repos).
In terms of python installs, there is a standard python way of doing it. We should follow that mechanism rather than inventing our own (for example a simple parsing of PYTHONPATH will not work in all cases). --Subutai On Fri, Apr 11, 2014 at 10:55 AM, David Ragazzi <[email protected]>wrote: > Hi Subutai, > > I agree in most parts, but in other ones I think we should keep the > current process.. it's tiny, simple, cross-platform, and is working well > (although this environment variables issue).. I think the only "stubmbling > block" is this manual configuration of env variables. However you raise an > important issue which I hadn't realized: > > > >To see why it would not work consider this: I should be able to login to > my home directory, type python, and then do "import nupic". It is necessary > to support this. Unless the NuPIC release is *placed* in a standard > location, python will have no idea where nupic is. > > If other apps beyond $NUPIC tests and examples want consume NuPIC python > libraries, we will have this problem mentioned by you.. However PYTHONPATH > solve this (as Matt said).. But suggesting the configuration of PYTHONPATH > goes > against the proposal to eliminate the manual configuration of envinroment > variables.. In this case, I fully agree with you on use of default > locations by platform... We should copy/install $NUPIC python stuff to a > default location which has global scope in an OS without need we config > environment variables... How we do that? Using you case as example: in > CMake we simply would get PYTHONPATH value from environment, and would copy > the content of the NuPIC python libraries to location pointed by it... > > David > > > > On 11 April 2014 13:58, Subutai Ahmad <[email protected]> wrote: > >> >> OK, so I think the spirit of this proposal is excellent but I don't think >> this mechanism will work in general. Also, it's not a standard way of doing >> things. >> >> To see why it would not work consider this: I should be able to login to >> my home directory, type python, and then do "import nupic". It is necessary >> to support this. Unless the NuPIC release is *placed* in a standard >> location, python will have no idea where nupic is. >> >> We also need to support a third type of user, which will eventually be >> the most common type of user: the user who does NOT clone the repo but >> instead just downloads a binary package. >> >> The standard unix/linux workflow for all this is: >> >> 1) build step started from the root of the repository. By default this >> build step puts all generated files inside the repository, but there is an >> optional argument if you want to place the build files somewhere specific. >> In python this is done through "setup.py". In C++ this is usually just >> "configure" and "make". We will use cmake for c++ but eventually may need >> to use setup.py for python (setup.py will just call cmake). >> >> 2) an install step started from the root of the repository. By default >> the install step puts all generated files inside standard locations, such >> as /usr/local/... In this case you need sudo access. There is also an >> optional --prefix argument if you want to put the release files somewhere >> specific. In python this is done through setup.py. In C++ this is "make >> install". >> >> 3) Optional test step. This is run from the root of the repository and >> assumes everything is in the standard locations. It can take optional >> arguments for non standard installs. >> >> I think we should move towards this standard workflow. This does not >> require environment variables for those who follow the standard install >> process. It also doesn't need any special cases in the code itself. It does >> not touch .bashrc or your global environment. If you need more >> sophisticated environments you have the option of setting variables. >> Temporarily, we could override if $NUPIC or $NTA is set, but ultimately >> these will also not be necessary. >> >> It also supports the third type of user mentioned above. They just need >> to run the install step from the downloaded package. >> >> --Subutai >> >> >> >> On Fri, Apr 11, 2014 at 9:35 AM, David Ragazzi <[email protected]>wrote: >> >>> > "We would like to see if there is a way that users don't have to set >>> any variables. The proposal is to instrument our code such that if the >>> variables are set, we use them. If they are not set, the code uses some >>> default locations that are platform specific." >>> >>> > I do have opinions, but before I give any, is my understanding of all >>> this correct? >>> >>> Yes, exactly, Subutai! (except that default locations maybe don't need >>> are platform specific, i.e. taking "build/release" as default location for >>> $NTA, for example).. IMO this new process saves time of configurarion, >>> don't confuse newbies, and still avoids a serious problem: suppose I move >>> my repo to another folder or even other machine.. with the current state of >>> process, I should set $NUPIC and $NTA again.. on the contrary, the binaries >>> will be generated to the old folder.. With the new process, eveything is >>> done in an automatic way, without need user intervention! >>> >>> >>> On 11 April 2014 13:01, Subutai Ahmad <[email protected]> wrote: >>> >>>> >>>> It's a bit hard reading through the comments to understand the actual >>>> proposal. Here's what I understand so far: >>>> >>>> This discussion is about environment variables and how they should be >>>> handled within NuPIC. This is an important discussion (thank you David for >>>> starting the discussion on the mailing list). >>>> >>>> We are talking about two variables, and only two variables. $NUPIC and >>>> $NTA. >>>> >>>> 1) $NUPIC is the location of the repository. >>>> >>>> 2) $NTA is the location of the release. The release can be placed >>>> anywhere. Specifically, it does not have to be in $NUPIC/build/release. >>>> >>>> Currently the above need to be set by the user. Now, here is the >>>> proposal as I understand it: >>>> >>>> "We would like to see if there is a way that users don't have to set >>>> any variables. The proposal is to instrument our code such that if the >>>> variables are set, we use them. If they are not set, the code uses some >>>> default locations that are platform specific." >>>> >>>> I do have opinions, but before I give any, is my understanding of all >>>> this correct? >>>> >>>> As I said before, I think this is an important discussion to have, so >>>> thank you guys for starting it! >>>> >>>> --Subutai >>>> >>>> >>>> >>>> On Fri, Apr 11, 2014 at 7:54 AM, David Ragazzi >>>> <[email protected]>wrote: >>>> >>>>> Hi everyone, >>>>> >>>>> I'm bringing this discussion to here, as asked by Utensil and Matt.. >>>>> >>>>> Recently I proposed a PR that automatically set environment variables >>>>> (specially $NUPIC and $NTA) with default variables directly into .bashrc >>>>> file. However as mentioned by Subutai, this might generate confusion in >>>>> machines that need customized configuration like those that use Grok or >>>>> others. >>>>> >>>>> Fortunatelly, Utensil drawn a scenario of our current situation which >>>>> explain some conflicting issues and how we could mitigate.. A quote: >>>>> >>>>> > There're two kinds of users: >>>>> >* Experienced nupic developers like @scottpurdy , they have been >>>>> using `$NUPIC` and `$NTA` for a long time and they preset them manually in >>>>> `.bashrc` or somewhere, and they need `$NTA` to point to somewhere other >>>>> than `$NUPIC/build/release` >>>>> > * New nupic users or developer like you and I, we intend to keep the >>>>> env var overhead minimal. >>>>> >>>>> From my understanding, I think that an ideal process it would be that >>>>> allows we would have 2 scenarios working fine: >>>>> >>>>> Scenario # 1: >>>>> >>>>> I'm a developer using NuPIC for simple software: I simply clone the >>>>> repo, run CMake, and voilá! >>>>> >>>>> Scenario # 2: >>>>> >>>>> I'm a developer using NuPIC for critical and customized software: I >>>>> simply clone the repo, *config $NUPIC and $NTA using .bashrc (or >>>>> other method) to the target locations*, run CMake and voilá! >>>>> >>>>> I really like this development workflow as this reduces steps of >>>>> configuration at same time that allows flexibility to those users that >>>>> need >>>>> customization.. >>>>> >>>>> In this link you'll find further information which includes my >>>>> previous suggestion, Utensil commentary, and my plan of how the current >>>>> code could address these 2 scenarios: >>>>> >>>>> https://github.com/numenta/nupic/pull/801 >>>>> >>>>> -- >>>>> David Ragazzi >>>>> OS Community Commiter >>>>> Numenta.org >>>>> -- >>>>> "I think James Connolly, the Irish revolutionary, is right when he >>>>> says that the only prophets are those who make their future. So we're >>>>> not anticipating, we're working for it." >>>>> >>>>> _______________________________________________ >>>>> nupic mailing list >>>>> [email protected] >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>> >>> >>> -- >>> David Ragazzi >>> OS Community Commiter >>> Numenta.org >>> -- >>> "I think James Connolly, the Irish revolutionary, is right when he says that >>> the only prophets are those who make their future. So we're not >>> anticipating, we're working for it." >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > > -- > David Ragazzi > OS Community Commiter > Numenta.org > -- > "I think James Connolly, the Irish revolutionary, is right when he says that > the only prophets are those who make their future. So we're not > anticipating, we're working for it." > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
