Re: [Scons-dev] stubprocess ?

[Dirk Bächle]

Hi Vasily,

On 23.12.2015 22:04, Vasily wrote:

Hi Bill and all,
Since we (Eugene - added, and me) were original authors, maybe we should work 
on making a pull request.
Can you share the requirements for successful pull request?

P.S. I think that making an env opt-out key is hard to do since it 
monkey-patches subprocess module globally, but making an
command-line option might be a good idea.



thanks for offering your help with this. I agree that we shouldn't care about an option for switching the stubprocess wrapper off. 
As far as I remember we discussed this some time ago, and the consensus was to always activate the wrapper under "posix"-like systems.


One remark though: Can we possibly rename the patched subprocess call to a different module? For our purposes (in SCons) we don't 
need a global replacement, but just a method that we can stuff into our env["SPAWN"] variable which gets used for every command-line 
action that has to get executed.

What I have observed is, that the global replacement of "subprocess" gives 
trouble when custom Builders or SConscripts do an
"import subprocess" again afterwards. How do we protect against this? If possible I'd like to have the wrapper decoupled from the 
"subprocess" module...


For the pull request, please fork the current "trunk" from the main repo as 
described at

  https://bitbucket.org/scons/scons/wiki/DeveloperGuide/Introduction

. The section "Fixing and developing the core sources" should get you going...if you have any questions, feel free to ask please. 
From my point of view, you won't have to provide any additional tests or documentation. We just have to ensure that all the current 
tests still work afterwards...and then there's one more detail: Licensing and copyrights. I'm not sure what the state of the 
stubprocess wrapper is, regarding copyright. I assume that it was (at least partly) developed during your work time for Intel, so we 
(SCons foundation) should get a document (email might be enough?, but I'm not a lawyer) allowing us to distribute the wrapper 
together with our sources, under the same MIT license (we'd mention Intel and your names in the copyright, can you make a proposal 
for the exact text?). I'd really like to have this in writing somehow...


That's what I can think of for the moment...happy hackin'! ;)

Best regards,

Dirk

___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] That was easy…

[Jason Kenny]

I agree, we can't get rid of src local packages.

If we start adding depends on third party packages, the current src local 
packages logic would break. I suggested that we should make some small tweaks 
to address this

Jason

From: Alexandre Feblot
Sent: ‎12/‎27/‎2015 1:06 PM
To: SCons developer list
Subject: Re: [Scons-dev] That was easy…

Hi,
If SCons should move to using pip to be installed, I do hope there would still 
be as well a simple archive delivery.
We try to keep our servers with as little installed softwares/packages as 
possible, and adding the need to take care of having a working pip on every 
linux/aix/solaris/windows server would be an additional hassle.
Also, I suppose pip would make life harder for projects that need to compile on 
servers with no internet access.



> Le 27 déc. 2015 à 18:02, Jason Kenny  a écrit :
>
> In the current form yes.
>
> We can use features in pip to get the packages and install them in a 
> directory contained within the src-local area of scons and modify the startup 
> script to add a python path to the packages. This can be zipped up and used 
> to solve this problem. I did something like this for one of my last projects 
> at Intel.
>
> Jason
>
> From: Bill Deegan 
> Sent: Saturday, December 26, 2015 9:23 PM
> To: SCons developer list 
> Subject: Re: [Scons-dev] That was easy…
>
> The only question is would this break the scons-local distributions?
>
> On Sat, Dec 26, 2015 at 6:13 PM, Jason Kenny  > wrote:
> +1
>
> Sent from my Windows Phone
>
> -Original Message-
> From: "Bill Deegan"  >
> Sent: ‎12/‎26/‎2015 7:00 PM
> To: "SCons developer list" >
> Subject: Re: [Scons-dev] That was easy…
>
> If we change to suggesting pip/easy_install to be the preferred method of 
> installing, then we can include any other packages we need and not have to 
> "Vendorize" them.
>
>
>
> On Sat, Dec 26, 2015 at 9:19 AM, Russel Winder  > wrote:
>
> On Fri, 2015-12-25 at 13:15 -0500, Neal Becker wrote:
> >
> […]
> >
> > I should have said, I don't think we can make installing future a
> > requirement for using scons.
> >
> > In my case, I was using lineprofiler.  It does something strange
> > with
> > builtins, and the reason lineprofiler mysteriously stopped working
> > after
> > future was installed was hard to find.
>
> So the consequence of this is that we can only go to a level 1 futurize
> and then the rest needs to be manually hacked so that there is no
> future dependency? I suspect this means we will end up rewriting quite
> a chunk of future, though not the bits needed for Python 2.6.
>
>
> --
> Russel.
> =
> Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net 
> 
> 41 Buckmaster Roadm: +44 7770 465 077    
> xmpp: rus...@winder.org.uk 
> London SW11 1EN, UK   w: www.russel.org.uk   
> skype: russel_winder
>
>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org 
> https://pairlist2.pair.net/mailman/listinfo/scons-dev 
> 
> ___
> Scons-dev mailing list
> Scons-dev@scons.org 
> https://pairlist2.pair.net/mailman/listinfo/scons-dev 
> 
>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org 
> https://pairlist2.pair.net/mailman/listinfo/scons-dev 
> 
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

[Scons-dev] Updating roadmap in wiki

[Bill Deegan]

Greetings,

I"m doing some updates on the roadmap.
Are there any missing items?
Any items listed which are already completed?

I've deleted/changed a few where plans have changed.

-Bill
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] stubprocess ?

[Jason Kenny]

HI Dirk.

I cannot speak to what Vasily or Eugene can do with tweaking this code as use 
suggest. I however can speak to the some legal stuff here and a technical issue.

First the technically issue. The point of the patch was that we had found that 
the “default” forking was a major slow down to the build on Linux. This became 
a bigger issue when we updated to a newer RHEL setup for the default OS to 
build some products. It was shown that this “stubprocess” as we had modified 
subprocess which was used in Parts override to SPAWN made everything faster on 
any POSIX based OS ( windows was fast already in this case). From a design 
point of view I totally get the idea to make this easier to switch between the 
modified subprocess and the “default” version of subprocess. I think what we 
are missing in SCons for this is a global “set” of Spawn value. The current 
setup as is requires Parts to set SPAWN for every environment we make/clone. I 
believe the IAPAT (based on Gregs N notes) stuff I am trying to implement in 
Parts as the Setting object would help address this problem better than some 
tweak to a default Environment.

Before I was let go from Intel I had finished the work with Parts and SCons 
open source work sharing. Eugene and Vasily are in the clear to share code with 
Parts and should be ok with SCons as long as they are employed at Intel ( They 
can double check Scons part with Serge P). Everything in Parts is safe for 
SCons to take and use. This has always been the case. As this work is derived 
from Parts work, this is already covered. Ideally we could get stuff in Parts 
faster than it would be to get it in to SCons given the “history” of SCons. 
What I did not finish, but looks like I have to set this up as with my employ 
at Yahoo is setting up a developer attestation, or code release form. Basically 
saying that any code coming in from a given developer does not have any stolen 
code or IP and that it is free to be used by project. SCons does not have such 
a form. I believe the SCons foundation needs to have this. Which is basically a 
legal blub saying what I stated above the developer can say they agree to in an 
e-mail or a pull request. Given the lack of this there was not a more formal 
way to say this code form Parts is OK to be used in SCons.

I think a more official “code release” process could be done in SCons this 
would help make this easier.

We have the text for Intel in the Parts MIT license. 
I should note the Yahoo and Intel approvers for Open source work are 
communicating to me that they may like a change to the license in the we have a 
contributor list vs the names in MIT license file. Given the holidays the 
details of the proposed changed has note been fully communicated yet. It has 
something to do with being less “messy” in the unlikely case there was some 
legal issue with some person. I can communicate more on that once I get some 
resolution. 

(Honestly I never knew open source project could be so much “extra” work) 

Hope this helps.
Jason

From: Dirk Bächle
Sent: Sunday, December 27, 2015 6:01 AM
To: scons-dev@scons.org
Subject: Re: [Scons-dev] stubprocess ?

Hi Vasily,

On 23.12.2015 22:04, Vasily wrote:
> Hi Bill and all,
> Since we (Eugene - added, and me) were original authors, maybe we should work 
> on making a pull request.
> Can you share the requirements for successful pull request?
>
> P.S. I think that making an env opt-out key is hard to do since it 
> monkey-patches subprocess module globally, but making an
> command-line option might be a good idea.
>

thanks for offering your help with this. I agree that we shouldn't care about 
an option for switching the stubprocess wrapper off. 
As far as I remember we discussed this some time ago, and the consensus was to 
always activate the wrapper under "posix"-like systems.

One remark though: Can we possibly rename the patched subprocess call to a 
different module? For our purposes (in SCons) we don't 
need a global replacement, but just a method that we can stuff into our 
env["SPAWN"] variable which gets used for every command-line 
action that has to get executed.
What I have observed is, that the global replacement of "subprocess" gives 
trouble when custom Builders or SConscripts do an
"import subprocess" again afterwards. How do we protect against this? If 
possible I'd like to have the wrapper decoupled from the 
"subprocess" module...

For the pull request, please fork the current "trunk" from the main repo as 
described at

   https://bitbucket.org/scons/scons/wiki/DeveloperGuide/Introduction

. The section "Fixing and developing the core sources" should get you 
going...if you have any questions, feel free to ask please. 
 From my point of view, you won't have to provide any additional tests or 
documentation. We just have to ensure that all the current 
tests still work afterwards...and then there's one more detail: Licensing and 
copyrights. I'm not sure what the state of the 

Re: [Scons-dev] That was easy…

[Alexandre Feblot]

Hi,
If SCons should move to using pip to be installed, I do hope there would still 
be as well a simple archive delivery.
We try to keep our servers with as little installed softwares/packages as 
possible, and adding the need to take care of having a working pip on every 
linux/aix/solaris/windows server would be an additional hassle.
Also, I suppose pip would make life harder for projects that need to compile on 
servers with no internet access.



> Le 27 déc. 2015 à 18:02, Jason Kenny  a écrit :
> 
> In the current form yes.
>  
> We can use features in pip to get the packages and install them in a 
> directory contained within the src-local area of scons and modify the startup 
> script to add a python path to the packages. This can be zipped up and used 
> to solve this problem. I did something like this for one of my last projects 
> at Intel.
>  
> Jason
> 
> From: Bill Deegan 
> Sent: Saturday, December 26, 2015 9:23 PM
> To: SCons developer list 
> Subject: Re: [Scons-dev] That was easy…
>  
> The only question is would this break the scons-local distributions?
>  
> On Sat, Dec 26, 2015 at 6:13 PM, Jason Kenny  > wrote:
> +1
> 
> Sent from my Windows Phone
> 
> -Original Message-
> From: "Bill Deegan"  >
> Sent: ‎12/‎26/‎2015 7:00 PM
> To: "SCons developer list" >
> Subject: Re: [Scons-dev] That was easy…
> 
> If we change to suggesting pip/easy_install to be the preferred method of 
> installing, then we can include any other packages we need and not have to 
> "Vendorize" them.
> 
> 
> 
> On Sat, Dec 26, 2015 at 9:19 AM, Russel Winder  > wrote:
> 
> On Fri, 2015-12-25 at 13:15 -0500, Neal Becker wrote:
> >
> […]
> >
> > I should have said, I don't think we can make installing future a
> > requirement for using scons.
> >
> > In my case, I was using lineprofiler.  It does something strange
> > with
> > builtins, and the reason lineprofiler mysteriously stopped working
> > after
> > future was installed was hard to find.
> 
> So the consequence of this is that we can only go to a level 1 futurize
> and then the rest needs to be manually hacked so that there is no
> future dependency? I suspect this means we will end up rewriting quite
> a chunk of future, though not the bits needed for Python 2.6.
> 
> 
> --
> Russel.
> =
> Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net 
> 
> 41 Buckmaster Roadm: +44 7770 465 077    
> xmpp: rus...@winder.org.uk 
> London SW11 1EN, UK   w: www.russel.org.uk   
> skype: russel_winder
> 
> 
> 
> ___
> Scons-dev mailing list
> Scons-dev@scons.org 
> https://pairlist2.pair.net/mailman/listinfo/scons-dev 
> 
> ___
> Scons-dev mailing list
> Scons-dev@scons.org 
> https://pairlist2.pair.net/mailman/listinfo/scons-dev 
> 
>  
>  
> ___
> Scons-dev mailing list
> Scons-dev@scons.org 
> https://pairlist2.pair.net/mailman/listinfo/scons-dev 
> 
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] New SCons website

[William Blevins]

Bill,

Very good! I think this last revision is great :)

Two nit-picks:
1. I still have the issue that the webpage stays centered at 800px it looks
like (IE. the page doesn't scale to the width of my monitor), but I guess
that's minor.
2. A light sensitivity mode or a change to coloring; it's currently hard on
the eyes with all the white.

V/R,
William

On Sun, Dec 27, 2015 at 10:30 PM, Bill Deegan 
wrote:

> All,
>
> I've made a few more changes.
> Thoughts?
>
> http://scons.org/new/
>
> -Bill
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] New SCons website

[William Blevins]

Bill,

1 other nit-pick: the icon in the upper-left needs to be scaled to match
the recommended menu height.

Functionally, it seems fine.

V/R,
William

On Sun, Dec 27, 2015 at 11:28 PM, William Blevins 
wrote:

> Bill,
>
> Very good! I think this last revision is great :)
>
> Two nit-picks:
> 1. I still have the issue that the webpage stays centered at 800px it
> looks like (IE. the page doesn't scale to the width of my monitor), but I
> guess that's minor.
> 2. A light sensitivity mode or a change to coloring; it's currently hard
> on the eyes with all the white.
>
> V/R,
> William
>
> On Sun, Dec 27, 2015 at 10:30 PM, Bill Deegan 
> wrote:
>
>> All,
>>
>> I've made a few more changes.
>> Thoughts?
>>
>> http://scons.org/new/
>>
>> -Bill
>>
>> ___
>> Scons-dev mailing list
>> Scons-dev@scons.org
>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>
>>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

[Scons-dev] A new architecture...

[Dirk Bächle]

Hi all,

the discussions about new features in SCons during the last weeks, have led me 
to start working on some architectural changes
for SCons. They aim at refactoring the module/package structure, such that it's 
possible to better support customized

- Taskmasters
- File signature calculation (currently fixed to MD5 hash sums)
- Node factories (and connected with that custom Nodes)
- Build and Node Info (and connected with that custom signature DBs, like a 
JSON implementation for easier debugging)

.
Please note that my pull request doesn't directly implement and provide these functionalities (well, some of it), but simply tries 
to lay out the foundation for them. There is definitely more work ahead...

Once we have this in place, it should be much easier to support e.g. additional 
Taskmasters like an

  EspeciallyParanoidTaskmaster

(see Tom Tanner's proposal at 
https://pairlist4.pair.net/pipermail/scons-users/2015-December/004320.html and 
following), or a

  DebuggingTaskmaster

where you get a lot of additional info about the current "machine state", and 
can walk through a complete build in single steps.

I don't know when this PR will be ready, but just wanted to inform you all that 
someone is working on this somewhere. :)
Your comments and suggestions are welcome.


Best regards,

Dirk
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev