Re: [sage-devel] Re: Checking out old branches

[Robert Bradshaw]

On Sat, Nov 19, 2016 at 12:40 AM, Jeroen Demeyer  wrote:
> On 2016-11-18 20:48, Robert Bradshaw wrote:
>>
>> Hopefully with a good ccache/cycache rebuilding from a fresh clone
>> should be fast too.
>
>
> Speaking of cycache, is there any chance that the following will be fixed
> some day?
>
> https://github.com/cython/cython/issues/1393
> https://github.com/cython/cython/issues/1397
> https://github.com/cython/cython/issues/1399

https://help.github.com/articles/creating-a-pull-request/ :)

Yes, I'd like to get to these, among other things.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Checking out old branches

[Jeroen Demeyer]

On 2016-11-18 20:48, Robert Bradshaw wrote:

Hopefully with a good ccache/cycache rebuilding from a fresh clone
should be fast too.


Speaking of cycache, is there any chance that the following will be 
fixed some day?


https://github.com/cython/cython/issues/1393
https://github.com/cython/cython/issues/1397
https://github.com/cython/cython/issues/1399

--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Checking out old branches

[Robert Bradshaw]

On Fri, Nov 18, 2016 at 12:39 AM, David Roe  wrote:
>
> On Fri, Nov 18, 2016 at 2:45 AM, Volker Braun  wrote:
>>
>> On Friday, November 18, 2016 at 8:12:32 AM UTC+1, David Roe wrote:
>>>
>>> Create a new git trac subcommand to replace `git trac checkout 1234`, say
>>> `git trac old 1234`.  This would fetch the branch, check it out into a
>>> completely separate folder within ($SAGE_ROOT/merge_tree or something),
>>> merge in develop.  If the merge is successful, create a new branch and pull
>>> the changes in.  This ends up with only a few files changing if you started
>>> at develop.  If the merge is not successful, report to the user and ask them
>>> to fix the merge in
>>> $SAGE_ROOT/merge_tree.  There would then be some way to resume and pull
>>> in the changes.
>>>
>>> There are some details to fill in, but I think that an approach like this
>>> can work.  It does mean having another 100MB working tree floating around
>>> just for merging into, and also stepping a bit further away from normal git
>>> practices.
>>>
>>> Any other ideas?
>>
>>
>> I'd just clone the repo to a separate directory, nothing except the
>> standard git commands necessary.

+1

Hopefully with a good ccache/cycache rebuilding from a fresh clone
should be fast too.

> Yeah, just cloning is probably better.  I was thinking for a bit that we
> could avoid the duplicated space of having a separate .git folder, but I
> don't think it's worth the added complications.
>
> Certainly, the ideal solution is to fix Cython so that the timestamp changes
> don't cause trouble, either by fixing the problems with cycache or by using
> hashes as William suggests.  But that's not going to happen quickly, and I'd
> like to have a workaround.  A new git trac command is opt-in and fairly easy
> to implement.  Volker, does this sound like a reasonable pull request for
> git-trac-command?

When you do a git checkout, it only changes the timestamps of changed
files. When you do a cython build, it skips the build if the .c file
is newer than the .pyx file *and all its dependencies*. When switching
between versions of Sage, you are likely to touch dependencies for a
large portion of the library. In other words, I think it's still
rebuilding as little as it can, but that happens to be a lot.
(Changing to a hash wouldn't help here--though it would add complexity
as you'd have to store the has of all dependencies somewhere to
compare against.)

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Checking out old branches

[Clemens Heuberger]

On 2016-11-18 10:05, David Roe wrote:
> 
> 
> On Fri, Nov 18, 2016 at 3:49 AM, Volker Braun  > wrote:
> 
> The .git dir doesn't have to be in the working tree:
> 
> git --git-dir= --work-tree=
> 
> But I think its very confusing to have another working tree inside the
> working tree, thats rather confusing and error prone. E.g. what does "git
> commit" do in that situation? IMHO its conceptually much easier to just 
> do a
> separate clone of the repo. And its completely standard git commands.
> 
> 
> I agree.  But I think we should still automate the creation and use of this
> separate clone.  Where should this clone go?  Ask the user when they first 
> call it?
> David

I suggest using "git worktree". This allows to have two working copies sharing
the same .git directory and helps with situations like that.

Clemens

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Checking out old branches

[David Roe]

On Fri, Nov 18, 2016 at 3:49 AM, Volker Braun  wrote:

> The .git dir doesn't have to be in the working tree:
>
> git --git-dir= --work-tree=
>
> But I think its very confusing to have another working tree inside the
> working tree, thats rather confusing and error prone. E.g. what does "git
> commit" do in that situation? IMHO its conceptually much easier to just do
> a separate clone of the repo. And its completely standard git commands.
>

I agree.  But I think we should still automate the creation and use of this
separate clone.  Where should this clone go?  Ask the user when they first
call it?
David

>
>
>
>
> On Friday, November 18, 2016 at 9:40:15 AM UTC+1, David Roe wrote:
>>
>>
>>
>> On Fri, Nov 18, 2016 at 2:45 AM, Volker Braun  wrote:
>>
>>> On Friday, November 18, 2016 at 8:12:32 AM UTC+1, David Roe wrote:

 Create a new git trac subcommand to replace `git trac checkout 1234`,
 say `git trac old 1234`.  This would fetch the branch, check it out into a
 completely separate folder within ($SAGE_ROOT/merge_tree or
 something), merge in develop.  If the merge is successful, create a new
 branch and pull the changes in.  This ends up with only a few files
 changing if you started at develop.  If the merge is not successful, report
 to the user and ask them to fix the merge in
 $SAGE_ROOT/merge_tree.  There would then be some way to resume and
 pull in the changes.

 There are some details to fill in, but I think that an approach like
 this can work.  It does mean having another 100MB working tree floating
 around just for merging into, and also stepping a bit further away from
 normal git practices.

 Any other ideas?

>>>
>>> I'd just clone the repo to a separate directory, nothing except the
>>> standard git commands necessary.
>>>
>>
>> Yeah, just cloning is probably better.  I was thinking for a bit that we
>> could avoid the duplicated space of having a separate .git folder, but I
>> don't think it's worth the added complications.
>>
>> Certainly, the ideal solution is to fix Cython so that the timestamp
>> changes don't cause trouble, either by fixing the problems with cycache or
>> by using hashes as William suggests.  But that's not going to happen
>> quickly, and I'd like to have a workaround.  A new git trac command is
>> opt-in and fairly easy to implement.  Volker, does this sound like a
>> reasonable pull request for git-trac-command?
>> David
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Checking out old branches

[Volker Braun]

The .git dir doesn't have to be in the working tree:

git --git-dir= --work-tree=

But I think its very confusing to have another working tree inside the 
working tree, thats rather confusing and error prone. E.g. what does "git 
commit" do in that situation? IMHO its conceptually much easier to just do 
a separate clone of the repo. And its completely standard git commands.




On Friday, November 18, 2016 at 9:40:15 AM UTC+1, David Roe wrote:
>
>
>
> On Fri, Nov 18, 2016 at 2:45 AM, Volker Braun  > wrote:
>
>> On Friday, November 18, 2016 at 8:12:32 AM UTC+1, David Roe wrote:
>>>
>>> Create a new git trac subcommand to replace `git trac checkout 1234`, 
>>> say `git trac old 1234`.  This would fetch the branch, check it out into a 
>>> completely separate folder within ($SAGE_ROOT/merge_tree or something), 
>>> merge in develop.  If the merge is successful, create a new branch and pull 
>>> the changes in.  This ends up with only a few files changing if you started 
>>> at develop.  If the merge is not successful, report to the user and ask 
>>> them to fix the merge in 
>>> $SAGE_ROOT/merge_tree.  There would then be some way to resume and pull 
>>> in the changes.
>>>
>>> There are some details to fill in, but I think that an approach like 
>>> this can work.  It does mean having another 100MB working tree floating 
>>> around just for merging into, and also stepping a bit further away from 
>>> normal git practices.
>>>
>>> Any other ideas?
>>>
>>
>> I'd just clone the repo to a separate directory, nothing except the 
>> standard git commands necessary. 
>>
>
> Yeah, just cloning is probably better.  I was thinking for a bit that we 
> could avoid the duplicated space of having a separate .git folder, but I 
> don't think it's worth the added complications.
>
> Certainly, the ideal solution is to fix Cython so that the timestamp 
> changes don't cause trouble, either by fixing the problems with cycache or 
> by using hashes as William suggests.  But that's not going to happen 
> quickly, and I'd like to have a workaround.  A new git trac command is 
> opt-in and fairly easy to implement.  Volker, does this sound like a 
> reasonable pull request for git-trac-command?
> David
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Checking out old branches

[David Roe]

On Fri, Nov 18, 2016 at 2:45 AM, Volker Braun  wrote:

> On Friday, November 18, 2016 at 8:12:32 AM UTC+1, David Roe wrote:
>>
>> Create a new git trac subcommand to replace `git trac checkout 1234`, say
>> `git trac old 1234`.  This would fetch the branch, check it out into a
>> completely separate folder within ($SAGE_ROOT/merge_tree or something),
>> merge in develop.  If the merge is successful, create a new branch and pull
>> the changes in.  This ends up with only a few files changing if you started
>> at develop.  If the merge is not successful, report to the user and ask
>> them to fix the merge in
>> $SAGE_ROOT/merge_tree.  There would then be some way to resume and pull
>> in the changes.
>>
>> There are some details to fill in, but I think that an approach like this
>> can work.  It does mean having another 100MB working tree floating around
>> just for merging into, and also stepping a bit further away from normal git
>> practices.
>>
>> Any other ideas?
>>
>
> I'd just clone the repo to a separate directory, nothing except the
> standard git commands necessary.
>

Yeah, just cloning is probably better.  I was thinking for a bit that we
could avoid the duplicated space of having a separate .git folder, but I
don't think it's worth the added complications.

Certainly, the ideal solution is to fix Cython so that the timestamp
changes don't cause trouble, either by fixing the problems with cycache or
by using hashes as William suggests.  But that's not going to happen
quickly, and I'd like to have a workaround.  A new git trac command is
opt-in and fairly easy to implement.  Volker, does this sound like a
reasonable pull request for git-trac-command?
David

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Checking out old branches

[Volker Braun]

On Friday, November 18, 2016 at 8:12:32 AM UTC+1, David Roe wrote:
>
> Create a new git trac subcommand to replace `git trac checkout 1234`, say 
> `git trac old 1234`.  This would fetch the branch, check it out into a 
> completely separate folder within ($SAGE_ROOT/merge_tree or something), 
> merge in develop.  If the merge is successful, create a new branch and pull 
> the changes in.  This ends up with only a few files changing if you started 
> at develop.  If the merge is not successful, report to the user and ask 
> them to fix the merge in 
> $SAGE_ROOT/merge_tree.  There would then be some way to resume and pull 
> in the changes.
>
> There are some details to fill in, but I think that an approach like this 
> can work.  It does mean having another 100MB working tree floating around 
> just for merging into, and also stepping a bit further away from normal git 
> practices.
>
> Any other ideas?
>

I'd just clone the repo to a separate directory, nothing except the 
standard git commands necessary. 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Checking out old branches

[Jean-Pierre Flori]

On Friday, November 18, 2016 at 8:12:32 AM UTC+1, David Roe wrote:
>
> If I checkout an old branch (say, from one or two versions of Sage ago), 
> it essentially forces a rebuild of all of Sage, even if I think better of 
> it and checkout develop immediately.  The rebuild is a consequence of the 
> fact that Cython builds based on timestamp and all of the files have been 
> touched.
>
> I'd like to solicit ideas to make this less painful.  Here's one, based on 
> a conversation with Julian Rüth.
>
> Create a new git trac subcommand to replace `git trac checkout 1234`, say 
> `git trac old 1234`.  This would fetch the branch, check it out into a 
> completely separate folder within ($SAGE_ROOT/merge_tree or something), 
> merge in develop.  If the merge is successful, create a new branch and pull 
> the changes in.  This ends up with only a few files changing if you started 
> at develop.  If the merge is not successful, report to the user and ask 
> them to fix the merge in 
> $SAGE_ROOT/merge_tree.  There would then be some way to resume and pull 
> in the changes.
>
> There are some details to fill in, but I think that an approach like this 
> can work.  It does mean having another 100MB working tree floating around 
> just for merging into, and also stepping a bit further away from normal git 
> practices.
>
> Any other ideas?
> David
>
Fix cycache ? :p
It seems there is a related discussion here:
https://trac.sagemath.org/ticket/17851 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.