Re: Workflow for editing the subversion website

[Johan Corveleyn]

On Thu, Oct 19, 2017 at 11:58 PM, Johan Corveleyn  wrote:
> On Thu, Oct 19, 2017 at 10:04 PM, Johan Corveleyn  wrote:
>> On Thu, Oct 5, 2017 at 11:42 PM, Branko Čibej  wrote:
>>> On 05.10.2017 22:36, Johan Corveleyn wrote:
 On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf  
 wrote:
> Devil's advocate hat on, and in light of Brane's sibling reply, let me
> describe how an svnmucc workflow might work.
 Thanks, but I prefer the merge workflow. It seems more natural to me,
 and I think it's more likely to be used by other svn users out there,
 in case they have such a workflow. So it seems like the more
 interesting dog food to me :-).

 I'm not very good at writing down an accurate procedure, but I still
 think it should be something like I wrote in my first mail in this
 thread:

> 1) Commit to staging. Other devs get the commit mail via the
> notifications@ list.
>
> 2) Wait for others to review (the commit mail is the notification that
> something needs to be reviewed). In case of large or sensitive
> changes, preferably send a mail to dev@ to draw extra attention.
>
> 3) If any other committer says +1, go ahead and "promote" (merge) to
> the live site.
>
> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
> live site (lazy consensus).
 As Brane suggested, let's do everything in this direction (test on
 staging first, then merge to publish), except for security
 announcements.

 And as Daniel suggested, let's serve the staging site via
 https://subversion-staging.apache.org/ (I'd say we ask infra to set
 this up for us).
>>>
>>> Sounds like a plan.
>>>
>>> -- Brane
>>
>> Sorry, I dropped this on the floor trying to catch some other things.
>>
>> I'll try to pick up the pieces now :-) ...
>>
>> svn cp $URL/site/publish $URL/site/staging
>> open infra ticket for serving /site/staging on subversion-staging.a.o
>> put a short description of our desired workflow in /site/README
>
> Done. The infra ticket is here:
> https://issues.apache.org/jira/browse/INFRA-15323
>
> The description of the workflow in site/README is more or less
> copy-pasted from this mail thread. Can probably be improved ...
> (anyone: feel free). While writing it down, I realized that this
> "promote from staging to publish with complete merges" isn't
> parallellizable (if two committers want to make two distinct changes,
> and not merge all of it in one go to publish). Oh well, it's a start.

It took a while for subversion-staging to be set up correctly (with
server-side includes etc), but now it's there:
http://subversion-staging.apache.org/

So we can now fully utilize the "edit website via staging" workflow as
described in http://svn.apache.org/repos/asf/subversion/site/README

-- 
Johan

Re: Workflow for editing the subversion website

[Johan Corveleyn]

On Thu, Oct 19, 2017 at 10:04 PM, Johan Corveleyn  wrote:
> On Thu, Oct 5, 2017 at 11:42 PM, Branko Čibej  wrote:
>> On 05.10.2017 22:36, Johan Corveleyn wrote:
>>> On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf  
>>> wrote:
 Devil's advocate hat on, and in light of Brane's sibling reply, let me
 describe how an svnmucc workflow might work.
>>> Thanks, but I prefer the merge workflow. It seems more natural to me,
>>> and I think it's more likely to be used by other svn users out there,
>>> in case they have such a workflow. So it seems like the more
>>> interesting dog food to me :-).
>>>
>>> I'm not very good at writing down an accurate procedure, but I still
>>> think it should be something like I wrote in my first mail in this
>>> thread:
>>>
 1) Commit to staging. Other devs get the commit mail via the
 notifications@ list.

 2) Wait for others to review (the commit mail is the notification that
 something needs to be reviewed). In case of large or sensitive
 changes, preferably send a mail to dev@ to draw extra attention.

 3) If any other committer says +1, go ahead and "promote" (merge) to
 the live site.

 4) If no response after 1 week? 3 days? ...? go ahead and promote to
 live site (lazy consensus).
>>> As Brane suggested, let's do everything in this direction (test on
>>> staging first, then merge to publish), except for security
>>> announcements.
>>>
>>> And as Daniel suggested, let's serve the staging site via
>>> https://subversion-staging.apache.org/ (I'd say we ask infra to set
>>> this up for us).
>>
>> Sounds like a plan.
>>
>> -- Brane
>
> Sorry, I dropped this on the floor trying to catch some other things.
>
> I'll try to pick up the pieces now :-) ...
>
> svn cp $URL/site/publish $URL/site/staging
> open infra ticket for serving /site/staging on subversion-staging.a.o
> put a short description of our desired workflow in /site/README

Done. The infra ticket is here:
https://issues.apache.org/jira/browse/INFRA-15323

The description of the workflow in site/README is more or less
copy-pasted from this mail thread. Can probably be improved ...
(anyone: feel free). While writing it down, I realized that this
"promote from staging to publish with complete merges" isn't
parallellizable (if two committers want to make two distinct changes,
and not merge all of it in one go to publish). Oh well, it's a start.

-- 
Johan

Re: Workflow for editing the subversion website

[Johan Corveleyn]

On Thu, Oct 5, 2017 at 11:42 PM, Branko Čibej  wrote:
> On 05.10.2017 22:36, Johan Corveleyn wrote:
>> On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf  
>> wrote:
>>> Devil's advocate hat on, and in light of Brane's sibling reply, let me
>>> describe how an svnmucc workflow might work.
>> Thanks, but I prefer the merge workflow. It seems more natural to me,
>> and I think it's more likely to be used by other svn users out there,
>> in case they have such a workflow. So it seems like the more
>> interesting dog food to me :-).
>>
>> I'm not very good at writing down an accurate procedure, but I still
>> think it should be something like I wrote in my first mail in this
>> thread:
>>
>>> 1) Commit to staging. Other devs get the commit mail via the
>>> notifications@ list.
>>>
>>> 2) Wait for others to review (the commit mail is the notification that
>>> something needs to be reviewed). In case of large or sensitive
>>> changes, preferably send a mail to dev@ to draw extra attention.
>>>
>>> 3) If any other committer says +1, go ahead and "promote" (merge) to
>>> the live site.
>>>
>>> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
>>> live site (lazy consensus).
>> As Brane suggested, let's do everything in this direction (test on
>> staging first, then merge to publish), except for security
>> announcements.
>>
>> And as Daniel suggested, let's serve the staging site via
>> https://subversion-staging.apache.org/ (I'd say we ask infra to set
>> this up for us).
>
> Sounds like a plan.
>
> -- Brane

Sorry, I dropped this on the floor trying to catch some other things.

I'll try to pick up the pieces now :-) ...

svn cp $URL/site/publish $URL/site/staging
open infra ticket for serving /site/staging on subversion-staging.a.o
put a short description of our desired workflow in /site/README

-- 
Johan

Re: Workflow for editing the subversion website

[Branko Čibej]

On 05.10.2017 22:36, Johan Corveleyn wrote:
> On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf  
> wrote:
>> Devil's advocate hat on, and in light of Brane's sibling reply, let me
>> describe how an svnmucc workflow might work.
> Thanks, but I prefer the merge workflow. It seems more natural to me,
> and I think it's more likely to be used by other svn users out there,
> in case they have such a workflow. So it seems like the more
> interesting dog food to me :-).
>
> I'm not very good at writing down an accurate procedure, but I still
> think it should be something like I wrote in my first mail in this
> thread:
>
>> 1) Commit to staging. Other devs get the commit mail via the
>> notifications@ list.
>>
>> 2) Wait for others to review (the commit mail is the notification that
>> something needs to be reviewed). In case of large or sensitive
>> changes, preferably send a mail to dev@ to draw extra attention.
>>
>> 3) If any other committer says +1, go ahead and "promote" (merge) to
>> the live site.
>>
>> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
>> live site (lazy consensus).
> As Brane suggested, let's do everything in this direction (test on
> staging first, then merge to publish), except for security
> announcements.
>
> And as Daniel suggested, let's serve the staging site via
> https://subversion-staging.apache.org/ (I'd say we ask infra to set
> this up for us).

Sounds like a plan.

-- Brane

Re: Workflow for editing the subversion website

[Johan Corveleyn]

On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf  wrote:
> Devil's advocate hat on, and in light of Brane's sibling reply, let me
> describe how an svnmucc workflow might work.

Thanks, but I prefer the merge workflow. It seems more natural to me,
and I think it's more likely to be used by other svn users out there,
in case they have such a workflow. So it seems like the more
interesting dog food to me :-).

I'm not very good at writing down an accurate procedure, but I still
think it should be something like I wrote in my first mail in this
thread:

> 1) Commit to staging. Other devs get the commit mail via the
> notifications@ list.
>
> 2) Wait for others to review (the commit mail is the notification that
> something needs to be reviewed). In case of large or sensitive
> changes, preferably send a mail to dev@ to draw extra attention.
>
> 3) If any other committer says +1, go ahead and "promote" (merge) to
> the live site.
>
> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
> live site (lazy consensus).

As Brane suggested, let's do everything in this direction (test on
staging first, then merge to publish), except for security
announcements.

And as Daniel suggested, let's serve the staging site via
https://subversion-staging.apache.org/ (I'd say we ask infra to set
this up for us).

-- 
Johan

Re: Workflow for editing the subversion website

[Daniel Shahaf]

Devil's advocate hat on, and in light of Brane's sibling reply, let me
describe how an svnmucc workflow might work.

Johan Corveleyn wrote on Thu, 05 Oct 2017 01:13 +0200:
> - Maybe we'll have some change on staging that we don't want to merge
> to publish (I mean, something like showing a different header or
> "staging" watermark, to indicate to users that it's not the production
> site). That should be easy: we just 'merge --record-only' that commit
> to publish.

With svnmucc, we'll have two options: either revert the change on
staging —

   % cd site/staging/
   % svn merge -c -42
   % svn commit

— or re-create staging from current publish:

   % svnmucc rm staging cp HEAD publish staging

> - Maybe we'll do some changes directly on 'publish' (intentionally, to
> be quick and efficient; or accidentally). Can we just merge those to
> 'staging', and expect this not to be a problem when merging staging
> back to publish later? Where does svn stand currently with holding two
> branches in sync, while merges can go in either direction?

To backport from publish to staging, we could either a run cherry-pick
merge, or run the above 'svnmucc rm cp' command.  The former would be
preferred if we have other pending changes on staging/ at the time of
the direct commit to publish/.  The latter would be more efficient in
terms of developer time.

>

Cheers,

Daniel

P.S. You may have noticed that the use of HEAD in the svnmucc command
introduces a race condition: what if Alice makes a commit to staging
(creating r42) exactly at the time Bob replaces staging by publish@HEAD
(creating r43, which replaces staging/ by a copy of publish@42), meaning
Alice's commit was immediately reverted by Bob's.  If that happens, then
commit notifications will betray it to us, and we'll run 'svn merge -c
42 staging@42', thereby dog-fooding the peg-revision codepaths of 'svn
merge' ☺.

P.P.S. Alternatively, Bob could pass "-r 41" to svnmucc, which would
cause his commit command to fail with a conflict.

Re: Workflow for editing the subversion website

[Branko Čibej]

On 05.10.2017 01:13, Johan Corveleyn wrote:
> On Wed, Oct 4, 2017 at 1:21 PM, Daniel Shahaf  wrote:
>> Bert Huijben wrote on Wed, 04 Oct 2017 11:07 +0200:
 -Original Message-
 From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]

 I'd like to understand the topology / flow of changes: what ensures that
 changes made directly to publish are not reverted by a subsequent
 promotion of staging?

 FWIW, in the Apache CMS, a "publish" operation uses 'svnmucc rm publish
 cp N staging publish', so it's an O(1) operation, but it literally 
 overwrites any
 changes that may have been made directly to publish/.  (I'm glossing over a
 detail but that's the gist)
>>> I think we should just use svn merge, to avoid these problems? No CMS here.
>> For clarity, I'm not proposing a move to the CMS; I'm simply pointing out 
>> that
>> the physical way by which commits port from staging to publish, or from
>> publish to staging, may be either 'svn merge + svn commit' or that 'svnmucc'
>> replace operation.
> Yes, I think we should go for 'svn merge + svn commit'. Committing new
> stuff to 'staging', and merging (with complete merges, not
> cherry-picks) to 'publish'. It would be good for us, as a project, to
> use this sort of workflow in practice, as I think it's a useful
> workflow that's used by some Subversion users. So in the interest of
> eating our own dogfood ...
>
> I'm just not sure if this will always work out perfectly.
>
> - Maybe we'll have some change on staging that we don't want to merge
> to publish (I mean, something like showing a different header or
> "staging" watermark, to indicate to users that it's not the production
> site). That should be easy: we just 'merge --record-only' that commit
> to publish.

Or merge and revert. We do that all the time with BRANCH-README files.

> - Maybe we'll do some changes directly on 'publish' (intentionally, to
> be quick and efficient; or accidentally). Can we just merge those to
> 'staging', and expect this not to be a problem when merging staging
> back to publish later? Where does svn stand currently with holding two
> branches in sync, while merges can go in either direction?

We do that all the time with regular development branches, too; sync
from trunk, then merge to trunk.

-- Brane

Re: Workflow for editing the subversion website

[Johan Corveleyn]

On Wed, Oct 4, 2017 at 1:21 PM, Daniel Shahaf  wrote:
> Bert Huijben wrote on Wed, 04 Oct 2017 11:07 +0200:
>> > -Original Message-
>> > From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]
>> >
>> > I'd like to understand the topology / flow of changes: what ensures that
>> > changes made directly to publish are not reverted by a subsequent
>> > promotion of staging?
>> >
>> > FWIW, in the Apache CMS, a "publish" operation uses 'svnmucc rm publish
>> > cp N staging publish', so it's an O(1) operation, but it literally 
>> > overwrites any
>> > changes that may have been made directly to publish/.  (I'm glossing over a
>> > detail but that's the gist)
>>
>> I think we should just use svn merge, to avoid these problems? No CMS here.
>
> For clarity, I'm not proposing a move to the CMS; I'm simply pointing out that
> the physical way by which commits port from staging to publish, or from
> publish to staging, may be either 'svn merge + svn commit' or that 'svnmucc'
> replace operation.

Yes, I think we should go for 'svn merge + svn commit'. Committing new
stuff to 'staging', and merging (with complete merges, not
cherry-picks) to 'publish'. It would be good for us, as a project, to
use this sort of workflow in practice, as I think it's a useful
workflow that's used by some Subversion users. So in the interest of
eating our own dogfood ...

I'm just not sure if this will always work out perfectly.

- Maybe we'll have some change on staging that we don't want to merge
to publish (I mean, something like showing a different header or
"staging" watermark, to indicate to users that it's not the production
site). That should be easy: we just 'merge --record-only' that commit
to publish.

- Maybe we'll do some changes directly on 'publish' (intentionally, to
be quick and efficient; or accidentally). Can we just merge those to
'staging', and expect this not to be a problem when merging staging
back to publish later? Where does svn stand currently with holding two
branches in sync, while merges can go in either direction?

-- 
Johan

RE: Workflow for editing the subversion website

[Bert Huijben]

> -Original Message-
> From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]
> Sent: woensdag 4 oktober 2017 02:05
> To: dev@subversion.apache.org
> Subject: Re: Workflow for editing the subversion website
> 
> Johan Corveleyn wrote on Tue, 03 Oct 2017 23:32 +0200:
> > The recent work on our quick-start.html page by Pavel Lyalyakin
> > prompted some thinking about how to better organize our site editing.
> > Pavel asked about lightweight branching and Daniel suggested to copy
> > site/publish to site/staging and having it served as
> > http://subversion.staging.apache.org/ to facilitate previewing [1].
> >
> 
> Small technical note: *.staging.apache.org is what the CMS uses, also it will
> cause SSL errors since the *.apache.org certificate won't match that
> hostname.  (Compare svn-eu.apache.org/svn.eu.apache.org: asterisk in the
> cert doesn't match dot in the URL's hostname)  We can get around both
> problems by having the staging preview site live on, say, https://subversion-
> staging.apache.org/ (or even on svn-qavm).
> 
> > Small changes and corrections can go directly to the live site. Maybe
> > we'll need some exceptions for things like news, release notes and
> > security pages, which are usually updated as part of releases and get
> > a lot of eyes already.
> >
> > Thoughts?
> 
> I'd like to understand the topology / flow of changes: what ensures that
> changes made directly to publish are not reverted by a subsequent
> promotion of staging?
> 
> FWIW, in the Apache CMS, a "publish" operation uses 'svnmucc rm publish
> cp N staging publish', so it's an O(1) operation, but it literally overwrites 
> any
> changes that may have been made directly to publish/.  (I'm glossing over a
> detail but that's the gist)

I think we should just use svn merge, to avoid these problems? No CMS here.

Bert
> 
> Cheers,
> 
> Daniel