Re: Making github.com/apache/sling a git repo, moving away old svn mirror (Re: apache/sling as github landing repository)

[Alexander Klimetschek]

On 27.02.2018, at 22:07, Robert Munteanu  wrote:
> IIUC the proposal was to import the old svn mirror as a git repo, and
> Betrand argued this would be a really large repo, and that's an
> impediment for something which would be heavily cloned.

You are right.

It could be two repos: the old one with all the old content and a new README 
landing page under the name apache/sling, and a separate 
apache/sling-aggregator which gets cloned and is lean. The landing page README 
can point to the aggregator repo.

Cheers,
Alex

Re: Making github.com/apache/sling a git repo, moving away old svn mirror (Re: apache/sling as github landing repository)

[Robert Munteanu]

On Thu, 2018-02-15 at 20:57 +, Alexander Klimetschek wrote:
> On 12.02.2018, at 01:23, Robert Munteanu  wrote:
> > The basic proposal as I see it would be to add a new 'sling' top-
> > level
> > github repo, which means that:
> > 
> > 1. we control what goes in there - README.md most importantly
> > 2. old links to sling commits and files will be broken .
> 
> No! It's absolutely not necessary to break the old links.
> 
> If apache/sling is based on the "old" repo, and you are only adding
> new stuff on the "master" branch (which is new, as it was never used
> in the subversion based Sling), you don't have any conflicts.
> 
> You can have your cake and eat it too in this case - that's what I am
> trying to say all the time :D It's just a matter of working with
> infra to copy things around and replace the github repo accordingly.

IIUC the proposal was to import the old svn mirror as a git repo, and
Betrand argued this would be a really large repo, and that's an
impediment for something which would be heavily cloned.

Robert

Re: Making github.com/apache/sling a git repo, moving away old svn mirror (Re: apache/sling as github landing repository)

[Alexander Klimetschek]

On 12.02.2018, at 01:23, Robert Munteanu  wrote:
> The basic proposal as I see it would be to add a new 'sling' top-level
> github repo, which means that:
> 
> 1. we control what goes in there - README.md most importantly
> 2. old links to sling commits and files will be broken .

No! It's absolutely not necessary to break the old links.

If apache/sling is based on the "old" repo, and you are only adding new stuff 
on the "master" branch (which is new, as it was never used in the subversion 
based Sling), you don't have any conflicts.

You can have your cake and eat it too in this case - that's what I am trying to 
say all the time :D It's just a matter of working with infra to copy things 
around and replace the github repo accordingly.

Cheers,
Alex

Re: Making github.com/apache/sling a git repo, moving away old svn mirror (Re: apache/sling as github landing repository)

[Bertrand Delacretaz]

On Mon, Feb 12, 2018 at 10:23 AM, Robert Munteanu  wrote:
> ...The basic proposal as I see it would be to add a new 'sling' top-level
> github repo, which means that:
>
> 1. we control what goes in there - README.md most importantly
> 2. old links to sling commits and files will be broken . ...

+1

-Bertrand

Making github.com/apache/sling a git repo, moving away old svn mirror (Re: apache/sling as github landing repository)

[Robert Munteanu]

Hi,

I usually am reluctant in revisiting old decisions, especially ones
that were so discussed, like this one. Some summaries of previous
discussion are at [1], [2] . But Alex makes some good points and after
the migration I think we should consider whether backwards
compatibility of incoming links is worth the limited control we have
over github.com/apache/sling, which many times is an entry point for
developers.

The basic proposal as I see it would be to add a new 'sling' top-level
github repo, which means that:

1. we control what goes in there - README.md most importantly
2. old links to sling commits and files will be broken .

TBH, I am not sure #2 is such a bad thing, given that I've seen people
still link to files in the old repo.

Thoughts?

Thanks,

Robert

[1]: https://lists.apache.org/thread.html/9e1ceee36af811ba3d2a0a2baa249
8077281f0d3c1da29747a833ea3@%3Cdev.sling.apache.org%3E
[2]: https://issues.apache.org/jira/browse/SLING-7183

Re: apache/sling as github landing repository

[Robert Munteanu]

Hi Jason,

On Sat, 2018-02-10 at 09:24 -0500, Jason E Bailey wrote:
> And this would be one of those occasions where I thought I knew what
> I was talking about and I was wrong.  What I was referencing was
> something I was led to believe was submodules but is so convoluted
> that I don't want to go there. I can see where submodules would be
> really useful in some situations but from what I've been hearing on
> this conversation thread it's not a solution.

Thanks for coming back to the list, even if the answer is that there is
nothing to include in a POC :-)

For the record, one additional point to consider with the ASF-hosted
infrastructure is that for most cases only actual 'people' can commit,
not bots.

Infra is pretty strong in their stance in making sure that all check-
ins are validated from an IP point of view, and a bot writing in the
git repos, even though technically it does not add source code, would
need pretty strong arguments in the 'risk vs gain' area.

Robert

> 
> - Jason
> 
> On Thu, Feb 8, 2018, at 11:03 AM, Bertrand Delacretaz wrote:
> > On Thu, Feb 8, 2018 at 4:27 PM, Jason E Bailey 
> > wrote:
> > > I'll work on standing something up as a POC so that there is
> > > something tangible...
> > 
> > Great, thanks!
> > 
> > -Bertrand

Re: apache/sling as github landing repository

[Jason E Bailey]

And this would be one of those occasions where I thought I knew what I was 
talking about and I was wrong.  What I was referencing was something I was led 
to believe was submodules but is so convoluted that I don't want to go there. I 
can see where submodules would be really useful in some situations but from 
what I've been hearing on this conversation thread it's not a solution.

- Jason

On Thu, Feb 8, 2018, at 11:03 AM, Bertrand Delacretaz wrote:
> On Thu, Feb 8, 2018 at 4:27 PM, Jason E Bailey  wrote:
> > I'll work on standing something up as a POC so that there is something 
> > tangible...
> 
> Great, thanks!
> 
> -Bertrand

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

On Thu, Feb 8, 2018 at 4:27 PM, Jason E Bailey  wrote:
> I'll work on standing something up as a POC so that there is something 
> tangible...

Great, thanks!

-Bertrand

Re: apache/sling as github landing repository

[Jason E Bailey]

I'll work on standing something up as a POC so that there is something 
tangible. 

- Jason

On Thu, Feb 8, 2018, at 9:29 AM, Bertrand Delacretaz wrote:
> On Thu, Feb 8, 2018 at 2:25 PM, Jason E Bailey  wrote:
> > ...an aggregated view that just reflects the status of submodules...
> 
> I'm +1 on at least experimenting with that if you can do it without
> too much effort.
> 
> -Bertrand

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

On Thu, Feb 8, 2018 at 2:25 PM, Jason E Bailey  wrote:
> ...an aggregated view that just reflects the status of submodules...

I'm +1 on at least experimenting with that if you can do it without
too much effort.

-Bertrand

Re: apache/sling as github landing repository

[Jason E Bailey]

Correct. That's exactly what I was trying to articulate, an aggregated view 
that just reflects the status of submodules.

- Jason

On Thu, Feb 8, 2018, at 4:15 AM, Bertrand Delacretaz wrote:
> Hi,
> 
> On Thu, Feb 8, 2018 at 1:37 AM, Jason E Bailey  wrote:
> > ...In either case, you'd end up with a central location with a snapshot of
> > the latest code that is individually managed in it's own repo
> 
> IIUC this has no impact on the existing repositories, it's just an
> additional one that uses submodules and Jenkins to keep that
> aggregated snapshot mostly up to date?
> 
> If yes I think that's interesting - I wouldn't want us to be too
> dependent on submodules, but if it's a distinct aggregated view that
> sounds useful.
> 
> -Bertrand

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

Hi,

On Thu, Feb 8, 2018 at 1:37 AM, Jason E Bailey  wrote:
> ...In either case, you'd end up with a central location with a snapshot of
> the latest code that is individually managed in it's own repo

IIUC this has no impact on the existing repositories, it's just an
additional one that uses submodules and Jenkins to keep that
aggregated snapshot mostly up to date?

If yes I think that's interesting - I wouldn't want us to be too
dependent on submodules, but if it's a distinct aggregated view that
sounds useful.

-Bertrand

Re: apache/sling as github landing repository

[Jason E Bailey]

What we have is a  jenkinsFile so that, as part of the commit ,  it triggers 
the repository that's acting as the uber repository to update it's  submodules. 
This could even be modified  slightly so that a given submodule is only updated 
on it's release rather then every update.

In either case, you'd end up with a central location with a snapshot of the 
latest code that is individually managed in it's own repo.

the whole relying on github to grep multiple search result to try to determine 
the root repo is friction that isn't necessary.

- Jason

On Wed, Feb 7, 2018, at 6:10 PM, Alexander Klimetschek wrote:
> Git submodules don't work well in this situation, as the will point to a 
> particular revision of the submodule. You'd have to constantly change 
> the super repository whenever there was a commit in a submodule.
> 
> Sling uses Android's repo for this purpose. The super repo is called 
> sling-aggregator [0].
> 
> A related tool we are using at our company is gitslave [1], which works 
> ok. Usually it ends being used for tools such as source code audit or 
> locale string extraction, that should see the entire code base. But 
> human devs usually leverage github's search features to find code in 
> individual repos if they don't know which repo yet.
> 
> [0] https://github.com/apache/sling-aggregator
> [1] http://gitslave.sourceforge.net
> 
> Cheers,
> Alex
> 
> > On 05.02.2018, at 18:45, Jason E Bailey  wrote:
> > 
> > Has there been any consideration of using git submodules to have a single 
> > git repository that reflects all the other repositories? It would allow for 
> > a central view,  facilitate search, and still maintain a separation of the 
> > individual projects.
> > 
> > - Jason
> > 
> > On Fri, Feb 2, 2018, at 10:16 PM, Alexander Klimetschek wrote:
> >> On 31.01.2018, at 00:23, Robert Munteanu  wrote:
> >>> Links to commits and files from the old sling repo. For example
> >>> 
> >>> * https://github.com/apache/sling/commit/368f5f9c9f6d4c0e2602065687d95e
> >>> b173f94b85
> >>> * https://github.com/apache/sling/blob/trunk/bundles/extensions/bundler
> >>> esource/src/main/java/org/apache/sling/bundleresource/impl/BundleResour
> >>> ce.java
> >>> 
> >>> These would break if we add another 'sling' repo but work since we
> >>> renamed 'sling' to 'sling-old-svn-mirror' and Github is adding
> >>> redirects.
> >>> […]
> >>> Right, but it's not about conflicts, it's about breaking old links.
> >>> Backwards compatiblity and all that :-)
> >> 
> >> Step 2 would include the old sling repo under apache/sling again, so 
> >> that all links work (as discussed below).
> >> 
> >>> Actually that's not a bad idea :-) The only downside would be that
> >>> cloning the repository would be really slow due to the large size. Not
> >>> sure if we can work around it.
> >> 
> >> Then maybe it's ok to have the aggregator list in a different repo, 
> >> prominently linked from apache/sling, and the sling one stays as just a 
> >> landing page repo, with mostly manually curated markdown files. Where 
> >> cloning a large repo is not such a big deal, as it would be on people's 
> >> local computer and already cloned (unlike the aggregator which might be 
> >> cloned a lot by CI tools and new users).
> >> 
> >> Cheers,
> >> Alex
> 

Re: apache/sling as github landing repository

[Alexander Klimetschek]

Git submodules don't work well in this situation, as the will point to a 
particular revision of the submodule. You'd have to constantly change the super 
repository whenever there was a commit in a submodule.

Sling uses Android's repo for this purpose. The super repo is called 
sling-aggregator [0].

A related tool we are using at our company is gitslave [1], which works ok. 
Usually it ends being used for tools such as source code audit or locale string 
extraction, that should see the entire code base. But human devs usually 
leverage github's search features to find code in individual repos if they 
don't know which repo yet.

[0] https://github.com/apache/sling-aggregator
[1] http://gitslave.sourceforge.net

Cheers,
Alex

> On 05.02.2018, at 18:45, Jason E Bailey  wrote:
> 
> Has there been any consideration of using git submodules to have a single git 
> repository that reflects all the other repositories? It would allow for a 
> central view,  facilitate search, and still maintain a separation of the 
> individual projects.
> 
> - Jason
> 
> On Fri, Feb 2, 2018, at 10:16 PM, Alexander Klimetschek wrote:
>> On 31.01.2018, at 00:23, Robert Munteanu  wrote:
>>> Links to commits and files from the old sling repo. For example
>>> 
>>> * https://github.com/apache/sling/commit/368f5f9c9f6d4c0e2602065687d95e
>>> b173f94b85
>>> * https://github.com/apache/sling/blob/trunk/bundles/extensions/bundler
>>> esource/src/main/java/org/apache/sling/bundleresource/impl/BundleResour
>>> ce.java
>>> 
>>> These would break if we add another 'sling' repo but work since we
>>> renamed 'sling' to 'sling-old-svn-mirror' and Github is adding
>>> redirects.
>>> […]
>>> Right, but it's not about conflicts, it's about breaking old links.
>>> Backwards compatiblity and all that :-)
>> 
>> Step 2 would include the old sling repo under apache/sling again, so 
>> that all links work (as discussed below).
>> 
>>> Actually that's not a bad idea :-) The only downside would be that
>>> cloning the repository would be really slow due to the large size. Not
>>> sure if we can work around it.
>> 
>> Then maybe it's ok to have the aggregator list in a different repo, 
>> prominently linked from apache/sling, and the sling one stays as just a 
>> landing page repo, with mostly manually curated markdown files. Where 
>> cloning a large repo is not such a big deal, as it would be on people's 
>> local computer and already cloned (unlike the aggregator which might be 
>> cloned a lot by CI tools and new users).
>> 
>> Cheers,
>> Alex

Re: apache/sling as github landing repository

[Jason E Bailey]

Has there been any consideration of using git submodules to have a single git 
repository that reflects all the other repositories? It would allow for a 
central view,  facilitate search, and still maintain a separation of the 
individual projects.

- Jason

On Fri, Feb 2, 2018, at 10:16 PM, Alexander Klimetschek wrote:
> On 31.01.2018, at 00:23, Robert Munteanu  wrote:
> > Links to commits and files from the old sling repo. For example
> > 
> > * https://github.com/apache/sling/commit/368f5f9c9f6d4c0e2602065687d95e
> > b173f94b85
> > * https://github.com/apache/sling/blob/trunk/bundles/extensions/bundler
> > esource/src/main/java/org/apache/sling/bundleresource/impl/BundleResour
> > ce.java
> > 
> > These would break if we add another 'sling' repo but work since we
> > renamed 'sling' to 'sling-old-svn-mirror' and Github is adding
> > redirects.
> > […]
> > Right, but it's not about conflicts, it's about breaking old links.
> > Backwards compatiblity and all that :-)
> 
> Step 2 would include the old sling repo under apache/sling again, so 
> that all links work (as discussed below).
> 
> > Actually that's not a bad idea :-) The only downside would be that
> > cloning the repository would be really slow due to the large size. Not
> > sure if we can work around it.
> 
> Then maybe it's ok to have the aggregator list in a different repo, 
> prominently linked from apache/sling, and the sling one stays as just a 
> landing page repo, with mostly manually curated markdown files. Where 
> cloning a large repo is not such a big deal, as it would be on people's 
> local computer and already cloned (unlike the aggregator which might be 
> cloned a lot by CI tools and new users).
> 
> Cheers,
> Alex

Re: apache/sling as github landing repository

[Alexander Klimetschek]

On 31.01.2018, at 00:23, Robert Munteanu  wrote:
> Links to commits and files from the old sling repo. For example
> 
> * https://github.com/apache/sling/commit/368f5f9c9f6d4c0e2602065687d95e
> b173f94b85
> * https://github.com/apache/sling/blob/trunk/bundles/extensions/bundler
> esource/src/main/java/org/apache/sling/bundleresource/impl/BundleResour
> ce.java
> 
> These would break if we add another 'sling' repo but work since we
> renamed 'sling' to 'sling-old-svn-mirror' and Github is adding
> redirects.
> […]
> Right, but it's not about conflicts, it's about breaking old links.
> Backwards compatiblity and all that :-)

Step 2 would include the old sling repo under apache/sling again, so that all 
links work (as discussed below).

> Actually that's not a bad idea :-) The only downside would be that
> cloning the repository would be really slow due to the large size. Not
> sure if we can work around it.

Then maybe it's ok to have the aggregator list in a different repo, prominently 
linked from apache/sling, and the sling one stays as just a landing page repo, 
with mostly manually curated markdown files. Where cloning a large repo is not 
such a big deal, as it would be on people's local computer and already cloned 
(unlike the aggregator which might be cloned a lot by CI tools and new users).

Cheers,
Alex

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

On Wed, Jan 31, 2018 at 4:05 PM, Robert Munteanu  wrote:
> ...perhaps we should consider making the aggregator the 'sling'
> repository, explaining where the old links went in a README, and
> keeping 'sling-old-svn-mirror' without a redirect...

That would work for me.

As long as people find instructions to get, build and explore Sling
when they land at github.com/apache/sling I think we're fine.

-Bertrand

Re: apache/sling as github landing repository

[Robert Munteanu]

On Wed, 2018-01-31 at 10:00 +0100, Bertrand Delacretaz wrote:
> Hi,
> 
> On Wed, Jan 31, 2018 at 9:23 AM, Robert Munteanu 
> wrote:
> > On Wed, 2018-01-31 at 00:13 +, Alexander Klimetschek wrote:
> > > ... 4. add the aggregator and new landing page readme on the
> > > master
> > > branch, which will be push-able for committers...
> > > 
> > 
> > ...The only downside would be that
> > cloning the repository would be really slow due to the large
> > size
> 
> I think we need to have the aggregator [0] run a regular build to
> update the list of Git repositories, soon. That becomes more
> important
> as that list starts including tags, contrib/core/etc status and other
> metadata that will change more often than now.
> 
> I don't think a large Git repository is a good place for such an
> automatically built list of Git repositories metadata, as Jenkins
> would need to transfer lots of unneeded data, a dedicated small
> repository is much better for that. And I'm not sure right now how to
> push the output of that to any branch from builds.apache.org, at this
> point I only know how to commit to an asf-site branch (by tying the
> Jenkins build to build nodes having the git-websites label).

Then perhaps we should consider making the aggregator the 'sling'
repository, explaining where the old links went in a README, and
keeping 'sling-old-svn-mirror' without a redirect.

Also see the 'Sling docker image' thread [1], it seems keeping the main
repo read-only has some disadvantages we did not consider.

Robert

[1] https://lists.apache.org/thread.html/2b0f66c9d9a756bc574c233d9d0904
8713944b602d7292fb8001d69c@%3Cdev.sling.apache.org%3E

> 
> -Bertrand
> 
> > [0] https://github.com/apache/sling-aggregator - which needs a
> > Maven build to be automated

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

Hi,

On Wed, Jan 31, 2018 at 9:23 AM, Robert Munteanu  wrote:
> On Wed, 2018-01-31 at 00:13 +, Alexander Klimetschek wrote:
>>... 4. add the aggregator and new landing page readme on the master
>> branch, which will be push-able for committers...
>>
> ...The only downside would be that
> cloning the repository would be really slow due to the large size

I think we need to have the aggregator [0] run a regular build to
update the list of Git repositories, soon. That becomes more important
as that list starts including tags, contrib/core/etc status and other
metadata that will change more often than now.

I don't think a large Git repository is a good place for such an
automatically built list of Git repositories metadata, as Jenkins
would need to transfer lots of unneeded data, a dedicated small
repository is much better for that. And I'm not sure right now how to
push the output of that to any branch from builds.apache.org, at this
point I only know how to commit to an asf-site branch (by tying the
Jenkins build to build nodes having the git-websites label).

-Bertrand

> [0] https://github.com/apache/sling-aggregator - which needs a Maven build to 
> be automated

Re: apache/sling as github landing repository

[Robert Munteanu]

On Wed, 2018-01-31 at 00:13 +, Alexander Klimetschek wrote:
> On 30.01.2018, at 01:38, Robert Munteanu  wrote:
> > The constraint that we worked with is that many links would be
> > pointing
> > to https://github.com/apache/sling and these would be broken if we
> > added another 'sling' repository to the mix.  So we decided to keep
> > the
> > old 'sling' repository in place, renamed, so links would not break.
> 
> What new links were you thinking of?

Links to commits and files from the old sling repo. For example

* https://github.com/apache/sling/commit/368f5f9c9f6d4c0e2602065687d95e
b173f94b85
* https://github.com/apache/sling/blob/trunk/bundles/extensions/bundler
esource/src/main/java/org/apache/sling/bundleresource/impl/BundleResour
ce.java

These would break if we add another 'sling' repo but work since we
renamed 'sling' to 'sling-old-svn-mirror' and Github is adding
redirects.

That's the main argument against adding a new Git "native" repo as
"sling".

> 
> IIUC, it would be only a README.md and some others markdown files,
> plus the few xml files in sling-aggregator [0] in the "master"
> branch. Which is by design not in the old repo which used subversions
> "trunk".
> 
> Note that any link to a file or subfolder in github would always
> include the branch name.  Not sure if you would ever create branches
> other than "master" for this "landing" repo, and any that would
> conflict with the few fixed branches from the old repo [1].
> 
> tl;dr I don't think there is any chance for conflicts.

Right, but it's not about conflicts, it's about breaking old links.
Backwards compatiblity and all that :-)

> 
> > The Sling SVN repository is read-only to make sure we don't
> > accidentally commit changes there. We would need to go through
> > infra
> > for each change so frequent updates
> 
> Would it be possible to transform the github apache/sling repo, being
> a read-only mirror of the svn one (IIUC), into an active github repo?
> This might be a one time change, but you can still protect the old
> stuff.
> 
> Something like this:
> 1. delete apache/sling-old-svn-mirror on github and disable the
> mirroring (infra team?)
> 2. re-create apache/sling from the git mirror repos available at ASF
> (infra team?)
> 3. make all the old branches protected so they can't be modified
> anymore
> 4. add the aggregator and new landing page readme on the master
> branch, which will be push-able for committers
> 
> wdyt?

Actually that's not a bad idea :-) The only downside would be that
cloning the repository would be really slow due to the large size. Not
sure if we can work around it.

What do others think?

Robert

> 
> [0] https://github.com/apache/sling-aggregator
> [1] https://github.com/apache/sling-old-svn-mirror/branches/all
> 
> Cheers,
> Alex

Re: apache/sling as github landing repository

[Alexander Klimetschek]

On 30.01.2018, at 01:38, Robert Munteanu  wrote:
> The constraint that we worked with is that many links would be pointing
> to https://github.com/apache/sling and these would be broken if we
> added another 'sling' repository to the mix.  So we decided to keep the
> old 'sling' repository in place, renamed, so links would not break.

What new links were you thinking of?

IIUC, it would be only a README.md and some others markdown files, plus the few 
xml files in sling-aggregator [0] in the "master" branch. Which is by design 
not in the old repo which used subversions "trunk".

Note that any link to a file or subfolder in github would always include the 
branch name.  Not sure if you would ever create branches other than "master" 
for this "landing" repo, and any that would conflict with the few fixed 
branches from the old repo [1].

tl;dr I don't think there is any chance for conflicts.

> The Sling SVN repository is read-only to make sure we don't
> accidentally commit changes there. We would need to go through infra
> for each change so frequent updates

Would it be possible to transform the github apache/sling repo, being a 
read-only mirror of the svn one (IIUC), into an active github repo? This might 
be a one time change, but you can still protect the old stuff.

Something like this:
1. delete apache/sling-old-svn-mirror on github and disable the mirroring 
(infra team?)
2. re-create apache/sling from the git mirror repos available at ASF (infra 
team?)
3. make all the old branches protected so they can't be modified anymore
4. add the aggregator and new landing page readme on the master branch, which 
will be push-able for committers

wdyt?

[0] https://github.com/apache/sling-aggregator
[1] https://github.com/apache/sling-old-svn-mirror/branches/all

Cheers,
Alex

Re: apache/sling as github landing repository

[Robert Munteanu]

Hi Alex,

Thanks for taking this discussion to the dev list, there is a real need
to improve our landing page.

Please see my individual response below. I'm explaining the process we
went through to arrive at this situation, not countering your
arguments.

In the end I'm happy to work with whatever solution we have an
agreement on, including reversing the current approach.

On Thu, 2018-01-25 at 23:54 +, Alexander Klimetschek wrote:
> 1. restore the apache/sling named repo, since that should help with
> SEO and "keeping" the existing brand

If you go to https://github.com/apache/sling it still works, just that
you're redirected to another repo. 

> 
> 2. have a readme in there as the github landing page, just like any
> github project nowadays, which should include an about project and
> most information how to use sling/download it, find source repos and
> how to build it. Similar to the project information page [3], but
> more easily digestible with less text.

Fully agreed, we can easily do this as a one-off change.

> 
> 3. move the aggregator [2] to apache/sling, as that feels like a
> natural place (since this all happens on the new "master" branch,
> there is no conflict with the old sling code base in there)

The constraint that we worked with is that many links would be pointing
to https://github.com/apache/sling and these would be broken if we
added another 'sling' repository to the mix.  So we decided to keep the
old 'sling' repository in place, renamed, so links would not break.

We also switched the default branch so that people would not start
browsing the code since it's now outdated. What is presented on the
landing page is actually

  https://svn.apache.org/repos/asf/sling/branches/archived/

> 4. have a curated list of repos in there (could be a separated
> markdown prominently linked from the main readme given its size),
> which would provide some categorization and e.g. start with the
> important stuff at the top. Guidelines for creating new
> repos/changing repos should hint at updating this readme. But even if
> it's not 100% up to date all the time, the vast majority of repos
> that don't change over time will be explorable.

The Sling SVN repository is read-only to make sure we don't
accidentally commit changes there. We would need to go through infra
for each change so frequent updates 

Thanks,

Robert

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

On Mon, Jan 29, 2018 at 7:00 PM, Alexander Klimetschek
 wrote:
> Just to clarify: you guys actually meant "apache/sling", right?

Yes, https://github.com/apache/sling should be the "natural" entry point.

-Bertrand

Re: apache/sling as github landing repository

[Alexander Klimetschek]

On 29.01.2018, at 07:53, Bertrand Delacretaz  wrote:
> On Mon, Jan 29, 2018 at 4:08 PM, Daniel Klco  wrote:
>> ...I'd
>> suspect that a majority of potential contributors would be coming through
>> GitHub...
> 
> Sure, and I agree to make sling/sling the default entry point

Just to clarify: you guys actually meant "apache/sling", right?

There is a user "sling" on github.com already [1], and IIUC, all Apache 
projects should be under the "apache" org on github.

[1] https://github.com/sling

Cheers,
Alex

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

Hi Dan,

On Mon, Jan 29, 2018 at 4:08 PM, Daniel Klco  wrote:
> ...I'd
> suspect that a majority of potential contributors would be coming through
> GitHub...

Sure, and I agree to make sling/sling the default entry point - but it
doesn't matter much whether that has the list of repos embedded or
linked, IMO.

> ...it's always nice to have multiple
> repetitions as long as we keep them in sync automatically...

I'm not convinced of that, I prefer having a single point of truth.
But as I said I don't mind moving
http://sling.apache.org/repolist.html to another place as long as it's
automated and someone has a good reason for that.

Right now the list is not fully automated, one needs to refresh both
the https://github.com/apache/sling-aggregator and the website
manually, but at least it's just simple commands with no room for data
errors.

-Bertrand

Re: apache/sling as github landing repository

[Daniel Klco]

@Bertrand,

Personally, I had a lot of trouble figuring out where to go to find / how
to efficiently check out the code for Sling after the move to GitHub. I'd
suspect that a majority of potential contributors would be coming through
GitHub, so I'd highly suggest we put some effort in to make it easier to
understand how the individual modules contribute to the overall Sling
application.

I do not think it would take a significant effort to do as @Alex proposes
and make the sling/sling repository as the starting point with the
aggregator, update the submodule readme files to have a getting started
shell and point users back to to the sling/sling repository to get started.

As for having a list on the site, it's always nice to have multiple
repetitions as long as we keep them in sync automatically.

On Mon, Jan 29, 2018 at 4:44 AM, Bertrand Delacretaz  wrote:

> On Mon, Jan 29, 2018 at 10:30 AM, Nicolas Peltier
>  wrote:
> > ...I agree having one list might be better, but wouldn’t it make sense
> to have the list in GitHub, to avoid ping-pong
> > if you come from GitHub browsing ?...
>
> I don't think it makes much of a difference as long as things are
> properly linked. It's not like we have millions of users hitting those
> URLs ;-)
>
> What's important for me is to have an autogenerated list so it doesn't
> become stale, if needed adding tags or badges to the repositories to
> get a better classification (see other thread about that from 2
> minutes ago).
>
> And whoever does the work decides ;-) ...so far I did that for
> http://sling.apache.org/repolist.html but I'm open to improvements of
> course, including moving if there's consensus on that
>
> -Bertrand
>

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

On Mon, Jan 29, 2018 at 10:30 AM, Nicolas Peltier
 wrote:
> ...I agree having one list might be better, but wouldn’t it make sense to 
> have the list in GitHub, to avoid ping-pong
> if you come from GitHub browsing ?...

I don't think it makes much of a difference as long as things are
properly linked. It's not like we have millions of users hitting those
URLs ;-)

What's important for me is to have an autogenerated list so it doesn't
become stale, if needed adding tags or badges to the repositories to
get a better classification (see other thread about that from 2
minutes ago).

And whoever does the work decides ;-) ...so far I did that for
http://sling.apache.org/repolist.html but I'm open to improvements of
course, including moving if there's consensus on that

-Bertrand

Re: apache/sling as github landing repository

[Nicolas Peltier]

I agree having one list might be better, but wouldn’t it make sense to have the 
list in GitHub, to avoid ping-pong if you come from GitHub browsing ? (Thinking 
about Alex’s description of the experience of someone arriving on apache/sling 
GitHub project)

> On 29 Jan 2018, at 10:27, Bertrand Delacretaz  wrote:
> 
> On Fri, Jan 26, 2018 at 6:05 PM, Bertrand Delacretaz
>  wrote:
>> ...I have created a list of all our repositories at
>> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsling.apache.org%2Frepolist.html=02%7C01%7Cnpeltier%40adobe.com%7Cdb0cb6b4a4354a93442208d566fa8b78%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636528148629706625=QOzPoQMzzOG5%2FTtcWbZwuG6cBwwIDz10P7FYy0vM2rg%3D=0
>>  based on the XML list of
>> projects generated at 
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fsling-aggregator=02%7C01%7Cnpeltier%40adobe.com%7Cdb0cb6b4a4354a93442208d566fa8b78%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636528148629706625=UE%2F0EMbCT%2FXTl4THYFtG%2FhV04cDYQpw6asOrEy3w1sA%3D=0
> 
> And I realize we've had several ongoing initiatives to generate this
> list of repositories.
> 
> I wanted to have a concrete starting point, and the template that
> generates the 
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsling.apache.org%2Frepolist.html=02%7C01%7Cnpeltier%40adobe.com%7Cdb0cb6b4a4354a93442208d566fa8b78%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636528148629706625=QOzPoQMzzOG5%2FTtcWbZwuG6cBwwIDz10P7FYy0vM2rg%3D=0
>  page [1] is easy
> to adapt if we want to switch to another list of repositories. As long
> as that's automatically generated we're flexible.
> 
> -Bertrand
> 
> [1] 
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fsling-site%2Fblob%2Fmaster%2Fsrc%2Fmain%2Fjbake%2Ftemplates%2Frepolist.tpl=02%7C01%7Cnpeltier%40adobe.com%7Cdb0cb6b4a4354a93442208d566fa8b78%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636528148629706625=ykoHLGeG%2F62%2Fxpcp29B6M5xe4ea10hn2qWBWprT1IPE%3D=0

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

On Fri, Jan 26, 2018 at 6:05 PM, Bertrand Delacretaz
 wrote:
> ...I have created a list of all our repositories at
> http://sling.apache.org/repolist.html based on the XML list of
> projects generated at https://github.com/apache/sling-aggregator

And I realize we've had several ongoing initiatives to generate this
list of repositories.

I wanted to have a concrete starting point, and the template that
generates the http://sling.apache.org/repolist.html page [1] is easy
to adapt if we want to switch to another list of repositories. As long
as that's automatically generated we're flexible.

-Bertrand

[1] 
https://github.com/apache/sling-site/blob/master/src/main/jbake/templates/repolist.tpl

Re: apache/sling as github landing repository

[Konrad Windszus]

Thanks, maybe we can use the branch from 
https://github.com/apache/sling-site/tree/scm-project-url-list 
 (mentioned in 
https://issues.apache.org/jira/browse/SLING-7161 
) as a basis instead. That 
provides already much more information and leveraging the already existing xml 
file from sling-aggregator is a good idea as long as 
https://issues.apache.org/jira/browse/SLING-7262 
 is not resolved.
Konrad

> On 26. Jan 2018, at 18:05, Bertrand Delacretaz  wrote:
> 
> Hi,
> 
> On Fri, Jan 26, 2018 at 12:54 AM, Alexander Klimetschek
>  wrote:
>> ...
>> 1. restore the apache/sling named repo...
>> 2. have a readme in there as the github landing page, just like any github 
>> project nowadays,
>> which should include an about project and most information how to use 
>> sling/download it...
> 
> I like the idea in general, thanks!
> 
> We already have download/build information on our website, I think we
> shouldn't duplicate - so either remove the webiste info or point to it
> from that README, but create something consistent and that does not
> easily get stale.
> 
> For now, I have created a list of all our repositories at
> http://sling.apache.org/repolist.html based on the XML list of
> projects generated at https://github.com/apache/sling-aggregator. It
> would be nice to expand it with a few words about each repository but
> I haven't checked how to generate that so far - and whether that would
> interfere with the repo tool.
> 
> -Bertrand

Re: apache/sling as github landing repository

[Bertrand Delacretaz]

Hi,

On Fri, Jan 26, 2018 at 12:54 AM, Alexander Klimetschek
 wrote:
>...
> 1. restore the apache/sling named repo...
> 2. have a readme in there as the github landing page, just like any github 
> project nowadays,
> which should include an about project and most information how to use 
> sling/download it...

I like the idea in general, thanks!

We already have download/build information on our website, I think we
shouldn't duplicate - so either remove the webiste info or point to it
from that README, but create something consistent and that does not
easily get stale.

For now, I have created a list of all our repositories at
http://sling.apache.org/repolist.html based on the XML list of
projects generated at https://github.com/apache/sling-aggregator. It
would be nice to expand it with a few words about each repository but
I haven't checked how to generate that so far - and whether that would
interfere with the repo tool.

-Bertrand

RE: apache/sling as github landing repository

[Stefan Seifert]

+1

stefan

>-Original Message-
>From: Alexander Klimetschek [mailto:aklim...@adobe.com.INVALID]
>Sent: Friday, January 26, 2018 12:54 AM
>To: dev@sling.apache.org
>Subject: apache/sling as github landing repository
>
>Hi everyone,
>
>after the move to github and source code modularization (cool), the
>previous sling repository apache/sling [0] is now empty. Except for a
>readme that currently mostly addresses active committers (few, experts that
>know the context) rather than users of sling (many, newbies that see Sling
>code for the first time). It was renamed to apache/sling-old-svn-mirror.
>Old links – if they include a branch or revision will still work, including
>trunk – which is good.
>
>However, sling now doesn't have a good github "landing" repository anymore.
>You know, a central place that tells you where to find the code, which is
>now scattered across many repos, or that allows you to browse it without
>knowing exactly what to search for.
>
>Currently the very well hidden answers are [1] and [2], which aren't
>obvious. When hitting apache/sling, you find a readme that links to the
>project info page [3] that among many other links and lots of texts has a
>link to [1], which is hard to find IMO and has too many steps.
>
>Furthermore, [1] is not curated and depends on whatever result order sling
>gives you. But it might be very useful to categorize the repos, say all
>"resource resolver" related stuff, or extensions. Oliver Lietz built [5]
>based on automatic processing of [2], but it's also not curated and hard to
>find things if you don't know exactly how things are split up and named.
>
>My humble suggestion would be:
>
>1. restore the apache/sling named repo, since that should help with SEO and
>"keeping" the existing brand
>
>2. have a readme in there as the github landing page, just like any github
>project nowadays, which should include an about project and most
>information how to use sling/download it, find source repos and how to
>build it. Similar to the project information page [3], but more easily
>digestible with less text.
>
>3. move the aggregator [2] to apache/sling, as that feels like a natural
>place (since this all happens on the new "master" branch, there is no
>conflict with the old sling code base in there)
>
>4. have a curated list of repos in there (could be a separated markdown
>prominently linked from the main readme given its size), which would
>provide some categorization and e.g. start with the important stuff at the
>top. Guidelines for creating new repos/changing repos should hint at
>updating this readme. But even if it's not 100% up to date all the time,
>the vast majority of repos that don't change over time will be explorable.
>
>WDYT?
>
>FYI, this was initially discussed on [4]. I was always looking at Sling
>source code on github before, searching, reading, looking at the version
>history, even when it was "just" a mirror of the svn. When you were only
>reading, this didn't matter, and github's web view is so much better than
>the svn web view. I believe many folks did the same. The switch now is a
>bit of a break in that approach, if you can't easily find the source code
>anymore.
>
>[0] https://github.com/apache/sling
>[1] https://github.com/apache/?q=sling
>[2] https://github.com/apache/sling-aggregator
>[3] https://sling.apache.org/project-information.html#source-repository
>[4] https://github.com/apache/sling-old-svn-
>mirror/commit/9c14db46650bd9b6017511e8a61d021e17b4e1d2#commitcomment-
>26941906
>[5] https://oliverlietz.github.io/apache-sling-aggregator/
>
>Cheers,
>Alex

Re: apache/sling as github landing repository

[Daniel Klco]

+1 this absolutely is there right approach for users

On Jan 25, 2018 3:54 PM, "Alexander Klimetschek" 
wrote:

Hi everyone,

after the move to github and source code modularization (cool), the
previous sling repository apache/sling [0] is now empty. Except for a
readme that currently mostly addresses active committers (few, experts that
know the context) rather than users of sling (many, newbies that see Sling
code for the first time). It was renamed to apache/sling-old-svn-mirror.
Old links – if they include a branch or revision will still work, including
trunk – which is good.

However, sling now doesn't have a good github "landing" repository anymore.
You know, a central place that tells you where to find the code, which is
now scattered across many repos, or that allows you to browse it without
knowing exactly what to search for.

Currently the very well hidden answers are [1] and [2], which aren't
obvious. When hitting apache/sling, you find a readme that links to the
project info page [3] that among many other links and lots of texts has a
link to [1], which is hard to find IMO and has too many steps.

Furthermore, [1] is not curated and depends on whatever result order sling
gives you. But it might be very useful to categorize the repos, say all
"resource resolver" related stuff, or extensions. Oliver Lietz built [5]
based on automatic processing of [2], but it's also not curated and hard to
find things if you don't know exactly how things are split up and named.

My humble suggestion would be:

1. restore the apache/sling named repo, since that should help with SEO and
"keeping" the existing brand

2. have a readme in there as the github landing page, just like any github
project nowadays, which should include an about project and most
information how to use sling/download it, find source repos and how to
build it. Similar to the project information page [3], but more easily
digestible with less text.

3. move the aggregator [2] to apache/sling, as that feels like a natural
place (since this all happens on the new "master" branch, there is no
conflict with the old sling code base in there)

4. have a curated list of repos in there (could be a separated markdown
prominently linked from the main readme given its size), which would
provide some categorization and e.g. start with the important stuff at the
top. Guidelines for creating new repos/changing repos should hint at
updating this readme. But even if it's not 100% up to date all the time,
the vast majority of repos that don't change over time will be explorable.

WDYT?

FYI, this was initially discussed on [4]. I was always looking at Sling
source code on github before, searching, reading, looking at the version
history, even when it was "just" a mirror of the svn. When you were only
reading, this didn't matter, and github's web view is so much better than
the svn web view. I believe many folks did the same. The switch now is a
bit of a break in that approach, if you can't easily find the source code
anymore.

[0] https://github.com/apache/sling
[1] https://github.com/apache/?q=sling
[2] https://github.com/apache/sling-aggregator
[3] https://sling.apache.org/project-information.html#source-repository
[4] https://github.com/apache/sling-old-svn-mirror/commit/
9c14db46650bd9b6017511e8a61d021e17b4e1d2#commitcomment-26941906
[5] https://oliverlietz.github.io/apache-sling-aggregator/

Cheers,
Alex