Re: Move web repo to pagure - considerations

2016-02-29 Thread Paul W. Frields 
On Fri, Feb 26, 2016 at 07:00:50PM +1000, Ryan Lerch wrote:
> On 02/25/2016 07:08 PM, Robert Mayr wrote:
> >Hi all,
> >in the last weeks we had some discussions on IRC about moving the
> >webrepository from fedorahosted to pagure and I also filed a ticket [1] to
> >collect opinions about this proposal. There are several advantages for us
> >to move over there and we will start moving with the next release cycle,
> >which has started these days (we will branch during weekend or next week
> >probably).
> >I've set up a testing repository [2] where I also wrote some docs to test
> >how this would look like. It is working really fine and having a unique
> >place for all our stuff where we can easily handle PRs is a big
> >enhancement.
> >
> >At this point there are several opinions about how to move to pagure.
> >Personally I want to reduce the weight of the web-repo, which is about
> >660MB or better, 870Mb uncompressed. Not only we have many branches in
> >there, the reason of all these MBs are static binaries and mainly images
> >we collected over all the last years.
> >If we want to solve this problem, we have two options, in both cases all
> >team members MUST rebase or even reclone the repository:
> >
> >1) We can run a "git gc --aggressive" on the repo, but a test (thanks nb)
> >confirmed we would not save too much space. Pros for this solution are:
> >* people can just rebase and setup the new pagure URL in the git config
> >file
> >* we keep all history and commit credits
> >* we can keep also branches where we have ongoing development
> >
> >2) We could start a complete new repository, which would give us also the
> >possibility to delete many heavy binaries we are not using at all. We
> >would probably end up with an uncompressed size of less than 200MB. There
> >are several cons, but in this specific moment we are able to handle them:
> >* we will loose commit credits and history ^
> >* we cannot build any other branch than master and f24-alpha ^^
> >* all members MUST reclone the repository (has the advantage that it would
> >be a clean solution and it will be much faster than now)
> >
> >^ fedorahosted.org  would be still there and if
> >needed we can look in the web UI if we need to find out how things changed
> >in a specific commit. Also, people can obviously keep the old fedora-web
> >repo locally and use it for investigations. Commit credits could be saved
> >in a CREDITS file we can put into the new repo.
> >^^ Actually all websites are built against master branch and we would
> >build staging with f24-alpha. Our devel branches are just 2, both of them
> >can be merged to master without creating conflicts or risks to break our
> >production websites.
> >
> >All this said, personally I'd really like to go for a new repository and
> >archive all the stuff we have now in the old fedorahosted.org
> > repo. Any opinions about that? Other ideas or
> >concerns?
> >
> >Thank you, specially for reading my long mail until the end.
> >
> >[1] https://fedorahosted.org/fedora-websites/ticket/374
> >[2] https://stg.pagure.io/Fedora_Websites_test
> >
> >-- 
> >Robert Mayr
> >(robyduck)
> >
> >
> >--
> >websites mailing list
> >websites@lists.fedoraproject.org
> >http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org
> Pagure also has the option to make groups, and everyone have the permissions
> on that group too,
> 
> Since creating new git repos in pagure is trivial, one other option might be
> to have sepeerate repos for each web page / project that fedora websites
> works on under a new fedora-web group. That way if i want to work on a
> particular website, i only have to pull down the repo for that site, not all
> the websites. Also, each repo could then have it's own neat README in the
> pagure interface on how to develop on that particular website etc.
> 
> One drawback (or maybe a postitive depending on what way you look at it)
> with this is that each repo will have a seperate issue tracker.
> 
> just a thought,

I don't have strong feelings about this.  But I think from the POV of
a casual user who finds a bug, it could be more difficult to know
where to file if the sites are split up.  Unless... perhaps the footer
of the site, or some recurring area, could have a direct link to file
a website bug?

-- 
Paul W. Frieldshttp://paul.frields.org/
  gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233  5906 ACDB C937 BD11 3717
  http://redhat.com/   -  -  -  -   http://pfrields.fedorapeople.org/
The open source story continues to grow: http://opensource.com
--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org

Re: Move web repo to pagure - considerations

2016-02-26 Thread Ryan Lerch 

On 02/25/2016 07:08 PM, Robert Mayr wrote:

Hi all,
in the last weeks we had some discussions on IRC about moving the 
webrepository from fedorahosted to pagure and I also filed a ticket 
[1] to collect opinions about this proposal. There are several 
advantages for us to move over there and we will start moving with the 
next release cycle, which has started these days (we will branch 
during weekend or next week probably).
I've set up a testing repository [2] where I also wrote some docs to 
test how this would look like. It is working really fine and having a 
unique place for all our stuff where we can easily handle PRs is a big 
enhancement.


At this point there are several opinions about how to move to pagure. 
Personally I want to reduce the weight of the web-repo, which is about 
660MB or better, 870Mb uncompressed. Not only we have many branches in 
there, the reason of all these MBs are static binaries and mainly 
images we collected over all the last years.
If we want to solve this problem, we have two options, in both cases 
all team members MUST rebase or even reclone the repository:


1) We can run a "git gc --aggressive" on the repo, but a test (thanks 
nb) confirmed we would not save too much space. Pros for this solution 
are:
* people can just rebase and setup the new pagure URL in the git 
config file

* we keep all history and commit credits
* we can keep also branches where we have ongoing development

2) We could start a complete new repository, which would give us also 
the possibility to delete many heavy binaries we are not using at all. 
We would probably end up with an uncompressed size of less than 200MB. 
There are several cons, but in this specific moment we are able to 
handle them:

* we will loose commit credits and history ^
* we cannot build any other branch than master and f24-alpha ^^
* all members MUST reclone the repository (has the advantage that it 
would be a clean solution and it will be much faster than now)


^ fedorahosted.org  would be still there and 
if needed we can look in the web UI if we need to find out how things 
changed in a specific commit. Also, people can obviously keep the old 
fedora-web repo locally and use it for investigations. Commit credits 
could be saved in a CREDITS file we can put into the new repo.
^^ Actually all websites are built against master branch and we would 
build staging with f24-alpha. Our devel branches are just 2, both of 
them can be merged to master without creating conflicts or risks to 
break our production websites.


All this said, personally I'd really like to go for a new repository 
and archive all the stuff we have now in the old fedorahosted.org 
 repo. Any opinions about that? Other ideas 
or concerns?


Thank you, specially for reading my long mail until the end.

[1] https://fedorahosted.org/fedora-websites/ticket/374
[2] https://stg.pagure.io/Fedora_Websites_test

--
Robert Mayr
(robyduck)


--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org
Pagure also has the option to make groups, and everyone have the 
permissions on that group too,


Since creating new git repos in pagure is trivial, one other option 
might be to have sepeerate repos for each web page / project that fedora 
websites works on under a new fedora-web group. That way if i want to 
work on a particular website, i only have to pull down the repo for that 
site, not all the websites. Also, each repo could then have it's own 
neat README in the pagure interface on how to develop on that particular 
website etc.


One drawback (or maybe a postitive depending on what way you look at it) 
with this is that each repo will have a seperate issue tracker.


just a thought,

cheers,
ryanlerch
--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org

Re: Move web repo to pagure - considerations

2016-02-25 Thread Joe Brockmeier 
On 02/25/2016 04:08 AM, Robert Mayr wrote:
> 
> All this said, personally I'd really like to go for a new repository and
> archive all the stuff we have now in the old fedorahosted.org
>  repo. Any opinions about that? Other ideas or
> concerns?
> 
> Thank you, specially for reading my long mail until the end.

I don't do a lot with the web repos, but I have pulled them a few times
- and would love to see a reduction in size for casual contributors. +1
to this!

Best,

jzb
-- 
Joe Brockmeier | Community Team, OSAS
j...@redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/



signature.asc
Description: OpenPGP digital signature
--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org

Re: Move web repo to pagure - considerations

2016-02-25 Thread Gianluca Sforna 
On Thu, Feb 25, 2016 at 10:35 AM, Kévin Raymond
 wrote:
> Also, we should think about git annexe or alike in order to move
> screenshots outside the repository.
> Any idea? (I might have read too fast if you aleady addressed this topic)

I had a decent experience by resorting to submodules for binary,
rarely modified assets

-- 
Gianluca Sforna

http://plus.google.com/+gianlucasforna - http://twitter.com/giallu
Tinker Garage - http://tinkergarage.it
--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org

Re: Move web repo to pagure - considerations

2016-02-25 Thread Gianluca Sforna 
On Thu, Feb 25, 2016 at 10:08 AM, Robert Mayr
 wrote:
>
> All this said, personally I'd really like to go for a new repository and
> archive all the stuff we have now in the old fedorahosted.org repo. Any
> opinions about that? Other ideas or concerns?

Having a cleanup is good, but if keeping history is really a concern,
I suggest looking into git shallow clones.

Anyway, a good summary of the available options is here:
http://blogs.atlassian.com/2014/05/handle-big-repositories-git/


-- 
Gianluca Sforna

http://plus.google.com/+gianlucasforna - http://twitter.com/giallu
Tinker Garage - http://tinkergarage.it
--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org

Re: Move web repo to pagure - considerations

2016-02-25 Thread Robert Mayr 
2016-02-25 10:35 GMT+01:00 Kévin Raymond :

> snip
>
> >
> > 2) We could start a complete new repository, which would give us also the
> > possibility to delete many heavy binaries we are not using at all. We
> would
> > probably end up with an uncompressed size of less than 200MB. There are
> > several cons, but in this specific moment we are able to handle them:
> > * we will loose commit credits and history ^
> > * we cannot build any other branch than master and f24-alpha ^^
> > * all members MUST reclone the repository (has the advantage that it
> would
> > be a clean solution and it will be much faster than now)
> +1
> Just have to include a reference to the archive repository and past
> contributors credits.
>
> Also, we should think about git annexe or alike in order to move
> screenshots outside the repository.
> Any idea? (I might have read too fast if you aleady addressed this topic)
>

I didn't address this topic, but for sure it would be a "nice to have",
although I think it's not mandatory.

-- 
Robert Mayr
(robyduck)
--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org

Re: Move web repo to pagure - considerations

2016-02-25 Thread Kévin Raymond 
snip

>
> 2) We could start a complete new repository, which would give us also the
> possibility to delete many heavy binaries we are not using at all. We would
> probably end up with an uncompressed size of less than 200MB. There are
> several cons, but in this specific moment we are able to handle them:
> * we will loose commit credits and history ^
> * we cannot build any other branch than master and f24-alpha ^^
> * all members MUST reclone the repository (has the advantage that it would
> be a clean solution and it will be much faster than now)
+1
Just have to include a reference to the archive repository and past
contributors credits.

Also, we should think about git annexe or alike in order to move
screenshots outside the repository.
Any idea? (I might have read too fast if you aleady addressed this topic)
--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org

Move web repo to pagure - considerations

2016-02-25 Thread Robert Mayr 
Hi all,
in the last weeks we had some discussions on IRC about moving the
webrepository from fedorahosted to pagure and I also filed a ticket [1] to
collect opinions about this proposal. There are several advantages for us
to move over there and we will start moving with the next release cycle,
which has started these days (we will branch during weekend or next week
probably).
I've set up a testing repository [2] where I also wrote some docs to test
how this would look like. It is working really fine and having a unique
place for all our stuff where we can easily handle PRs is a big enhancement.

At this point there are several opinions about how to move to pagure.
Personally I want to reduce the weight of the web-repo, which is about
660MB or better, 870Mb uncompressed. Not only we have many branches in
there, the reason of all these MBs are static binaries and mainly images we
collected over all the last years.
If we want to solve this problem, we have two options, in both cases all
team members MUST rebase or even reclone the repository:

1) We can run a "git gc --aggressive" on the repo, but a test (thanks nb)
confirmed we would not save too much space. Pros for this solution are:
* people can just rebase and setup the new pagure URL in the git config file
* we keep all history and commit credits
* we can keep also branches where we have ongoing development

2) We could start a complete new repository, which would give us also the
possibility to delete many heavy binaries we are not using at all. We would
probably end up with an uncompressed size of less than 200MB. There are
several cons, but in this specific moment we are able to handle them:
* we will loose commit credits and history ^
* we cannot build any other branch than master and f24-alpha ^^
* all members MUST reclone the repository (has the advantage that it would
be a clean solution and it will be much faster than now)

^ fedorahosted.org would be still there and if needed we can look in the
web UI if we need to find out how things changed in a specific commit.
Also, people can obviously keep the old fedora-web repo locally and use it
for investigations. Commit credits could be saved in a CREDITS file we can
put into the new repo.
^^ Actually all websites are built against master branch and we would build
staging with f24-alpha. Our devel branches are just 2, both of them can be
merged to master without creating conflicts or risks to break our
production websites.

All this said, personally I'd really like to go for a new repository and
archive all the stuff we have now in the old fedorahosted.org repo. Any
opinions about that? Other ideas or concerns?

Thank you, specially for reading my long mail until the end.

[1] https://fedorahosted.org/fedora-websites/ticket/374
[2] https://stg.pagure.io/Fedora_Websites_test

-- 
Robert Mayr
(robyduck)
--
websites mailing list
websites@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/websites@lists.fedoraproject.org