Re: repodata size

2015-10-09 Thread Neal Gompa 
On Fri, Oct 9, 2015 at 7:33 PM, Reindl Harald 
wrote:

>
>
> Am 09.10.2015 um 21:41 schrieb Orion Poplawski:
>
>> On 10/08/2015 01:08 PM, Matthew Miller wrote:
>>
>>>
>>>  From an unrelated practical point of view: consider that the metadata
>>> pulled down so DNF can operate is basically the same size as the entire
>>> (compressed) Fedora Cloud Base image. It'd be very nice to not have
>>> that overhead (but still have wider package set available when you want
>>> it).
>>>
>>>
>> Perhaps every product should produce a os/{repodata,Packages} directory as
>> well as an updates/VERSION/PRODUCT/ tree with
>>
>
> the problem are not only the metadata, but the .solv stuff which did not
> exist before DNF was introduced - what the hell is that, why is it that
> large and how sould fragment the distribution fix that issue?
>
>
​Did you think that the faster dependency resolution came for free in DNF?
Yum used SQLite databases, DNF uses solv data. solv data is generated by
libsolv, which parses the repository data and caches it in a form that can
be rapidly re-read and used for doing repository queries and dependency
resolution.​



-- 
真実はいつも一つ！/ Always, there's only one truth!
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: repodata size

2015-10-09 Thread Kevin Kofler 
Orion Poplawski wrote:
> Perhaps every product should produce a os/{repodata,Packages} directory as
> well as an updates/VERSION/PRODUCT/ tree with .

Please no! Let's not fragment Fedora even more than it already is with those 
"products".

* Would packages belonging to multiple products (kernel, glibc, glib2,
  systemd, NetworkManager etc.) be copied into each of those repositories?
* Where would packages that belong to a non-"product" spin (e.g. KDE) go in
  that plan? Into Workstation? Into a "nonproduct" dumping ground? Neither
  is really an ideal situation. If neither, then we are actually talking
  about a repository per spin, which means a dozen repositories with
  significant overlap.
* And what about niche packages not clearly associated to any "product"?
  Would those also end up in a "nonproduct" dumping ground that is not
  enabled by default on any "product"?

I think there is a lot of value in having a common repository that ensures 
interoperability to the maximum possible extent. Even Ubuntu with their 
separately marketed products (Kubuntu even being released by a separate 
company these days) draws from a shared repository. Let's not throw this 
away.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: repodata size

2015-10-09 Thread Orion Poplawski 

On 10/09/2015 05:20 PM, Kevin Kofler wrote:

Orion Poplawski wrote:

Perhaps every product should produce a os/{repodata,Packages} directory as
well as an updates/VERSION/PRODUCT/ tree with .


Please no! Let's not fragment Fedora even more than it already is with those
"products".

* Would packages belonging to multiple products (kernel, glibc, glib2,
   systemd, NetworkManager etc.) be copied into each of those repositories?


yes, via hard links


* Where would packages that belong to a non-"product" spin (e.g. KDE) go in
   that plan? Into Workstation? Into a "nonproduct" dumping ground? Neither
   is really an ideal situation. If neither, then we are actually talking
   about a repository per spin, which means a dozen repositories with
   significant overlap.


Well, that's essentially what we have now with the Everything repo and 
the whole "updates" repo.



* And what about niche packages not clearly associated to any "product"?
   Would those also end up in a "nonproduct" dumping ground that is not
   enabled by default on any "product"?

I think there is a lot of value in having a common repository that ensures
interoperability to the maximum possible extent. Even Ubuntu with their
separately marketed products (Kubuntu even being released by a separate
company these days) draws from a shared repository. Let's not throw this
away.


I'm not suggesting getting rid of the the everything and full updates 
repos.  But maybe some focused products would benefit from a smaller 
repo set.  I'm also just tossing out the idea - I personally don't have 
any need for it.


--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA DivisionFAX: 303-415-9702
3380 Mitchell Lane  or...@cora.nwra.com
Boulder, CO 80301  http://www.cora.nwra.com
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: repodata size

2015-10-09 Thread Reindl Harald 



Am 09.10.2015 um 21:41 schrieb Orion Poplawski:

On 10/08/2015 01:08 PM, Matthew Miller wrote:


 From an unrelated practical point of view: consider that the metadata
pulled down so DNF can operate is basically the same size as the entire
(compressed) Fedora Cloud Base image. It'd be very nice to not have
that overhead (but still have wider package set available when you want
it).



Perhaps every product should produce a os/{repodata,Packages} directory as
well as an updates/VERSION/PRODUCT/ tree with


the problem are not only the metadata, but the .solv stuff which did not 
exist before DNF was introduced - what the hell is that, why is it that 
large and how sould fragment the distribution fix that issue?


-rw-r- 1 root root 6,1K 2015-10-09 11:44 adobe-linux-x86_64.solv
-rw-r- 1 root root  15M 2015-10-09 11:44 fedora.solv
-rw-r- 1 root root 7,8K 2015-10-09 11:44 google-chrome.solv
-rw-r- 1 root root  15M 2015-10-08 17:31 koji.solv
-rw-r- 1 root root  80K 2015-10-09 11:44 rhsoft-fedora.solv
-rw-r- 1 root root 8,3K 2015-10-09 11:44 rhsoft-generic.solv
-rw-r- 1 root root 244K 2015-10-09 11:44 rpmfusion-free.solv
-rw-r- 1 root root  34K 2015-10-09 11:44 rpmfusion-free-updates.solv
-rw-r- 1 root root 4,8K 2015-10-09 11:44 
rpmfusion-free-updates-testing.solv

-rw-r- 1 root root  80K 2015-10-09 11:44 rpmfusion-nonfree.solv
-rw-r- 1 root root  20K 2015-10-09 11:44 rpmfusion-nonfree-updates.solv
-rw-r- 1 root root 4,8K 2015-10-09 11:44 
rpmfusion-nonfree-updates-testing.solv

-rw-r- 1 root root 2,0M 2015-10-09 12:17 @System.solv
-rw-r- 1 root root 5,6M 2015-10-09 11:44 updates.solv
-rw-r- 1 root root 3,0M 2015-10-09 11:44 updates-testing.solv
-rw-r- 1 root root  612 2015-10-09 11:44 
adobe-linux-x86_64-filenames.solvx

-rw-r- 1 root root  31M 2015-10-09 11:44 fedora-filenames.solvx
-rw-r- 1 root root 1,8K 2015-10-09 11:44 google-chrome-filenames.solvx
-rw-r- 1 root root  30M 2015-10-08 17:29 koji-filenames.solvx
-rw-r- 1 root root 121K 2015-10-09 11:44 rhsoft-fedora-filenames.solvx
-rw-r- 1 root root 2,4K 2015-10-09 11:43 rhsoft-generic-filenames.solvx
-rw-r- 1 root root 335K 2015-10-09 11:44 rpmfusion-free-filenames.solvx
-rw-r- 1 root root  53K 2015-10-09 11:44 
rpmfusion-free-updates-filenames.solvx
-rw-r- 1 root root   82 2015-10-09 11:44 
rpmfusion-free-updates-testing-filenames.solvx
-rw-r- 1 root root 104K 2015-10-09 11:44 
rpmfusion-nonfree-filenames.solvx
-rw-r- 1 root root  20K 2015-10-09 11:44 
rpmfusion-nonfree-updates-filenames.solvx
-rw-r- 1 root root   82 2015-10-09 11:43 
rpmfusion-nonfree-updates-testing-filenames.solvx

-rw-r- 1 root root  13M 2015-10-09 11:44 updates-filenames.solvx
-rw-r- 1 root root 3,3M 2015-10-09 11:43 updates-testing-filenames.solvx
-rw-r- 1 root root 941K 2015-10-09 11:43 
updates-testing-updateinfo.solvx

-rw-r- 1 root root 2,8M 2015-10-09 11:44 updates-updateinfo.solvx



signature.asc
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: repodata size

2015-10-09 Thread Reindl Harald 



Am 10.10.2015 um 01:59 schrieb Neal Gompa:

On Fri, Oct 9, 2015 at 7:33 PM, Reindl Harald

Re: repodata size

2015-10-09 Thread Matthew Miller 
On Fri, Oct 09, 2015 at 01:41:15PM -0600, Orion Poplawski wrote:
> > From an unrelated practical point of view: consider that the metadata
> > pulled down so DNF can operate is basically the same size as the entire
> > (compressed) Fedora Cloud Base image. It'd be very nice to not have
> > that overhead (but still have wider package set available when you want
> > it).
> Perhaps every product should produce a os/{repodata,Packages} directory as
> well as an updates/VERSION/PRODUCT/ tree with .

For the cloud case, I think it'd be nice to have one collection of
packages which are on the image itself, plus another with stuff likely
to be installed (nginx, various language stacks, databases,
caching...), and then maybe everything else.

But, maybe we say "eh, too hard", and just focus on Atomic, which works
entirely differently and therefore doesn't have this issue.

-- 
Matthew Miller

Fedora Project Leader
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: repodata size

2015-10-09 Thread Orion Poplawski 
On 10/08/2015 01:08 PM, Matthew Miller wrote:
> 
> From an unrelated practical point of view: consider that the metadata
> pulled down so DNF can operate is basically the same size as the entire
> (compressed) Fedora Cloud Base image. It'd be very nice to not have
> that overhead (but still have wider package set available when you want
> it).
> 

Perhaps every product should produce a os/{repodata,Packages} directory as
well as an updates/VERSION/PRODUCT/ tree with .


-- 
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane   or...@nwra.com
Boulder, CO 80301   http://www.nwra.com
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct