Re: Announcing DNF 3 development

[Kevin Kofler]

Matěj Cepl wrote:
> When switching the programming langauge than I would think there
> are some better C-successors than C++, namely Rust? Mad rush of
> giving up on 46 years old language and switching to one which is
> just 33 years old seems a bit bizarre to me.

But the "33 years old" language WORKS. :-) As for why switching from C to 
C++, that's because C++ has some features that C does not have and that are 
useful for the DNF developers, in particular, native support for object 
orientation (without the GObject hack).

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Neal Gompa]

On Mon, Mar 26, 2018 at 12:10 PM, Martin Kolman  wrote:
> On Mon, 2018-03-26 at 11:59 -0400, Neal Gompa wrote:
>> On Mon, Mar 26, 2018 at 11:02 AM, Vít Ondruch  wrote:
>> >
>> >
>> > Dne 26.3.2018 v 10:16 Tom Hughes napsal(a):
>> > > On 26/03/18 09:06, Marcin Juszkiewicz wrote:
>> > > > W dniu 22.03.2018 o 13:40, Daniel Mach pisze:
>> > > > > We are pleased to announce that development of DNF 3 has started. 
>> > > > > This
>> > > > > version is focused on performance improvements, new API and
>> > > > > consolidating
>> > > > > the whole software management stack.
>> > > > >
>> > > > > Please read more details on our blog:
>> > > > > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>> > > > >
>> > > >
>> > > > Will it FINALLY use one copy of metadata for all system users?
>> > >
>> > > Do you have a proposal as to how that might be possible in a
>> > > secure way?
>> >
>> > If DNF was just frontend for some service/daemon, that would be one
>> > possibility. It would also help with other issues like updates of X
>> > server crashing whole user session and therefore the update.
>> >
>>
>> It would be nice if dnfdaemon was actually merged into dnf itself, but
>> I'm not sure whether they'd consider it to be desirable or not.
> I have a hunch there will be some gotchas, otherwise everybody would be doing 
> it.
>
> Maybe issues with chrooting or something like that ?

We'd probably want a no-daemon switch for changing behavior, and
trigger that change for circumstances where the daemon cannot run.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Colin Walters]

On Mon, Mar 26, 2018, at 11:02 AM, Vít Ondruch wrote:

> If DNF was just frontend for some service/daemon, that would be one
> possibility. It would also help with other issues like updates of X
> server crashing whole user session and therefore the update.

FWIW rpm-ostree is always a daemon today; we're really focused on the
"host system" management case so we can just assume that.  Obviously
there's PackageKit as well.

That said I spent a while thinking about this a while ago, and the thing
is a vast array of tooling assumes that package managers are CLI tools.
Not least e.g. `Dockerfile` style container builds.  And `mock`.
So you'd really end up with something that splits up the "managing a host
system" from "buildroot tool", which is how I think of rpm-ostreed
versus dnf today.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Martin Kolman]

On Mon, 2018-03-26 at 11:59 -0400, Neal Gompa wrote:
> On Mon, Mar 26, 2018 at 11:02 AM, Vít Ondruch  wrote:
> > 
> > 
> > Dne 26.3.2018 v 10:16 Tom Hughes napsal(a):
> > > On 26/03/18 09:06, Marcin Juszkiewicz wrote:
> > > > W dniu 22.03.2018 o 13:40, Daniel Mach pisze:
> > > > > We are pleased to announce that development of DNF 3 has started. This
> > > > > version is focused on performance improvements, new API and
> > > > > consolidating
> > > > > the whole software management stack.
> > > > > 
> > > > > Please read more details on our blog:
> > > > > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
> > > > > 
> > > > 
> > > > Will it FINALLY use one copy of metadata for all system users?
> > > 
> > > Do you have a proposal as to how that might be possible in a
> > > secure way?
> > 
> > If DNF was just frontend for some service/daemon, that would be one
> > possibility. It would also help with other issues like updates of X
> > server crashing whole user session and therefore the update.
> > 
> 
> It would be nice if dnfdaemon was actually merged into dnf itself, but
> I'm not sure whether they'd consider it to be desirable or not.
I have a hunch there will be some gotchas, otherwise everybody would be doing 
it.

Maybe issues with chrooting or something like that ?
> 
> 
> -- 
> 真実はいつも一つ！/ Always, there's only one truth!
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Neal Gompa]

On Mon, Mar 26, 2018 at 11:02 AM, Vít Ondruch  wrote:
>
>
> Dne 26.3.2018 v 10:16 Tom Hughes napsal(a):
>> On 26/03/18 09:06, Marcin Juszkiewicz wrote:
>>> W dniu 22.03.2018 o 13:40, Daniel Mach pisze:
 We are pleased to announce that development of DNF 3 has started. This
 version is focused on performance improvements, new API and
 consolidating
 the whole software management stack.

 Please read more details on our blog:
 https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/

>>>
>>> Will it FINALLY use one copy of metadata for all system users?
>>
>> Do you have a proposal as to how that might be possible in a
>> secure way?
>
> If DNF was just frontend for some service/daemon, that would be one
> possibility. It would also help with other issues like updates of X
> server crashing whole user session and therefore the update.
>

It would be nice if dnfdaemon was actually merged into dnf itself, but
I'm not sure whether they'd consider it to be desirable or not.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Nicolas Mailhot]

Le lundi 26 mars 2018 à 17:02 +0200, Vít Ondruch a écrit :
> 
> If DNF was just frontend for some service/daemon, that would be one
> possibility. It would also help with other issues like updates of X
> server crashing whole user session and therefore the update.

Yes that would be pretty awesome

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Vít Ondruch]

Dne 26.3.2018 v 10:16 Tom Hughes napsal(a):
> On 26/03/18 09:06, Marcin Juszkiewicz wrote:
>> W dniu 22.03.2018 o 13:40, Daniel Mach pisze:
>>> We are pleased to announce that development of DNF 3 has started. This
>>> version is focused on performance improvements, new API and
>>> consolidating
>>> the whole software management stack.
>>>
>>> Please read more details on our blog:
>>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>>>
>>
>> Will it FINALLY use one copy of metadata for all system users?
>
> Do you have a proposal as to how that might be possible in a
> secure way?

If DNF was just frontend for some service/daemon, that would be one
possibility. It would also help with other issues like updates of X
server crashing whole user session and therefore the update.

V.


>
>> I do not see how fetching megabytes of metadata is better than using
>> copy present in /var/cache/dnf/ directory.
>
> Well it could just use the cached copy sure, but if it was out of
> date then it wouldn't be able to update it?
>
>> It is faster to enter long password to use sudo than to wait until dnf
>> fetch useless copy of metadata.
>
> Agreed, which is why I always run dnf under sudo even for query
> operations.
>
>> Debian has APT. APT uses one copy of metadata. Be wise. Like Debian.
>
> Do we know how? Do they just not allow non-root users to get up
> to date data, or do they do something cleverer?
>
> Tom
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Neal Gompa]

On Mon, Mar 26, 2018 at 9:58 AM, Martin Sehnoutka  wrote:
>
>
> On 03/26/2018 02:30 PM, Neal Gompa wrote:
>> On Mon, Mar 26, 2018 at 8:26 AM, Martin Sehnoutka  
>> wrote:
>>>
>>>
>>> On 03/26/2018 01:38 PM, Neal Gompa wrote:
 On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl  wrote:
> On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
>> On 03/22/2018 01:40 PM, Daniel Mach wrote:
>>> Please read more details on our blog:
>>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>>
>> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should
>> use Developer Toolset to compile on Red Hat Enterprise Linux
>> 7 if you need C++11 support.  The system compiler, GCC 4.8,
>> has limited support only.
>
> When switching the programming langauge than I would think there
> are some better C-successors than C++, namely Rust? Mad rush of
> giving up on 46 years old language and switching to one which is
> just 33 years old seems a bit bizarre to me.
>
>>>
>>> Take a look into the code, it is mostly C with few features from C++.
>>>
>>> btw what is the motivation to use GOBjects? Is the libdnf api supposed
>>> to be consumed by dnf frontend via gi repository?
>>>
>>
>> It was a thought a while ago with libhif, and as part of the final
>> rationalization for libdnf, it's being dropped. Because libdnf is
>> going to be in C++, it's going to use SWIG for bindings generation.
>>
>
> Thanks for clarification.
>

 I'm okay with not dealing with LLVM for my system package manager,
 thank you very much. I'd be more open to Rust if Rust also could be
 built with GCC, and thus supported across literally everything, but no
 one is investing in that effort.

>>>
>>> Well, investment like this will need some justification, not saying that
>>> dnf should be the one, but you will definitely need a big, important
>>> project.
>>>
>>
>> Considering all the other "big important things" people don't invest
>> in anyway, I don't think that'd help any.
>>
 And frankly, Rust is harder to program in than C++, and creating
 bindings is no walk in the park.

>>>
>>> Purely personal opinion. You are probably referring to the learning
>>> curve, which is known to be steep, but after this period it is well
>>> worth the effort.
>>>
>>
>> Not my personal opinion. That's the opinion of several developers I
>> know who are working on Rust based projects. Not everyone gets the
>> benefit of GNOME forcing all the things so that stuff _must_ work.
>>
>
> I don't really get the last sentence. What is GNOME forcing a what must
> work?
>

Basically, when you work outside of the GNOME ecosystem, things get
much harder because you can't guarantee everything interfaces through
GObject and other stuff like it.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Martin Sehnoutka]

On 03/26/2018 02:30 PM, Neal Gompa wrote:
> On Mon, Mar 26, 2018 at 8:26 AM, Martin Sehnoutka  wrote:
>>
>>
>> On 03/26/2018 01:38 PM, Neal Gompa wrote:
>>> On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl  wrote:
 On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
> On 03/22/2018 01:40 PM, Daniel Mach wrote:
>> Please read more details on our blog:
>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>
> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should
> use Developer Toolset to compile on Red Hat Enterprise Linux
> 7 if you need C++11 support.  The system compiler, GCC 4.8,
> has limited support only.

 When switching the programming langauge than I would think there
 are some better C-successors than C++, namely Rust? Mad rush of
 giving up on 46 years old language and switching to one which is
 just 33 years old seems a bit bizarre to me.

>>
>> Take a look into the code, it is mostly C with few features from C++.
>>
>> btw what is the motivation to use GOBjects? Is the libdnf api supposed
>> to be consumed by dnf frontend via gi repository?
>>
> 
> It was a thought a while ago with libhif, and as part of the final
> rationalization for libdnf, it's being dropped. Because libdnf is
> going to be in C++, it's going to use SWIG for bindings generation.
> 

Thanks for clarification.

>>>
>>> I'm okay with not dealing with LLVM for my system package manager,
>>> thank you very much. I'd be more open to Rust if Rust also could be
>>> built with GCC, and thus supported across literally everything, but no
>>> one is investing in that effort.
>>>
>>
>> Well, investment like this will need some justification, not saying that
>> dnf should be the one, but you will definitely need a big, important
>> project.
>>
> 
> Considering all the other "big important things" people don't invest
> in anyway, I don't think that'd help any.
> 
>>> And frankly, Rust is harder to program in than C++, and creating
>>> bindings is no walk in the park.
>>>
>>
>> Purely personal opinion. You are probably referring to the learning
>> curve, which is known to be steep, but after this period it is well
>> worth the effort.
>>
> 
> Not my personal opinion. That's the opinion of several developers I
> know who are working on Rust based projects. Not everyone gets the
> benefit of GNOME forcing all the things so that stuff _must_ work.
> 

I don't really get the last sentence. What is GNOME forcing a what must
work?

>> Regarding the bindings, if libdnf is meant to be used via gir (see my
>> question above), then there is already an effort to make this much
>> easier (I'm referring to gnome-class).
>>
> 
> As I noted earlier in this email, gir is a leftover and is being removed.
> 

-- 
Martin Sehnoutka | Associate Software Engineer
PGP: 5FD64AF5
UTC+1 (CET)
RED HAT | TRIED. TESTED. TRUSTED.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Neal Gompa]

On Mon, Mar 26, 2018 at 8:26 AM, Martin Sehnoutka  wrote:
>
>
> On 03/26/2018 01:38 PM, Neal Gompa wrote:
>> On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl  wrote:
>>> On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
 On 03/22/2018 01:40 PM, Daniel Mach wrote:
> Please read more details on our blog:
> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/

 “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should
 use Developer Toolset to compile on Red Hat Enterprise Linux
 7 if you need C++11 support.  The system compiler, GCC 4.8,
 has limited support only.
>>>
>>> When switching the programming langauge than I would think there
>>> are some better C-successors than C++, namely Rust? Mad rush of
>>> giving up on 46 years old language and switching to one which is
>>> just 33 years old seems a bit bizarre to me.
>>>
>
> Take a look into the code, it is mostly C with few features from C++.
>
> btw what is the motivation to use GOBjects? Is the libdnf api supposed
> to be consumed by dnf frontend via gi repository?
>

It was a thought a while ago with libhif, and as part of the final
rationalization for libdnf, it's being dropped. Because libdnf is
going to be in C++, it's going to use SWIG for bindings generation.

>>
>> I'm okay with not dealing with LLVM for my system package manager,
>> thank you very much. I'd be more open to Rust if Rust also could be
>> built with GCC, and thus supported across literally everything, but no
>> one is investing in that effort.
>>
>
> Well, investment like this will need some justification, not saying that
> dnf should be the one, but you will definitely need a big, important
> project.
>

Considering all the other "big important things" people don't invest
in anyway, I don't think that'd help any.

>> And frankly, Rust is harder to program in than C++, and creating
>> bindings is no walk in the park.
>>
>
> Purely personal opinion. You are probably referring to the learning
> curve, which is known to be steep, but after this period it is well
> worth the effort.
>

Not my personal opinion. That's the opinion of several developers I
know who are working on Rust based projects. Not everyone gets the
benefit of GNOME forcing all the things so that stuff _must_ work.

> Regarding the bindings, if libdnf is meant to be used via gir (see my
> question above), then there is already an effort to make this much
> easier (I'm referring to gnome-class).
>

As I noted earlier in this email, gir is a leftover and is being removed.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Martin Kolman]

On Mon, 2018-03-26 at 13:22 +0200, Matěj Cepl wrote:
> On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
> > On 03/22/2018 01:40 PM, Daniel Mach wrote:
> > > Please read more details on our blog:
> > > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
> > 
> > “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should 
> > use Developer Toolset to compile on Red Hat Enterprise Linux 
> > 7 if you need C++11 support.  The system compiler, GCC 4.8, 
> > has limited support only.
> 
> When switching the programming langauge than I would think there 
> are some better C-successors than C++, namely Rust? Mad rush of 
> giving up on 46 years old language and switching to one which is 
> just 33 years old seems a bit bizarre to me.
I think it is not bad to be a bit conservative when core system components
are concerned.

> 
> Best,
> 
> Matěj
> -- 
> https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
> GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8
>  
> I am a Roman Catholic, so that I do not expect `history' to be
> anything but a `long defeat' -- though it contains (and in
> a legend may contain more clearly and movingly) some samples or
> glimpses of final victory.
>   -- J.R.R. Tolkien
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Martin Sehnoutka]

On 03/26/2018 01:38 PM, Neal Gompa wrote:
> On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl  wrote:
>> On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
>>> On 03/22/2018 01:40 PM, Daniel Mach wrote:
 Please read more details on our blog:
 https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>>>
>>> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should
>>> use Developer Toolset to compile on Red Hat Enterprise Linux
>>> 7 if you need C++11 support.  The system compiler, GCC 4.8,
>>> has limited support only.
>>
>> When switching the programming langauge than I would think there
>> are some better C-successors than C++, namely Rust? Mad rush of
>> giving up on 46 years old language and switching to one which is
>> just 33 years old seems a bit bizarre to me.
>>

Take a look into the code, it is mostly C with few features from C++.

btw what is the motivation to use GOBjects? Is the libdnf api supposed
to be consumed by dnf frontend via gi repository?

> 
> I'm okay with not dealing with LLVM for my system package manager,
> thank you very much. I'd be more open to Rust if Rust also could be
> built with GCC, and thus supported across literally everything, but no
> one is investing in that effort.
> 

Well, investment like this will need some justification, not saying that
dnf should be the one, but you will definitely need a big, important
project.

> And frankly, Rust is harder to program in than C++, and creating
> bindings is no walk in the park.
> 

Purely personal opinion. You are probably referring to the learning
curve, which is known to be steep, but after this period it is well
worth the effort.

Regarding the bindings, if libdnf is meant to be used via gir (see my
question above), then there is already an effort to make this much
easier (I'm referring to gnome-class).

-- 
Martin Sehnoutka | Associate Software Engineer
PGP: 5FD64AF5
UTC+1 (CET)
RED HAT | TRIED. TESTED. TRUSTED.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Neal Gompa]

On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl  wrote:
> On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
>> On 03/22/2018 01:40 PM, Daniel Mach wrote:
>>> Please read more details on our blog:
>>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>>
>> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should
>> use Developer Toolset to compile on Red Hat Enterprise Linux
>> 7 if you need C++11 support.  The system compiler, GCC 4.8,
>> has limited support only.
>
> When switching the programming langauge than I would think there
> are some better C-successors than C++, namely Rust? Mad rush of
> giving up on 46 years old language and switching to one which is
> just 33 years old seems a bit bizarre to me.
>

I'm okay with not dealing with LLVM for my system package manager,
thank you very much. I'd be more open to Rust if Rust also could be
built with GCC, and thus supported across literally everything, but no
one is investing in that effort.

And frankly, Rust is harder to program in than C++, and creating
bindings is no walk in the park.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Matěj Cepl]

On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
> On 03/22/2018 01:40 PM, Daniel Mach wrote:
>> Please read more details on our blog:
>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>
> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should 
> use Developer Toolset to compile on Red Hat Enterprise Linux 
> 7 if you need C++11 support.  The system compiler, GCC 4.8, 
> has limited support only.

When switching the programming langauge than I would think there 
are some better C-successors than C++, namely Rust? Mad rush of 
giving up on 46 years old language and switching to one which is 
just 33 years old seems a bit bizarre to me.

Best,

Matěj
-- 
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8
 
I am a Roman Catholic, so that I do not expect `history' to be
anything but a `long defeat' -- though it contains (and in
a legend may contain more clearly and movingly) some samples or
glimpses of final victory.
  -- J.R.R. Tolkien
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Florian Weimer]

On 03/22/2018 01:40 PM, Daniel Mach wrote:

We are pleased to announce that development of DNF 3 has started. This
version is focused on performance improvements, new API and consolidating
the whole software management stack.

Please read more details on our blog:
https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/


“C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should use 
Developer Toolset to compile on Red Hat Enterprise Linux 7 if you need 
C++11 support.  The system compiler, GCC 4.8, has limited support only.


Thanks,
Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Jonathan Dieter]

On Mon, 2018-03-26 at 11:44 +0300, Jonathan Dieter wrote:
> I've been working on zchunk which will allow downloading only the delta
>  of the metadata.

For reference, the proposal starts at http://lists.rpm.org/pipermail/rp
m-ecosystem/2018-February/000534.html.

Jonathan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Benjamin Kircher]

> On 26. Mar 2018, at 10:39, Oron Peled  wrote:
> 
> On Monday, 26 March 2018 11:16:14 IDT Tom Hughes wrote:
>> On 26/03/18 09:06, Marcin Juszkiewicz wrote:
>>> Debian has APT. APT uses one copy of metadata. Be wise. Like Debian.
>> 
>> Do we know how? Do they just not allow non-root users to get up
>> to date data, or do they do something cleverer?
> 
> With APT, cache update and using the cache are orthogonal operations:
> * Running "apt-get update" is privileged and update the cache.
> * There's obviously an optional service to run "apt-get update" periodically 
> if root prefer this mechanism.
> * Doing just query (e.g: apt list ...) always use latest cache data.
> * So query by root or any user are the same -- no privileged and work on the 
> cache (which may/may-not be updated).

Question: Would splitting-up cache update and package upgrade operations worth 
considering for dnf 3.0?


(As a long time CentOS, Fedora user I am always envious at how snappy (no pun 
intended) apt-get update && apt-get upgrade feel when I get my hands on one of 
these :-)

BK
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Jonathan Dieter]

On Mon, 2018-03-26 at 09:30 +0100, Richard Hughes wrote:
> With version 3.0 dnf is switching to the codebase we originally wrote
> for PackageKit (libzif->libhif->libdnf), and so it inherits the
> "download complete cache and switch atomically" logic. This means we
> get a shared cache for free.

I've been working on zchunk which will allow downloading only the delta
 of the metadata.  AIUI, currently dnf uses librepo to download the
metadata, so that's where I was planning to do the zchunk integration. 
Is there somewhere else I should be looking at for dnf 3.0, or is it
still using librepo to do the actual downloads?

Jonathan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Oron Peled]

On Monday, 26 March 2018 11:16:14 IDT Tom Hughes wrote:
> On 26/03/18 09:06, Marcin Juszkiewicz wrote:
> > Debian has APT. APT uses one copy of metadata. Be wise. Like Debian.
> 
> Do we know how? Do they just not allow non-root users to get up
> to date data, or do they do something cleverer?

With APT, cache update and using the cache are orthogonal operations:
 * Running "apt-get update" is privileged and update the cache.
 * There's obviously an optional service to run "apt-get update" periodically 
if root prefer this mechanism.
 * Doing just query (e.g: apt list ...) always use latest cache data.
 * So query by root or any user are the same -- no privileged and work on the 
cache (which may/may-not be updated).


KISS,

-- 
Oron Peled Voice: +972-4-8228492

May the Source be with you!


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Tom Hughes]

On 26/03/18 09:30, Richard Hughes wrote:

On 26 March 2018 at 09:16, Tom Hughes  wrote:

Will it FINALLY use one copy of metadata for all system users?

Do you have a proposal as to how that might be possible in a
secure way?


I think he means removing the duplication of the cache between
PackageKit and dnf. The former only uses a separate cache as it
updates the cache all at one, and switches to new versions atomically
so applications like gnome-software can always run transactions with
known latencies. dnf (the command line tool) only downloaded the
metadata files it needed for the current request, and so it could be
you had to start downloading something like the filelists when you
actually depsolved a transaction (which is somewhat incompatible with
offline updates for instance).


Well that's an issue as well, but he specifically referred to the
per-user caching behaviour of dnf and the need to run it under sudo
to use the system cache instead of building a new one.

In other words if I do "dnf search foo" as myself it will fetch a
new cache from scratch into /tmp instead of using the system one.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Marcin Juszkiewicz]

W dniu 26.03.2018 o 10:16, Tom Hughes pisze:
> On 26/03/18 09:06, Marcin Juszkiewicz wrote:

>> Will it FINALLY use one copy of metadata for all system users?
> 
> Do you have a proposal as to how that might be possible in a
> secure way?

Use. Not download. And inform (like always) how old that data is.

>> I do not see how fetching megabytes of metadata is better than using
>> copy present in /var/cache/dnf/ directory.
> 
> Well it could just use the cached copy sure, but if it was out of
> date then it wouldn't be able to update it?

Then it is out-of-date. But still is. And not everyone runs rawhide to
make it a difference when metadata is few days old.

>> It is faster to enter long password to use sudo than to wait until dnf
>> fetch useless copy of metadata.
> 
> Agreed, which is why I always run dnf under sudo even for query
> operations.

I hate having to do that. Also dislike fact that if I do not do that
then have to wait and wait and wait until it finally fetch metadata to
show me result.

DNF started as user can not install packages so why it can not just use
systemwide metadata?

>> Debian has APT. APT uses one copy of metadata. Be wise. Like Debian.
> 
> Do we know how? Do they just not allow non-root users to get up
> to date data, or do they do something cleverer?

APT has separate command for updating metadata. YUM/DNF tries to do that
every time they think that metadata is too old.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Richard Hughes]

On 26 March 2018 at 09:16, Tom Hughes  wrote:
>> Will it FINALLY use one copy of metadata for all system users?
> Do you have a proposal as to how that might be possible in a
> secure way?

I think he means removing the duplication of the cache between
PackageKit and dnf. The former only uses a separate cache as it
updates the cache all at one, and switches to new versions atomically
so applications like gnome-software can always run transactions with
known latencies. dnf (the command line tool) only downloaded the
metadata files it needed for the current request, and so it could be
you had to start downloading something like the filelists when you
actually depsolved a transaction (which is somewhat incompatible with
offline updates for instance).

With version 3.0 dnf is switching to the codebase we originally wrote
for PackageKit (libzif->libhif->libdnf), and so it inherits the
"download complete cache and switch atomically" logic. This means we
get a shared cache for free.

Richard.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Tom Hughes]

On 26/03/18 09:06, Marcin Juszkiewicz wrote:

W dniu 22.03.2018 o 13:40, Daniel Mach pisze:

We are pleased to announce that development of DNF 3 has started. This
version is focused on performance improvements, new API and consolidating
the whole software management stack.

Please read more details on our blog:
https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/


Will it FINALLY use one copy of metadata for all system users?


Do you have a proposal as to how that might be possible in a
secure way?


I do not see how fetching megabytes of metadata is better than using
copy present in /var/cache/dnf/ directory.


Well it could just use the cached copy sure, but if it was out of
date then it wouldn't be able to update it?


It is faster to enter long password to use sudo than to wait until dnf
fetch useless copy of metadata.


Agreed, which is why I always run dnf under sudo even for query
operations.


Debian has APT. APT uses one copy of metadata. Be wise. Like Debian.


Do we know how? Do they just not allow non-root users to get up
to date data, or do they do something cleverer?

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Marcin Juszkiewicz]

W dniu 22.03.2018 o 13:40, Daniel Mach pisze:
> We are pleased to announce that development of DNF 3 has started. This
> version is focused on performance improvements, new API and consolidating
> the whole software management stack.
> 
> Please read more details on our blog:
> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/

Will it FINALLY use one copy of metadata for all system users?

I do not see how fetching megabytes of metadata is better than using
copy present in /var/cache/dnf/ directory.

It is faster to enter long password to use sudo than to wait until dnf
fetch useless copy of metadata.

10:01 (1s) hrw@puchatek:~$ LANGUAGE=C LANG=C COLUMNS=60  dnf list nano
Spotify (negativo17) 31 kB/s |  11 kB 00:00
Fedora 27 - x86_64 - Update 8.4 MB/s |  21 MB 00:02
Fedora 27 - x86_64   15 MB/s |  58 MB 00:03
Google Chrome (stable)   57 kB/s | 3.7 kB 00:00
google-earth 72 kB/s | 4.7 kB 00:00
RPM Fusion for Fedora 27 -  984 kB/s | 351 kB 00:00
RPM Fusion for Fedora 27 -  1.3 MB/s | 717 kB 00:00
RPM Fusion for Fedora 27 -   50 kB/s |  87 kB 00:01
RPM Fusion for Fedora 27 -  128 kB/s | 163 kB 00:01
Skype Repository 40 kB/s | 1.8 kB 00:00
Last metadata expiration check: 0:00:00 ago on Mon Mar 26 10:03:01 2018.
Available Packages
nano.x86_64   2.8.7-1.fc27fedora
10:02 (40s) hrw@puchatek:~$

40 seconds just because of good download speed.


Debian has APT. APT uses one copy of metadata. Be wise. Like Debian.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Adam Williamson]

On Fri, 2018-03-23 at 10:55 -0500, Ian Pilcher wrote:
> On 03/22/2018 07:40 AM, Daniel Mach wrote:
> > We are pleased to announce that development of DNF 3 has started. This 
> > version is focused on performance improvements, new API and 
> > consolidating the whole software management stack.
> > 
> > Please read more details on our blog:
> > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
> 
> Can someone explain how DNF 2 can be considered "finished", when it
> still can't provide the dependency information that yum did?

Neither the mail nor the announcement makes any claim that DNF 2 is
considered "finished".
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Ian Pilcher]

On 03/22/2018 07:40 AM, Daniel Mach wrote:
We are pleased to announce that development of DNF 3 has started. This 
version is focused on performance improvements, new API and 
consolidating the whole software management stack.


Please read more details on our blog:
https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/


Can someone explain how DNF 2 can be considered "finished", when it
still can't provide the dependency information that yum did?

  https://bugzilla.redhat.com/show_bug.cgi?id=1549851

--

Ian Pilcher arequip...@gmail.com
 "I grew up before Mark Zuckerberg invented friendship" 

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Fabio Valentini]

On Thu, Mar 22, 2018, 23:13 Nico Kadel-Garcia  wrote:

> On Thu, Mar 22, 2018 at 5:49 PM, John Reiser  wrote:
> > On 03/22/2018 01:51 PM, Nico Kadel-Garcia wrote:
> >>
> >> On Thu, Mar 22, 2018 at 10:52 AM, John Reiser 
> >> wrote:
> >>>
> >>> On 03/22/2018 05:40 AM, Daniel Mach wrote:
> 
> 
>  We are pleased to announce that development of DNF 3 has started. This
>  version is focused on performance improvements, new API and
>  consolidating
>  the whole software management stack.
> >>>
> >>>
> >>>
> >>> How does RPM fit into DNF's view of "the whole software management
> >>> stack"?
> >>> RPM is a slug (moves very slowly): no parallelism (at any point all
> >>> packages
> >>> with no remaining predecessors could be updated/installed in parallel),
> >>> not even manually pipelined (decompress to memory, manipulate
> filesystem,
> >>> update database.)
> >>
> >>
> >> Parallelizing software updates or installations would be *begging* for
> >> pain. It would be difficult for me to recommend strongly enough
> >> against this.
> >
> >
> > Please be specific about the pain points that you fear.
>
> RPM, itself, is single threaded.
>
> %pre and %post operations would have to be re-evaluated for
> parallelization. system account creation, in particular, would have to
> be made thread safe.
>
> RPM installation can fail partly through deployment due to SELinux,
> disk space, or network based mount point failure: keeping it single
> threaded makes it much safer to unravel failed or partial RPM
> installation.
>
> Unweaving partial dependency deployment could be quite destructive
> with a parallelized approach.
>
> Daemons that need to be restarted and may have incompatible component
> updates, such httpd with its modules, are particularly vulnerable to
> fascinating failures from the daemon restarting with only some updated
> components. Avoiding that would seem to require even more dependency
> management for RPM installation, rather than each update itself
> triggering an update.
>
> > The three-stage "manual" pipeline achieves 2x to 3x faster throughput
> > with error states that are isomorphic to present RPM.  (Consider the
> > Turning machine model: if you don't write to the filesystem, then
> > there is no change of external state.)
>
> Turing machines don't have to deal with all the possible
>
> > The "parallelize everything that has no remaining predecessors" strategy
> > requires parallel transactions in the database (they cannot interfere
> > because that would be a predecessor constraint) and checking for
> > resource exhaustion (file space, inodes, etc.) as a global
> > predecessor constraint.  What else?
>
> Parallelizing the installations means losing the milestones at which
> one update has succeeded, and the second update has not. Unweaving
> that to find out which update triggered the failure sounds like pain,
> and makes testing the update process more difficult. It becomes
> difficult to manage or guess what the state of the system was at the
> time of the RPM update, since another RPM update may be in progress at
> the time.
>
> There is an infamous quote by Donald Knuth that "premature
> optimization is the root of all evil". There are systems that benefit
> the time benefits of parallelization, but for ordinary RPM
> installations and system updates, I think that the slow update time is
> because of other factors, such as disk IO and download time of
> repodata, RPM database updates, and download times for the packages.
>

Well ... if you want to take this argument to the extreme, an update
process where the update is downloaded and prepared in the background, and
then applied atomically, would be the most "efficient" (because it happens
instantaneous, with a reboot) and resilient against errors (at no point in
time there is a broken, half-updated system) - just like rpm-ostree does
things ...

*ducks*

___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Nico Kadel-Garcia]

On Thu, Mar 22, 2018 at 5:49 PM, John Reiser  wrote:
> On 03/22/2018 01:51 PM, Nico Kadel-Garcia wrote:
>>
>> On Thu, Mar 22, 2018 at 10:52 AM, John Reiser 
>> wrote:
>>>
>>> On 03/22/2018 05:40 AM, Daniel Mach wrote:


 We are pleased to announce that development of DNF 3 has started. This
 version is focused on performance improvements, new API and
 consolidating
 the whole software management stack.
>>>
>>>
>>>
>>> How does RPM fit into DNF's view of "the whole software management
>>> stack"?
>>> RPM is a slug (moves very slowly): no parallelism (at any point all
>>> packages
>>> with no remaining predecessors could be updated/installed in parallel),
>>> not even manually pipelined (decompress to memory, manipulate filesystem,
>>> update database.)
>>
>>
>> Parallelizing software updates or installations would be *begging* for
>> pain. It would be difficult for me to recommend strongly enough
>> against this.
>
>
> Please be specific about the pain points that you fear.

RPM, itself, is single threaded.

%pre and %post operations would have to be re-evaluated for
parallelization. system account creation, in particular, would have to
be made thread safe.

RPM installation can fail partly through deployment due to SELinux,
disk space, or network based mount point failure: keeping it single
threaded makes it much safer to unravel failed or partial RPM
installation.

Unweaving partial dependency deployment could be quite destructive
with a parallelized approach.

Daemons that need to be restarted and may have incompatible component
updates, such httpd with its modules, are particularly vulnerable to
fascinating failures from the daemon restarting with only some updated
components. Avoiding that would seem to require even more dependency
management for RPM installation, rather than each update itself
triggering an update.

> The three-stage "manual" pipeline achieves 2x to 3x faster throughput
> with error states that are isomorphic to present RPM.  (Consider the
> Turning machine model: if you don't write to the filesystem, then
> there is no change of external state.)

Turing machines don't have to deal with all the possible

> The "parallelize everything that has no remaining predecessors" strategy
> requires parallel transactions in the database (they cannot interfere
> because that would be a predecessor constraint) and checking for
> resource exhaustion (file space, inodes, etc.) as a global
> predecessor constraint.  What else?

Parallelizing the installations means losing the milestones at which
one update has succeeded, and the second update has not. Unweaving
that to find out which update triggered the failure sounds like pain,
and makes testing the update process more difficult. It becomes
difficult to manage or guess what the state of the system was at the
time of the RPM update, since another RPM update may be in progress at
the time.

There is an infamous quote by Donald Knuth that "premature
optimization is the root of all evil". There are systems that benefit
the time benefits of parallelization, but for ordinary RPM
installations and system updates, I think that the slow update time is
because of other factors, such as disk IO and download time of
repodata, RPM database updates, and download times for the packages.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Stephen John Smoogen]

On 22 March 2018 at 17:49, John Reiser  wrote:
> On 03/22/2018 01:51 PM, Nico Kadel-Garcia wrote:
>>
>> On Thu, Mar 22, 2018 at 10:52 AM, John Reiser 
>> wrote:
>>>
>>> On 03/22/2018 05:40 AM, Daniel Mach wrote:


 We are pleased to announce that development of DNF 3 has started. This
 version is focused on performance improvements, new API and
 consolidating
 the whole software management stack.
>>>
>>>
>>>
>>> How does RPM fit into DNF's view of "the whole software management
>>> stack"?
>>> RPM is a slug (moves very slowly): no parallelism (at any point all
>>> packages
>>> with no remaining predecessors could be updated/installed in parallel),
>>> not even manually pipelined (decompress to memory, manipulate filesystem,
>>> update database.)
>>
>>
>> Parallelizing software updates or installations would be *begging* for
>> pain. It would be difficult for me to recommend strongly enough
>> against this.
>
>
> Please be specific about the pain points that you fear.
>
> The three-stage "manual" pipeline achieves 2x to 3x faster throughput
> with error states that are isomorphic to present RPM.  (Consider the
> Turning machine model: if you don't write to the filesystem, then
> there is no change of external state.)
>
> The "parallelize everything that has no remaining predecessors" strategy
> requires parallel transactions in the database (they cannot interfere
> because that would be a predecessor constraint) and checking for
> resource exhaustion (file space, inodes, etc.) as a global
> predecessor constraint.  What else?
>

Having some way for triggers/postun/pre etc scripts to know they
actually interfere with a predecessor constraint. This would mean
itemizing everything in every script which could be run during these
steps and working out blocks/conflicts/etc. [AKA if bash is not there
when a parallel scriplet runs... boom.]

[Not saying it is impossible.. but it might mean a multipass update
where certain things are updated, then the next set and then the third
... ]



> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org



-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[John Reiser]

On 03/22/2018 01:51 PM, Nico Kadel-Garcia wrote:

On Thu, Mar 22, 2018 at 10:52 AM, John Reiser  wrote:

On 03/22/2018 05:40 AM, Daniel Mach wrote:


We are pleased to announce that development of DNF 3 has started. This
version is focused on performance improvements, new API and consolidating
the whole software management stack.



How does RPM fit into DNF's view of "the whole software management stack"?
RPM is a slug (moves very slowly): no parallelism (at any point all packages
with no remaining predecessors could be updated/installed in parallel),
not even manually pipelined (decompress to memory, manipulate filesystem,
update database.)


Parallelizing software updates or installations would be *begging* for
pain. It would be difficult for me to recommend strongly enough
against this.


Please be specific about the pain points that you fear.

The three-stage "manual" pipeline achieves 2x to 3x faster throughput
with error states that are isomorphic to present RPM.  (Consider the
Turning machine model: if you don't write to the filesystem, then
there is no change of external state.)

The "parallelize everything that has no remaining predecessors" strategy
requires parallel transactions in the database (they cannot interfere
because that would be a predecessor constraint) and checking for
resource exhaustion (file space, inodes, etc.) as a global
predecessor constraint.  What else?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[Nico Kadel-Garcia]

On Thu, Mar 22, 2018 at 10:52 AM, John Reiser  wrote:
> On 03/22/2018 05:40 AM, Daniel Mach wrote:
>>
>> We are pleased to announce that development of DNF 3 has started. This
>> version is focused on performance improvements, new API and consolidating
>> the whole software management stack.
>
>
> How does RPM fit into DNF's view of "the whole software management stack"?
> RPM is a slug (moves very slowly): no parallelism (at any point all packages
> with no remaining predecessors could be updated/installed in parallel),
> not even manually pipelined (decompress to memory, manipulate filesystem,
> update database.)

Parallelizing software updates or installations would be *begging* for
pain. It would be difficult for me to recommend strongly enough
against this.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Announcing DNF 3 development

[John Reiser]

On 03/22/2018 05:40 AM, Daniel Mach wrote:

We are pleased to announce that development of DNF 3 has started. This version 
is focused on performance improvements, new API and consolidating the whole 
software management stack.


How does RPM fit into DNF's view of "the whole software management stack"?
RPM is a slug (moves very slowly): no parallelism (at any point all packages
with no remaining predecessors could be updated/installed in parallel),
not even manually pipelined (decompress to memory, manipulate filesystem,
update database.)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org