Hi Martin, Let's go over the subsystems in your list specifically:
- Secushare + social + psyc + psycstore + multicast
IMO these should go together. Nothing but
Secushare uses the other 4 yet, and they are all
similarly experimental. As even the SecuShare team
seems unsure about the code base's viability, splitting
this off into its own Git makes some sense to me.
- voting + secretsharing: voting is already in gnunet-java
and separate because the Java build system is completely
different; secretsharing is overall pretty generic
and dependency-wise unproblematic. I really don't see
a good reason for pulling it out of core. If we did,
rps, scalarproduct are in the same category (basically
cryptographic building blocks that are not yet in real
use). But for me, part of the GNUnet approach is that we
need to raise the abstraction of the networking API and
provide better cryptographic building blocks at the heart
of our infrastructure.
- Conversation + speaker + microphone / fs (+datastore)
How do you propose to build gnunet-gtk with this being
moved out of core?
Do you really want anyone working on this to first
build gnunet-core, then gnunet-conversation, then
gnunet-gtk-core, and then gnunet-conversation-gtk?
If we merged gnunet-gtk-core into gnunet-core, that
would at least be semi-sane, but as proposed, this seems
just insane.
- pt (not 100% sure what it does maybe it can stay in code
depending on how niche the API is)
Well, PT (protocol translation, IPv4/IPv6) works with
DNS, VPN and EXIT to translate DNS replies (think DNS6to4).
As GNS needs VPN for some record types, it is not so trivial
to unbundle VPN/EXIT/PT from core.
This is a good example why it is not easy to unbundle.
- Gtk UI (already in separate repo)
If we atomize gnunet.git as you propose, either the gnunet-gtk
configure.ac must increase drastically in terms of complexity
to test which of the gnunet-atoms is available (welcome to 500
character configure invocations) or we atomize this as well,
which then means you can write an novel for the
installation-from-source handbook on the dependency graph.
- rest
This confuses me the most of your proposal. We have json/,
which requires libjansson, not just rest. And there are DB
adapters (pq/my/sq) which also can (or maybe ought to) handle
JSON data.
More importantly, IMO the various rest-plugins probably
belong with their respective subsystems, and
some of those are in core, like for IDENTITY, the DHT or GNS.
So if REST isn't in core, then we get build hierarchies like this:
gnunet-core (with GNS)
gnunet-gtk
gnunet-rest
gnunet-dht-rest
gnunet-identity-rest
gnunet-conversation
gnunet-conversation-rest
gnunet-conversation-gtk
gnunet-fs-rest
gnunet-fs-gtk
Is this really what you believe will attract new developers!?
Dozens of repositories with crazy dependency chains? I also do
not quite see whatever you want to do with the GitLab CI that
could not properly handle the bigger repo we have today
(especially given that Buildbot is today fine with it)
- reclaim (not in that architecture figure because its outdated)
--- some of it is already separate, right? Still, I don't quite
see why you want to split it off. The result would just be that
when I change stuff in GNS and break reclaim, I couldn't even see
I broke reclaim and you would get to pick up my mess. Ok, you
sometimes get to do that anyway, but I predict this would increase
the frequency ;-).
So here, I would rather see libgabe being dragged *into* our
build (src/gabe/) than reclaim being pushed out. After all,
ABE might be useful for other things in the future (there is
a feature proposal for Taler related to ABE, for example).
So from this list, the one thing that today for me would make sense to
move to another repo is the Secushare family. In terms of sanity when
building from source (and that *is* what developers do) I'd rather
integrate gabe than split off reclaim.
Note again that this has nothing to do with the binary package
structure, this is just for sanity when building from source. The
proposed structure that might end up with 50+ interdependent Gits would
drive *me* bananas, and I can't see it being helpful for downstream either.
But most importantly, I simply don't share your aversion to *optional*
dependencies (libgabe, libgstreamer, libpq, libmysql, libjansson). None
of these really justifies a change in our *Git* structure to me.
For me, any Git structure imposes two main constraints: (1) access
control (who has commit rights) --- and there we have so far never even
considered giving some devs access to some repos but not others. (2) is
that the resulting TGZs usually are 1:1 per source Git. Here, we DO have
some people building from source (downstream, students, developers, me),
and each additional TGZ/repo they need to download, configure and build
takes time without being inherently helpful for anything.
Sure, if libgabe was NOT an optional dependency and I had to install it
first, that would be a pain if I don't care about reclaim, but as long
as it is optional, I really don't see any pain for users to have reclaim
in the main tree. OTOH, it will take longer to install from source if
reclaim is a separate repo -- and it'll be easier to install reclaim
from source if libgabe is integrated.
But central to the entire argument seems to be that for CI, I don't at
all see the benefit of more repos either. It just means that the CI
configuration becomes also more complex, as I now have to teach the CI
the repo and dependency structure. Sure, we get in theory a
combinatorial explosion of feature on/off states, but in practice
building once with Postgres and once without Postgres doesn't mean we
have to also try this with all _other_ possible on/off switches as the
features are usually sufficiently orthogonal.
Anyway, I should get back to actual coding and cleaning, maybe we should
resume this discussion at the GNUnet Hacker Meeting? At least I suspect
that might be more productive and I don't see any urgency here.
Happy hacking!
Christian
On 2/9/19 11:16 AM, Schanzenbach, Martin wrote:
> Well I proposed that we look at https://stage.gnunet.org/en/architecture.html
> (which is not up to date currently).
> IMO you can quickly identify which components are basically just applications
> that do not provide fundamental functionality (ie. what I referred to as
> "gnunet core"-- Note: this has nothing to do with the GNUnet core service).
>
> Example:
>
> - Secushare + social
> - Conversation + speaker + microphone
> - psyc + psycstore
> - voting + secretsharing
> - fs (+datastore)
> - pt (not 100% sure what it does maybe it can stay in code depending on how
> niche the API is)
> - reclaim (not in that architecture figure because its outdated)
> - rest
> - Gtk UI (already in separate repo)
>
> I would argue that each of the above _could_ go into a separate repo for 2
> major reasons:
>
> 1. In the overwhelming case, they do not provide core functions to other
> services/apps (they do not have "incoming" arrows in the figure)
> 2. Often, they pull in unreasonable dependencies not required by any other
> component (rest: jansson, reclaim: libgabe, conversation: gstreamer etc)
>
> As soon as a component does not fit into any of the above categories, it
> could be merged into the "core" repo again.
>
> BR
>
>> On 8. Feb 2019, at 15:53, t3sserakt <t...@posteo.de> wrote:
>>
>> Hey *,
>>
>> I also think it is better to have several repos. I can not tell how to split
>> up the gnunet.git repo, but we should not merge gnunet-gtk.git into
>> gnunet.git.
>>
>> cheers
>>
>> t3sserakt
>>
>> On 08.02.19 15:00, Schanzenbach, Martin wrote:
>>> Yes, I do not think this is a good idea at all and is contrary to the
>>> initial motivation of this thread.
>>>
>>> We already agree the from a user perspective, the packages (.deb/.rpm et
>>> al) should ideally be split into
>>> the respective services/applications and, of course, also Gtk+. For sane
>>> dependency resolution at least.
>>>
>>> But it is also reasonable to separate things at source level as I already
>>> gave various reasons, to which I have not heard a counterargument yet
>>> except:
>>> Usability (???).
>>> You cannot argue with usability because USERS DO NOT INSTALL FROM THE GIT
>>> REPO THEY INSTALL PACKAGES.
>>> And even the packages should be separate as you already agreed!
>>>
>>> A monolith _will_ bite us when it comes to testing and CI.
>>> Working on a single, huge codebase with a variety of build switches is a
>>> pain for testing, development and deployment.
>>> Not to mention it is difficult to ascertain and ensure for an application
>>> what components are built.
>>> Example: Do you really want to test everthing of the core gnunet functions
>>> if a Gtk widget changes?
>>> Because that will inevitably happen.
>>> It will be really difficult to setup a CI/automated testing that correctly
>>> separates this.
>>> It will be possible, maybe, but then we have a test process that is equally
>>> difficult as our build process.
>>>
>>>
>>>
>>>> On 8. Feb 2019, at 14:39, Christian Grothoff <christ...@grothoff.org>
>>>> wrote:
>>>>
>>>> On 2/7/19 3:21 PM, Hartmut Goebel wrote:
>>>>
>>>>> Am 02.02.19 um 16:09 schrieb Christian Grothoff:
>>>>>
>>>>>> And I wonder if it wouldn't make sense to have the gnunet.git
>>>>>> configure.ac test for Gtk+ and *if* libgtk is detected, _then_ build Gtk
>>>>>> GUIs that are _included_ in gnunet.git, instead of requiring the user to
>>>>>> download and configure yet another TGZ.
>>>>>>
>>>>> *If* the gui is merged into the main repo, I suggest adding
>>>>> configure-options like `--without-gui`(which AFAIK is a autotools
>>>>> standard thing) to avoid building the gui even if libgtk is detected.
>>>>> This might happen if e.g. one is developing on her/his desktop.
>>>>>
>>>> Sure, that makes sense. Any opinions from the silent masses on merging
>>>> gnunet-gtk.git into gnunet.git and merging the source TGZs?
>>>>
>>>>
>>>> _______________________________________________
>>>> GNUnet-developers mailing list
>>>>
>>>> GNUnet-developers@gnu.org
>>>> https://lists.gnu.org/mailman/listinfo/gnunet-developers
>>>
>>>
>>> _______________________________________________
>>> GNUnet-developers mailing list
>>>
>>> GNUnet-developers@gnu.org
>>> https://lists.gnu.org/mailman/listinfo/gnunet-developers
>
>
> _______________________________________________
> GNUnet-developers mailing list
> GNUnet-developers@gnu.org
> https://lists.gnu.org/mailman/listinfo/gnunet-developers
>
signature.asc
Description: OpenPGP digital signature
_______________________________________________ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
