Re: convention on listen port local or all network interfaces etc. - revision 2

[Holger Levsen]

Hi Patrick,

On Sun, Mar 26, 2017 at 11:22:00AM +, Patrick Schleizer wrote:
> A convention on listen port local or all network interfaces etc. would
> be desirable.

I agree, though I'm not sure we can get consensus on this…

However, I also think that you should restart this discussion once stretch has
been released. Now it's not the best time to have such discussions…

> Any questions? Any suggestions? What do you think?
> 
> Feedback on the ticket [0] and pull requests on the repository [1] are
> welcome.
 
(I haven't looked at neither…) but I'd suggest you file a bug against the
"general" pseudo-package in the Debian BTS as that's a more proper place to
discuss changes in Debian.

Thanks!


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: convention on listen port local or all network interfaces etc. - revision 2

[Patrick Schleizer]

A convention on listen port local or all network interfaces etc. would
be desirable.

At the moment it looks like there is no convention for where server
applications are configured to listen by default, on localhost vs. all
interfaces. Looks like deciding that is up to the upstream author of the
software as well as the packager. Then it's up to the system
administrator to decide on where the server application should listen.
There is no great place for derivatives to globally modify this setting.

Usually applications using Tor ephemeral hidden services such as
ricochet-im, onionshare, ZeroNet, unMessage listen on localhost only.

Whonix is a Debian derivative with focus on anonymity, privacy and
security. To oversimplify it, we preconfigure Debian with these goals in
mind.

Due to Whonix's workstation, gateway split design, applications using
Tor ephemeral hidden services need to listen on the workstation's
external interface rather than on the workstation's localhost.

A global configuration file such as `/etc/ricochet-im.conf` works for
system administrators, but not for derivatives.

Why is a config folder `/etc/ricochet-im.d` strongly preferred over a
config file `/etc/ricochet-im.conf`? When a config file such as
`/etc/ricochet-im.conf` is owned by one package `ricochet-im`, it cannot
be owned by another package. If another package was to modify it using
`sed` or so, then dpkg would regard that file as user modified. The
problem is, next time that config file is changed by upstream, this
throws an interactive dpkg conflict resolution dialog at the user, which
is a usability bug. (example [2]) `sed` style config modifications are a
Debian policy violation as well.

So far we at Whonix had discussions with ricochet-im, onionshare,
ZeroNet and unMessage. They are all interested to make their
applications compatible with Whonix. However, asking each individual
project to `/etc/application-specific.d` folder where Whonix then could
drop a `/etc/application-specific.d/30_whonix.conf` that says
`listen=10.152.152.10` is a lot duplicate effort and not that desirable
for these applications because they have not yet any need for
`/etc/application-specific.d/`.

Having these applications auto detect Whonix also does not seem like
great solution. Seems unsafe. If the auto detection code kicks in as a
false positive, users would be at risk. Since it's Whonix specific and
general solutions reusable by anyone are to be preferred. At least that
is my interpretation of *nix philosophy.

May the following convention be suggested.

* Parse in lexical order.
** `/usr/lib/listen.d`
** `/usr/local/etc/listen.d`
** `/etc/listen.d`
** Optionally, custom folders
** Similar to how systemd would parse these folders. I.e. for example
start with parsing `/usr/lib/listen.d/30_default.conf`, followed by
`/usr/lib/listen.d/31_other.conf`, followed by
`/usr/local/etc/listen.d/30_user.conf`, followed by
`/etc/listen.d/30_user.conf`, followed by `/etc/listen.d/40_user.conf`
and so forth.
* The file used to group the configurations would be `/etc/listen.conf`,
with the contents:


```
# lines starting with # are ignored

# predefined
include /usr/lib/listen.d/*.conf
include /usr/local/etc/listen.d/*.conf
include /etc/listen.d/*.conf

# custom configurations may be added here
```

** In theory, any path could be added to this file, even within `$HOME`.
However, we are still uncertain of the usefulness of adding
configurations to `$HOME` instead of one of the ones within `/usr/` and
`/etc/`. It would make it possible to make customizations when the user
does not have privileges to write outside of `$HOME`, but running a
server on a system which you do not have such access seems unlikely.
Feedback on this is also appreciated.

* `.conf` files would contain options such as:

```
# lines starting with # are ignored

# global fallback setting for all listeners
listen_ip=127.0.0.1
listen_ip=10.152.152.10
listen_ip=0.0.0.0
listen_ip=UNIX-LISTEN:/path/to/.sock
listen_ip=eth0

# web interfaces
listen_ip_web=127.0.0.1
listen_ip_web=10.152.152.10
listen_ip_web=0.0.0.0
listen_ip_web=UNIX-LISTEN:/path/to/_web.sock
listen_ip_web=eth0

# listen incoming IP
listen_ip_incoming=127.0.0.1
listen_ip_incoming=10.152.152.10
listen_ip_incoming=0.0.0.0
listen_ip_incoming=UNIX-LISTEN:/path/to/_incoming.sock
listen_ip_incoming=eth0

# optional application specific listen port
listen_port_=15000

listen_range_=16000-17000
```

For example...

```
# /etc/listen.d/30_default.conf
listen_ip=0.0.0.0

# /etc/listen.d/50_user.conf
listen_ip=127.0.0.1
```

Would mean listen on `0.0.0.0` as well as on `127.0.0.1`.

This disable listeners by previous lower priority configuration files,
one could use `listen_ip=`. For example:

```
# /etc/listen.d/30_default.conf
listen_ip=0.0.0.0

# /etc/listen.d/50_user.conf
listen_ip=
listen_ip=127.0.0.1
```

Would result in listening on `127.0.0.1` only. This is similar to how
systemd parses systemd unit files.

To prevent different 

Re: convention on listen port local or all network interfaces etc.

[Tomas Pospisek]

Am 23.02.2017 um 03:26 schrieb Patrick Schleizer:
> Tomas Pospisek:
>> Am 21.02.2017 um 01:55 schrieb Patrick Schleizer:
>>
>>> for file_name in /usr/lib/server-config.d/*.conf ; do
>>>file_list="$file_list $file_name"
>>> done
>>>
>>> for file_name in /etc/server-config.d/*.conf ; do
>>>file_list="$file_list $file_name"
>>> done
>>>
>>> for file_name in /home/.config/server-config.d/*.conf ; do
>>>file_list="$file_list $file_name"
>>> done
>>>
>>> for item in $file_list ; do
>>>source "$item"
>>> done
>>
>> I like this in principle. However, I'd rather make stuff explicit than
>> implicit. Implicit means you need to have a priory knowledge, explicit
>> means you see stuff. So for the above that would be:
>>
>> $ cat /etc/server/main.config
>> ...
>> # predefined
>> include /usr/lib/server/config.d/*.conf
>> # system config
>> include /etc/server/config.d/*.conf
>>
>> *t
> 
> That is a great idea! Will try to keep explicit vs implicit in mind.
> Explicit is great. I'll be updating this convention proposal when no
> more comments come in.
> 
> Would you suggest just a single file /etc/server/main.config?
> 
> Or should there be also:
> 
> - /usr/lib/server/main.config
> - /etc/server/main.config
> - /home/.config/server/main.config
> 
> ?
> 
> Should main.config file[s] only contain `include` statements and
> comments and nothing else?

I'd suggest to use principles as orientation lights, but not as dogmas.

So f.ex. if you have some config file that is not expected to be
modified much (see /etc/* - most of the files there I never touch but,
in good Debian tradition, it's good *to be able* to customize stuff, if
need arises). So the configs that should work in most environments out
of the box could be in one piece - and contain the includes and maybe a
few other things.

On the other hand if you have configs that are pages and pages long
(which converges towards unreadable), or are expected to be extended or
customized heavily - apache comes to mind for both characteristics -
then you probably want to split the config into as orthogonal and
topical pieces as useful. Again, you can see the evolution of this
principle f.ex. with apache which started as a single config file and
today has quite a nice separation of concerns into different subdir.d's.

There are other examples of the latter principle f.ex. in Debian's
nginx' layout, which I consider very well done, or Debian's exim config,
which I consider non-understandable without a nuclear physics degree and
multiple years of study. (I am also the kind of person who thinks that
maybe systemd has surpassed exim in that regard and entered a whole new
dimension, maybe of a string theory quality, a place that only sendmail
enlighteneds are reported to have seen before. I have not much studied
selinux yet though, which has been created by some of the brightest
cryptographers of our time I hear).

Have a look around in /etc on desktop machines and on servers and have a
look at those configs that are ugly, uncomprehensible or beautiful and
intuitive and try to determine what the reasons for the beauty or
ugliness are.

*t

Re: convention on listen port local or all network interfaces etc.

[Patrick Schleizer]

Tomas Pospisek:
> Am 21.02.2017 um 01:55 schrieb Patrick Schleizer:
> 
>> for file_name in /usr/lib/server-config.d/*.conf ; do
>>file_list="$file_list $file_name"
>> done
>>
>> for file_name in /etc/server-config.d/*.conf ; do
>>file_list="$file_list $file_name"
>> done
>>
>> for file_name in /home/.config/server-config.d/*.conf ; do
>>file_list="$file_list $file_name"
>> done
>>
>> for item in $file_list ; do
>>source "$item"
>> done
> 
> I like this in principle. However, I'd rather make stuff explicit than
> implicit. Implicit means you need to have a priory knowledge, explicit
> means you see stuff. So for the above that would be:
> 
> $ cat /etc/server/main.config
> ...
> # predefined
> include /usr/lib/server/config.d/*.conf
> # system config
> include /etc/server/config.d/*.conf
> 
> *t

That is a great idea! Will try to keep explicit vs implicit in mind.
Explicit is great. I'll be updating this convention proposal when no
more comments come in.

Would you suggest just a single file /etc/server/main.config?

Or should there be also:

- /usr/lib/server/main.config
- /etc/server/main.config
- /home/.config/server/main.config

?

Should main.config file[s] only contain `include` statements and
comments and nothing else?

Best regards,
Patrick

Re: convention on listen port local or all network interfaces etc.

[Patrick Schleizer]

Marco d'Itri:
>> So far we at Whonix had discussions with ricochet-im, onionshare,
>> ZeroNet and unMessage. They are all interested to make their
>> applications compatible with Whonix. However, asking each individual
>> project to `/etc/application-specific.d` folder where Whonix then could
>> drop a `/etc/application-specific.d/30_whonix.conf` that says
>> `listen=10.152.152.10` is a lot duplicate effort and not that desirable
>> for these applications because they have not yet any need for
>> `/etc/application-specific.d/`.
> Indeed. This can easily be solved in a general way by implementing 
> systemd socket activation in these daemons.
> This way you will be able to drop files in /etc/systemd/system/ to make 
> them bind to a specific address or interface.

ricochet-im, onionshare and unMessage aren't daemons. Not comparable to
to daemons like apache that get started by the init system. These are
applications to be started by the user on demand. These are more like
Pidgin / XChat etc. Therefore socket activation may not be the best tool
here?

Best regards,
Patrick

Re: convention on listen port local or all network interfaces etc.

[Marco d'Itri]

On Feb 21, Patrick Schleizer  wrote:

> At the moment it looks like there is no convention for where server
> applications are configured to listen by default, on localhost vs. all
> interfaces. Looks like deciding that is up to the upstream author of the
> software as well as the packager. Then it's up to the system
As it should be: different packages reasonably have different purposes 
and so different defaults to be as useful as possibile when installed.

> So far we at Whonix had discussions with ricochet-im, onionshare,
> ZeroNet and unMessage. They are all interested to make their
> applications compatible with Whonix. However, asking each individual
> project to `/etc/application-specific.d` folder where Whonix then could
> drop a `/etc/application-specific.d/30_whonix.conf` that says
> `listen=10.152.152.10` is a lot duplicate effort and not that desirable
> for these applications because they have not yet any need for
> `/etc/application-specific.d/`.
Indeed. This can easily be solved in a general way by implementing 
systemd socket activation in these daemons.
This way you will be able to drop files in /etc/systemd/system/ to make 
them bind to a specific address or interface.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: convention on listen port local or all network interfaces etc.

[Tomas Pospisek]

Am 21.02.2017 um 01:55 schrieb Patrick Schleizer:

> for file_name in /usr/lib/server-config.d/*.conf ; do
>file_list="$file_list $file_name"
> done
> 
> for file_name in /etc/server-config.d/*.conf ; do
>file_list="$file_list $file_name"
> done
> 
> for file_name in /home/.config/server-config.d/*.conf ; do
>file_list="$file_list $file_name"
> done
> 
> for item in $file_list ; do
>source "$item"
> done

I like this in principle. However, I'd rather make stuff explicit than
implicit. Implicit means you need to have a priory knowledge, explicit
means you see stuff. So for the above that would be:

$ cat /etc/server/main.config
...
# predefined
include /usr/lib/server/config.d/*.conf
# system config
include /etc/server/config.d/*.conf

*t