Re: Summary: the state of the karaf debian package

[Steinar Bang]

I have posted an update to the RFP bug describing the latest
fixes/improvements made to the karaf debian package:
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=881297#40

The following fixes/improvements have been made:
 1. I've run lintian with the following argument and fixed all of the
complaints:
 lintian -vIiE --pedantic --color=auto karaf*.changes 
karaf_4.1.4-10~9.30_all.deb

 2. I've corrected the ownership of /etc/karaf:
- The ownership has been changed from karaf.karaf to root.karaf
- The mode has been set to 2770 on /etc/karaf/ itself (karaf,
  running as user karaf, must be allowed to create new .cfg files in
  this directory)
- Set ownership root.karaf and mode 2750 on all files in /etc/karaf/
  except for the .cfg files and config.properties
- Set ownership karaf.karaf and mode 2770 on all of the .cfg files
  and config.properties, because karaf needs to be able to write to
  them

 3. Moved KARAF_HOME to /var/lib/karaf and let /usr/share/karaf/ have
ownership root.root.
$KARAF_HOME/bin, $KARAF_HOME/lib and $KARAF_HOME/system are all
symlinks to /usr/share/karaf/

 4. Added SYSV init.d files
Note: these are not extensively tested, because the only way I had
to test them was to build .deb packages with the systemd stuff
removed and try to install them and see that the daemon started, and
that was a lot of work

 5. Moved karaf.log from $KARAF_DATA/log/ to /var/log/karaf/

 6. Bumped the upstream karaf version from 4.1.4 to 4.1.5

Re: Summary: the state of the karaf debian package

[Steinar Bang]

> Steinar Bang :

> I believe the following steps are needed to make this an official
> package:
>  1. Replace the remaining non-karaf boot jars with debian packages,
> these packages are:
> a. apache felix framework 5.6.10
>- Already a debian package, but currently in version 4.6.12 in
>  stretch, testing and unstable [3]

I've filed a bug to have this updated:
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=890143

> b. apache felix metatype 1.1.6 [4]

I've filed an RFP for this package:
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=890144

> c. apache felix configadmin 1.8.16 [5]

I've filed an RFP for this package:
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=890145

> d. apache felix file install 3.6.4 [6]

I've filed an RFP for this package:
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=890146

> e. OPS4J PAX URL Aether 2.5.3 [7]

I've filed an RFP for the parentt projec for this package:
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=890181

> f. OPS4J PAX Logging API 1.10.1 [8]
> g. OPS4J PAX Logging Log4J2 1.10.1 [9]

I've filed an RFP for the parent project for these two packages:
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=890182

Re: Summary: the state of the karaf debian package

[Thorsten Glaser]

On Fri, 2 Feb 2018, Steinar Bang wrote:

> >> 6. A karaf service is added to systemd, running as user karaf and
> 
> > Does it work without systemd?
> 
> Hm... do you mean being started from the command line, or with SYSV
> script instead of systemd?

The latter.

> I don't much like systemd myself.  But I just figured "if there are two
> alternatives, pick one, and pick the one most likely to be used in the
> future".

No, packages should support all init systems if feasible.

> Switching to SYSV wouldn't be hard (copy the SYSV scripts from the
> source tarball and remove the disabing of SYSV maintscript generation,
> and remove the use of systemd).

You should not “switch”, just add an appropriate init script
(which, with Debian mechanisms, is easy… sometimes it’s even
easier if upstream already supplies one, sometimes that makes
it harder though because those can be broken) and let dh
install it to the right place. Then, the correct (init script
or unit file) will be used depending on what the user boots
with.

> What's preferred from a debian viewpoint? SYSV? Or systemd?

There’s no preference, only a default for new installations.

> > /etc/karaf should be root-owned. If it contains secrets,
> > then root:karaf 2750, otherwise root:root 755.
> 
> The directory has to be karaf-writable, because karaf allows
> configuration changes from the karaf console command line as well as
> from the API.

OK.

> > /usr/share/karaf MUST be root-owned, it is NOT writable
> > during normal system operation (admins MAY mount /usr as
> > read-only filesystem).
> 
> Hm... that's harder.  Karaf creates stuff under $KARAF_HOME/instances/
> 
> One way of doing it would be to switch KARAF_HOME to /var/lib/karaf/

Yes, that’s what Tomcat does.

> But there are stuff in KARAF_HOME that are read-only: the jar files (or
> symlinks to jar files) in $KARAF_HOME/lib and $KARAF_HOME/system.

Tomcat places symlinks under /var pointing to the directories
under /usr, and it has a clear distinction, some directories
are read-only, some read-write, but not a mix.

So if Karaf never writes to $KARAF_HOME/lib and $KARAF_HOME/system
during runtime, you can install them to /usr/share/karaf and put
symlinks /var/lib/karaf/lib@ -> /usr/share/karaf/lib (and similar
for …/system).

> Would it be proper to install these under /var/lib/karaf/?  Or would it

It would not be “proper” (/usr is intended for shareable data)
but if the above cannot work it’s completely acceptable to do that.

bye,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

Re: Summary: the state of the karaf debian package

[Steinar Bang]

> Thorsten Glaser :

> Hi Steinar,
> congratulations on getting this far, that’s a nontrivial
> amount of work!

Thanks! :-)

> On Thu, 1 Feb 2018, Steinar Bang wrote:

>> 4. lintian no longer gives any warnings on the package (I have only run
>> lintian with "lintian karaf_xxx.deb", I haven't tried with any
>> arguments)

> You want "lintian -vIiE --pedantic --color=auto karaf*.changes" normally.

Ok, I will try that.

>> 6. A karaf service is added to systemd, running as user karaf and

> Does it work without systemd?

Hm... do you mean being started from the command line, or with SYSV
script instead of systemd?

Karaf by itself works fine without being started by systemd or SYSV init
for that matter.  Karaf can be started from the command line and result
in the same console one sees when SSHing in.

But the way I've packaged it the start from command line behaviour isn't
easily accessible (and also easily combined with the daemon behaviour,
ie. diretories owned by user karaf.  A karaf started from the command
line is typically used for development and run as the developers user).

I don't much like systemd myself.  But I just figured "if there are two
alternatives, pick one, and pick the one most likely to be used in the
future".

Switching to SYSV wouldn't be hard (copy the SYSV scripts from the
source tarball and remove the disabing of SYSV maintscript generation,
and remove the use of systemd).

What's preferred from a debian viewpoint? SYSV? Or systemd?

>> logging to /var/log/syslog and using the following directories:
>> KARAF_ETC = /etc/karaf/
>> KARAF_HOME = /usr/share/karaf/
>> KARAF_DATA = /var/lib/karaf/data/
>> all directories are owned by user karaf, group karaf

> /etc/karaf should be root-owned. If it contains secrets,
> then root:karaf 2750, otherwise root:root 755.

The directory has to be karaf-writable, because karaf allows
configuration changes from the karaf console command line as well as
from the API.

There is one file that contains secrets (and that file does not have to
be writable by karaf, I think.  But it has to be readable by karaf).

Applications may create their own configuration files in this directory,
so karaf needs to be allowed to create new files here.

> /usr/share/karaf MUST be root-owned, it is NOT writable
> during normal system operation (admins MAY mount /usr as
> read-only filesystem).

Hm... that's harder.  Karaf creates stuff under $KARAF_HOME/instances/

One way of doing it would be to switch KARAF_HOME to /var/lib/karaf/

But there are stuff in KARAF_HOME that are read-only: the jar files (or
symlinks to jar files) in $KARAF_HOME/lib and $KARAF_HOME/system.

Would it be proper to install these under /var/lib/karaf/?  Or would it
be best to leave them where they are, under /usr/share/karaf but to
create symlinks to them from /var/lib/karaf/ ?

>> IMO I'm not sure if this is worth the effort, because the karaf jar
>> files aren't really meaningful outside of karaf, or in a version
>> independent manner

> I agree. So I guess it boils down to packaging all libraries
> used to run karaf so it doesn’t use anything that’s not in
> Debian (otherwise, it’s at best usable in the contrib suite).

Ok on that. :-)

>> Thanks in advance to anyone who will create an official debian package,
>> whether it is based on my package or not! :-)

> Why not you, as part of the team, getting help from other
> team members, and someone sponsoring your uploads? You’re
> a user, so you can actually test changes and new versions;
> the rest is just mentoring you and doesn’t use it themselves.

Ok, I will try to create an official state package, when/if the
dependencies becomes available (and maintain it going forward, if noone
else wants to).

> Also: I’m impressed with the amount of work you did to
> document this all! Hats off.

Thanks! :-)

It was actually quite fun.  And also easier to use the standard debian
tools than fpm that was used to package the package I found and forked
when googling for "karaf debian package", back in November 2016.

(I guess it helped that I was familiar with make and GNUMakefile syntax
so that the debian/rules file syntax wasn't a mystery).

Re: Summary: the state of the karaf debian package

[Thorsten Glaser]

Hi Steinar,

congratulations on getting this far, that’s a nontrivial
amount of work!

On Thu, 1 Feb 2018, Steinar Bang wrote:

>  4. lintian no longer gives any warnings on the package (I have only run
> lintian with "lintian karaf_xxx.deb", I haven't tried with any
> arguments)

You want "lintian -vIiE --pedantic --color=auto karaf*.changes" normally.

>  6. A karaf service is added to systemd, running as user karaf and

Does it work without systemd?

> logging to /var/log/syslog and using the following directories:
>  KARAF_ETC = /etc/karaf/
>  KARAF_HOME = /usr/share/karaf/
>  KARAF_DATA = /var/lib/karaf/data/
> all directories are owned by user karaf, group karaf

/etc/karaf should be root-owned. If it contains secrets,
then root:karaf 2750, otherwise root:root 755.

/usr/share/karaf MUST be root-owned, it is NOT writable
during normal system operation (admins MAY mount /usr as
read-only filesystem).

> IMO I'm not sure if this is worth the effort, because the karaf jar
> files aren't really meaningful outside of karaf, or in a version
> independent manner

I agree. So I guess it boils down to packaging all libraries
used to run karaf so it doesn’t use anything that’s not in
Debian (otherwise, it’s at best usable in the contrib suite).

> Thanks in advance to anyone who will create an official debian package,
> whether it is based on my package or not! :-)

Why not you, as part of the team, getting help from other
team members, and someone sponsoring your uploads? You’re
a user, so you can actually test changes and new versions;
the rest is just mentoring you and doesn’t use it themselves.

Also: I’m impressed with the amount of work you did to
document this all! Hats off.

bye,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

*

**Besuchen Sie uns auf der EuroCIS!**
27. Februar bis 01. März 2018, Messe Düsseldorf / **Halle 10,** ** Stand
F15**
Leading Trade Fair for Retail Technology
[www.tarent.de/eurocis](http://www.tarent.de/eurocis)

Wir empfehlen unsere Vorträge

?Preisbeobachtung, Händlermonitoring, Plagiaterkennung: Ihre
Wettbewerbsvorteile?
am 27. Februar, 14:00 Uhr im EuroCIS Forum / Halle 10, Stand F04

?Internet of Things ? Der Handel im Wandel?
am 01. März, 11:30 Uhr im Omnichannel Forum / Halle 10, Stand A70

*

**Visit us at EuroCIS!** 27th February to 1st March, 2018, Messe
Düsseldorf
/ **Hall 10,** ** Booth F15**
Leading Trade Fair for Retail Technology
[www.tarent.de/eurocis](http://www.tarent.de/eurocis)

We recommend our presentations

?Your view on prices, retailers and plagiarism: Competitive advantages
with
monitoring apps?
on 27th February, 2 pm at EuroCIS Forum / Hall 10, Booth F04

?Internet of Things ? Retailing in a Changing World?
on 1st March, 11:30 am at Omnichannel Forum / Hall 10, Booth A70

*

Re: Summary: the state of the karaf debian package

[Emmanuel Bourg]

Le 01/02/2018 à 20:00, Steinar Bang a écrit :

> Thanks in advance to anyone who will create an official debian package,
> whether it is based on my package or not! :-)

Hi Steinar,

Congratulations for going thus far. Would you be willing to maintain the
karaf package in Debian if someone packages the missing libraries
required to make it an official package?

Emmanuel Bourg

Summary: the state of the karaf debian package

[Steinar Bang]

My karaf debian package[1] is now in a state where it does what I need
for my own use.  I'm serving it from my own unofficial APT repository
(which I will discontinue if the RFP[2] results in an official karaf
package).

I don't plan to do any more work on this packaging, except what is
needed to make new stable karaf releases run (ie. I will roll a new
version of the package when karaf 4.1.5 is out).

I'm leaving some notes on the current state of the packaging here
(ie. in the mailing list archives) in case someone wants to use this
package as a starter for an official package.  I will send a message to
the RFP with the same information.

The current state of the karaf debian package, is:
 1. The package is created using native debian packaging tools

 2. The package is built from the source tarball of karaf releases

 3. The source is built with maven and openjdk-8

 4. lintian no longer gives any warnings on the package (I have only run
lintian with "lintian karaf_xxx.deb", I haven't tried with any
arguments)

 5. The package creates a system user named karaf, with group karaf, and
home directory /var/lib/karaf

 6. A karaf service is added to systemd, running as user karaf and
logging to /var/log/syslog and using the following directories:
 KARAF_ETC = /etc/karaf/
 KARAF_HOME = /usr/share/karaf/
 KARAF_DATA = /var/lib/karaf/data/
all directories are owned by user karaf, group karaf

 7. The karaf package works with "apt-get dist-upgrade", config changes
in KARAF_ETC gets the usual APT dialog on modified config change
(ie. "user the package maintainer's version or the modified file
from the old version?), the karaf cache is flushed during an
upgrade, which means that all installed features are lost and must
be reinstalled, contents in KARAF_ETC not installed by the karaf
package is untouched

 8. Those karaf boot requirements that could be satisfied by current
debian stable (debian 9, stretch) uses debian dependencies:
 - Java 8 RE
 - OSGi core 6
 - libjna-java (not strictly needed, but part of the boot directory)
 - libjansi-java
   - This one can be used as a pattern for how to setup
 startup.properties dependencies, the changes are[3]
 a. Add a debian dependency to the control file
 b. Replace the jansi version number with "debian" in
$KARAF_ETC/startup.properties
 c. Remove jansi from the $KARAF_HOME/system maven repository
 d. Symlink the "debian" version from debian's maven repository
into the $KARAF_HOME/system repository
 
 9. The jar files that make up karaf are structured as a maven
repository under $KARAF_HOME/system

10. The initial boot is done from the jar files found in
$KARAF_HOME/lib/boot/ (I have replaced non-karaf dependencies here
with symlinks to the jar files of debian packages)

11. The jar files needed to boot to a state where karaf can start
downloading dependencies from maven central (or other maven
repositories), are listed in $KARAF_ETC/startup.properties

12. The $KARAF_HOME/system maven repository has been cleaned for files
not built by source that aren't needed for karaf boot.  The
non-karaf jar files currently in there are needed for boot, and
cannot be satisfied by existing debian java packages


I believe the following steps are needed to make this an official
package:
 1. Replace the remaining non-karaf boot jars with debian packages,
these packages are:
a. apache felix framework 5.6.10
   - Already a debian package, but currently in version 4.6.12 in
 stretch, testing and unstable [4]
b. apache felix metatype 1.1.6 [5]
c. apache felix configadmin 1.8.16 [6]
d. apache felix file install 3.6.4 [7]
e. OPS4J PAX URL Aether 2.5.3 [8]
f. OPS4J PAX Logging API 1.10.1 [9]
g. OPS4J PAX Logging Log4J2 1.10.1 [10]
 2. Restructure karaf into two packages, karaf and libkaraf-java, with
libkaraf-java containing the jar files structured like debian java
packages (ie. in /usr/share/maven-repo both with their actual
version and with the "debian" version).
IMO I'm not sure if this is worth the effort, because the karaf jar
files aren't really meaningful outside of karaf, or in a version
independent manner


Thanks in advance to anyone who will create an official debian package,
whether it is based on my package or not! :-)


- Steinar


References:
 [1] 
 [2] 
 [3] 

 [4] 
 [5] 

 [6] 

 [7]